An open-source web retrieval MCP server that fetches, crawls, and searches the web, returning clean markdown for AI agents. It integrates with Claude MCP, LangChain, and other frameworks for agentic web access.
Claim it to get a verified publisher badge, a free copy of our full audit findings, and direct contact for any high-priority issues we find.
Install from
M8ven verifies MCPs across every public registry — install directly from whichever one you prefer.
process.env. You'll be asked to provide them before it can run.REDIS_URL— Redis connection for caching + job queueSEARXNG_URL— SearXNG instance for search (falls back to DuckDuckGo + Google + Bing)ANTHROPIC_API_KEY— For Claude-powered agent_extractANTHROPIC_MODEL— claude-3-haiku-20240307 Claude model name for extractionOLLAMA_URL— Ollama endpoint for local LLM extractionAGENTFETCH_CACHE_TTL— 300 In-memory LRU cache TTL (seconds)OLLAMA_MODEL— llama3.2 Ollama model nameAGENTFETCH_ROBOTS_CHECK— false Enable robots.txt complianceAGENTFETCH_CRAWL_DB— agentfetch_crawl.db SQLite path for crawl job persistenceAGENTFETCH_PROXY_LIST— Comma-separated proxy URLs or JSON arrayAGENTFETCH_PROXY_STRATEGY— round-robin round-robin or randomAGENTFETCH_STATIC_TIMEOUT— 15 HTTP fetch timeout (seconds)AGENTFETCH_BROWSER_TIMEOUT— 30 Playwright browser timeout (seconds)AGENTFETCH_COOKIES_FILE— Path to cookies file (Netscape or JSON)AGENTFETCH_MAX_RETRIES— 2 Max retries for failed requestsAGENTFETCH_DOMAIN_DELAY— 0.5 Delay between requests to same domainAGENTFETCH_CACHE_SIZE— 100 Max entries in in-memory LRU cacheAGENTFETCH_MIN_PROSE_RATIO— 0.4 Minimum alpha-char ratio for quality checkAGENTFETCH_MIN_WORDS— 10 Minimum word count for quality checkAGENTFETCH_JA3_PROFILE— JA3 TLS profile override for curl_cffiAGENTFETCH_STEALTH— true Enable browser stealth evasions in PlaywrightAGENTFETCH_STEALTH_BASIC_FALLBACK— true Fall back to non-stealth browser if stealth failsBRAVE_SEARCH_API_KEY— Brave Search API key (enables brave engine, preferred over DuckDuckGo)SERPAPI_KEY— SerpAPI key (enables serpapi engine, preferred over Google scraping)GOOGLE_API_KEY— Google Custom Search API key (used by google engine when both key and CX are set)GOOGLE_CX— Google Custom Search CX (required with GOOGLE_API_KEY)AGENTFETCH_SEARCH_RETRIESAGENTFETCH_SEARCH_TIMEOUT[](https://m8ven.ai/mcp/sid1art-agentfetch-181krs)