Fetch MCP Server: Let AI Read Web Pages

⏱️ 2 min read 🔌 MCP Server

What it is: The reference fetch server (mcp-server-fetch on PyPI), one of the original MCP reference implementations and still actively maintained. It fetches a URL and converts the HTML to markdown before handing it to the model — simple, and one of the most commonly wired-up servers because so many tasks start with "go read this page."

Quick answer: Run uvx mcp-server-fetch with no arguments — it needs no credentials, just outbound network access. It respects robots.txt by default, which is the right default and worth leaving alone unless you have a specific, deliberate reason to override it.

Why it matters for data work

Documentation, API references, a vendor's changelog, a blog post someone linked in Slack explaining a methodology — a huge share of the context an agent needs for data work lives on the open web, not in a database. This server is the simplest bridge: point it at a URL and the agent gets clean, readable markdown back instead of raw HTML it would otherwise have to parse itself.

Install & configure

No credentials needed, which makes this one of the simpler servers to wire up:

# Claude Code
claude mcp add fetch -- uvx mcp-server-fetch

// Claude Desktop / Cursor config
{
  "mcpServers": {
    "fetch": {
      "command": "uvx",
      "args": ["mcp-server-fetch"]
    }
  }
}

That's the entire setup. The agent now has a fetch tool it can call with any URL and an optional max-length/starting-offset for paginating through long pages.

Should it ignore robots.txt?

The server checks robots.txt before fetching a page and can be launched with an --ignore-robots-txt flag to skip that check. Leave it on by default — robots.txt is a site's explicit statement of what it's comfortable with automated tools reading, and respecting it costs you nothing for the vast majority of "go read this documentation page" tasks. Only consider the override for sites you control yourself, and understand it's a deliberate policy exception, not a bug fix.

How do I keep it safe?

The real risk with a fetch tool isn't the server's own behavior — it's that it hands the model text from a source you don't control. Treat every fetched page as untrusted input: a page's content, not just the model's own reasoning, can contain text engineered to look like instructions ("ignore previous instructions and…"). Don't chain a fetch tool with write-capable tools (a database, a filesystem with write access, an email sender) on tasks that fetch attacker-reachable URLs without a human review step in between.

Troubleshooting

This server has a small surface, so issues are usually one of a few things:

Blocked by robots.txt: expected behavior for sites that disallow crawlers — this isn't a bug.
Page returns garbled or truncated content: some sites render content client-side via JavaScript, which this server (a plain HTTP fetch) won't execute — you'll need a headless-browser-based tool for those pages.
uvx: command not found: install uv first (pip install uv or the standalone installer).
Timeouts on slow sites: expected for very large pages or slow servers; ask the agent to fetch a narrower URL if possible.

Author & links

Author: Model Context Protocol project (Anthropic-maintained)

Repo: github.com/modelcontextprotocol/servers

License: MIT

Related skills

Pair this with the Filesystem MCP server to save fetched content locally, or the GitHub MCP server for reading linked repos and issues directly instead of scraping their HTML.

← Back to MCP Servers