MCP gateway model: notes on a better architecture for MCP management

Anyone who's spent time configuring MCP servers knows the pain: JSON files proliferating across every project directory, credentials scattered across tools like confetti after a bad deployment, and the constant nagging question of which servers should be active for which context. I currently have MCP configs in Claude Code globally, per-project, in Claude Desktop, and I've lost track of which credentials live where. It's the kind of situation where you start sketching out alternatives on the back of a napkin during lunch, and that's essentially what the MCP Gateway Model repository represents: my napkin sketches, slightly tidied up.

The two-tier gateway architecture

The core idea is deceptively simple. Instead of configuring individual MCP servers for every tool and every project, you aggregate them into two tiers: a LAN gateway for servers running on your local machine or home network, and a WAN gateway for cloud-hosted MCP endpoints. An intelligent routing layer sits in front of both, dynamically matching requests from AI agents to the appropriate MCP server based on what the agent is actually trying to do. Your AI assistant doesn't need to know that your calendar integration runs locally while your email MCP is cloud-hosted — it just asks for calendar data and the gateway figures out where to send the request. The topology maps naturally onto how most people's tools are already distributed: some things run locally for speed and privacy, some things live in the cloud for accessibility, and right now you're managing both sets manually with no abstraction layer.

The context overhead problem nobody talks about

The benefit that excites me most isn't routing — it's what a gateway does for context window management. Right now, when you attach MCP servers to an AI assistant, every single tool definition from every attached server gets loaded into the context window whether you need it or not. I've counted: my typical Claude Code setup loads around 40-50 tool definitions from various MCPs, and most of them are irrelevant to any given conversation. That's context window space being burned on tool descriptions I won't use, and it's also noise that can confuse the model about which tools are relevant. A well-designed gateway could feed tool definitions on an as-needed basis, using semantic matching to expose only the tools relevant to the current request. The model says it needs to read a calendar event; the gateway exposes just the calendar tools, not the 30 other tools from your other connected servers. This is the kind of optimisation that compounds — every tool definition you don't load is context space that can be used for actual conversation.

The abstraction layer we're missing

The second big win is pure abstraction. End users — and honestly, even power users like me — shouldn't need to think about transport protocols, JSON configurations, or which MCP server handles what. If I want my AI assistant to be able to manage my email, read my calendar, and query my database, I should be able to express that intent once, in a GUI, and have a gateway handle all the plumbing: spinning up servers, managing credentials, restarting crashed processes, and presenting a single clean endpoint to whatever AI client I happen to be using that day. The current state of MCP feels like the early days of the internet, when you needed to know IP addresses and DNS was a novelty. We're missing the equivalent of a browser — a friendly layer that hides the complexity and just lets you use the thing.

This is firmly in the ideation and planning stage. The repository contains notes and voice transcripts describing the architecture rather than working code. I'm sharing it because I think the direction matters and because several projects — from Microsoft's gateway work to MetaMCP to the various MCP aggregators appearing weekly — are converging on similar ideas from different angles. If you're thinking about similar problems, I'd love to compare notes.