Internal APIs Are All You Need

Abstract

Autonomous web agents repeatedly pay a discovery tax: opening sites, inspecting DOMs, and mapping callable routes. We argue that the internal APIs already powering those websites are the machine-native interface agents should target, and that browser-first architectures are the wrong default. Unbrowse turns repeated private reverse-engineering into a shared route graph so agents can call cached first-party routes directly, with a browser retained only when a route does not yet exist.

Claim

Across 94 production sites, every observed task sat on top of a JSON or GraphQL endpoint the front-end was already calling. The bottleneck is not “agents need browsers.” It is “agents lack a shared index of routes and a way to keep it fresh.”

Measured

• 3.6× mean and 5.4× median speedup on warmed cached routes versus Playwright
• 90–96% per-task cost reduction for warmed-cache execution
• Voluntary three-path model: local cache, shared graph, or browser fallback — never browser-only by default
• ~8,000 tokens of DOM reduced to ~200 tokens of structured JSON on typical reads

What Unbrowse ships

A resumable YAML harness, a hosted identity broker, and a remote MCP so any agent can use first-party APIs without a human driving a browser. The first request is fulfilled while routes are passively indexed. Publication as a trusted tool requires validation; HTTP 200 is not a business outcome.