Your Agent Won't Get an Official API. We Checked 425 Sites.

Singapore's government tender portal lists every open contract. Its official data feed lists only the ones already awarded.

That gap is the whole problem with waiting for official APIs.

I'm Lewis Tham. I'm first author of "Internal APIs Are All You Need" (arXiv:2604.00694), and I run Unbrowse, which turns the requests websites already make into tools agents can call. Every week someone asks me which sites "have an API". So we counted.

Six in ten sites have no API at all

We took every site in Unbrowse's public catalog on 5 October 2026: 425 websites where an agent already has a published, verified capability. Most of those capabilities are searches of the site's own listings, records or catalog.

For each site we checked two things. Does the operator offer an official API? And if so, does it cover the task agents actually do there?

425  sites agents already work with
──────────────────────────────────────────────
253  no official API of any kind
 53  official API for approved partners only
 63  self-serve API that misses the task
──────────────────────────────────────────────
 56  self-serve API that covers the task

Your agent needed an API on 425 sites. Sign-up-and-go covered 56. That is 13%.

Count partner programs too, and the answer barely moves. On 362 of the 425 sites, 85%, no official API covers the task, whether you can sign up for it or not.

Small sites have nothing. Big sites stop short.

The 253 with nothing are not all hobby blogs. They include Redfin (bulk data downloads only), the U.S. Supreme Court's site, ZDNet, the Chicago Tribune and BBC Good Food. Even the Vue.js docs: the search box is a third-party widget, with no official API behind it.

The 53 partner-only sites include some of the biggest names. Airbnb's API is closed to approved software providers and covers host listing management, not searching for stays. Craigslist offers a bulk posting interface and nothing for search. Reddit and O'Reilly are in this group too.

The self-serve APIs are where it gets strange. They exist, they are documented, and they skip the thing users do:

Search PyPI by keyword? The API's search endpoint is disabled.

Full-text search SEC filings? The website does it. The documented EDGAR APIs don't.

Search itch.io? The API is for managing your own games.

Search GIFs on Tenor? Google closed the public API to third parties on 2026-06-30.

Government publishes data, not the search

Government is the sector most committed to open data, so it should be the easy case.

Of the 28 government sites in the sample, 19 publish an official API. On 9 of them, that API covers the search people use on the site.

Singapore's GeBIZ is a clean example. The government publishes awarded tenders as an open dataset. There is no API for the open tenders a supplier actually needs to see. For an agent, the task goes back to a human.

The site already has an API

Open almost any modern web app with devtools on and run a search. The page sends a structured request to the company's own backend and gets structured data back. The button you clicked only fires that request.

The SEC is the cleanest case in the sample. Its documented EDGAR APIs cover company submissions and XBRL financial data. Its full-text search page, the feature people actually use, calls a JSON endpoint on efts.sec.gov that is not in that documentation.

The site already ships an API. Its own frontend is the only client it was built for.

The browser gets the full product. Your agent gets whatever someone chose to expose, which on most sites is nothing.

9 in 10 capabilities came from the frontend

You can see the same thing in how Unbrowse's public registry was built. On 12 October 2026 it listed 702 capabilities, and very few came from an interface a site published for machines:

702  capabilities in the public registry (2026-10-12)
──────────────────────────────────────────────
 30  OpenAPI descriptions
 23  OpenSearch descriptors
 15  MCP servers
  4  GraphQL schemas
──────────────────────────────────────────────
 72  from published machine interfaces
630  from the site's own frontend
       260  search forms shown to human visitors
       370  requests the page makes in a browser

You can check this yourself: GET https://unbrowse.ai/api/v1/sites returns the count by source. Limit your agent to what sites formally expose, and about one in ten of these capabilities exists.

Check any site in five minutes

You don't have to take my counts on trust. Pick the site your agent keeps failing on:

Open it in Chrome, then DevTools (F12) → Network → filter Fetch/XHR.
Do the task by hand: run the search, open the record, load the next page.
Click the request that returned the results. Look at Response: if it's JSON with the data you saw on screen, the site has an API. It just isn't documented.
Note what it needs: query parameters, a cookie, a header token. Public listings usually need nothing. Your own account needs your login.
Now open the official developer docs and search for that task. On 362 of our 425 sites, you won't find it.

Nothing appears in step 3? The page renders server-side or the search runs in a third-party widget. That's one of the cases below where learning fails, and the browser stays.

How an agent learns a site

The browser stays. It just stops being the permanent operating mode.

Watch. The agent does the task once in Unbrowse's cloud browser, signed in with the user's saved login if the task needs one, and the request the page sends is captured.
Distill. The route, its inputs and the auth it needs are compiled into a capability.
Call. Later calls replay the route over HTTP, without rendering the page. If the site changes and replay fails, the browser takes over again.

Then share it. A read-only route that needs no secrets is published to a public registry only after a scrubbed copy, with no cookies, answers fresh queries correctly. The next agent does not have to work it out from scratch.

	Official API	Learned frontend route
Covers the task on	56 of 425 sites (self-serve)	Sites whose frontend calls its own backend
Access	Developer program or partner approval	The user's own login and permissions
Scope	What the vendor chose to expose	What the site's own page does
Breaks when	The vendor deprecates it	The frontend changes, so it is re-checked

Delegation, not circumvention

This only works if the agent acts as the user, with the user's permissions, and nothing more. In practice that means accepting limits:

When we crawled the Tranco top 10,000 websites in September 2026, we skipped 474 because their robots.txt asked crawlers to stay out.
A CAPTCHA or multifactor prompt goes to the person the agent works for. It is not solved or bypassed.
Passwords sit in a sealed vault and are filled in by the system. The agent never sees them.
Anything that depends on a login, a private session or a write action is never published to the public registry.

Where this breaks

Not every site can be learned. Some run search inside the page or through a third-party provider, so there is no reusable request. Others use rate limits or bot management that stop a learned call from being replayed. In our September test on 100 sites we had never seen, a generic script got a working replay on 42.

Learned routes decay. Sites change, so public routes are re-checked. In a sample on 26 September 2026, 166 of 200 public tools answered a fresh input (83%), and none returned a false success. A route that fails loudly can fall back to the browser. One that fails quietly does damage, so verification matters as much as discovery.

The sample is not the whole web. These 425 sites are ones where agents already have a working capability, so they lean toward sites with useful search.

The labels came from research agents with web search, and we checked them. A separate agent re-checked 60 random sites blind. It confirmed 34 of 35 "no API" labels (the one miss was an RSS embed service, which our rules exclude) and agreed on task coverage for 57 of 60. In four of its six disagreements on API category, it found less official API than the first pass did. Drop the 53 rows we marked low confidence and the headline holds: 59% have no API, and on 83% no official API covers the task.

What counted as an API: a self-serve documented developer API, an official open-data API or an official MCP server. RSS feeds, OpenSearch descriptors, sitemaps, bulk downloads and device SDKs did not count. Partner-only programs were counted separately.

What we built

Unbrowse does those five steps for your agent, then keeps the result. When an agent learns a site in the course of a task, the read-only, secret-free part of what it learned can go into the public registry, so the next agent starts with it. The first request on a site nobody has learned still runs in the cloud browser; we don't promise a browserless first call.

Plug it into any MCP client at https://unbrowse.ai/mcp, or start from the docs. For why the official-interface plan stalls in the first place, read The Agentic Web Will Not Wait, and for the benchmark behind the paper, the deep dive.

Stop waiting for the official API. The site's frontend already uses one.