Working MachinesDOCS

OPEN DATA / DISCOVERY SNAPSHOT

Can an agent find the smallest useful action?

This first public snapshot records four real search_actions calls through Working Machines MCP. It publishes the exact prompts, provider filters, ranked action identifiers, manually reviewed minimal action sets, client-observed timing, and the case where the best action did not rank first.

DOWNLOAD RAW JSON READ METHOD

TOP-ONE ACCURACY

75%

RECALL @ 5

100%

MEAN RECIPROCAL RANK

0.83

MEDIAN OBSERVED TIME

125 ms

FOUR TEST CASES

Ranked results, including the miss.

All calls used a provider filter and returned at most five ranked action IDs.

STRIPE / 500 MS OBSERVED

“list products and prices”

Both actions needed for the requested inventory appeared in the first two ranks. Broader search and mutation actions also appeared below them.

FIRST RELEVANT #1P@5 40%

EXPECTED MINIMAL SET

stripe.list_pricesstripe.list_products
  1. 1stripe.list_prices
  2. 2stripe.list_products
  3. 3stripe.search_prices
  4. 4stripe.create_price
  5. 5stripe.search_products

NPM / 125 MS OBSERVED

“get package metadata and versions”

The smallest sufficient action appeared at rank three, so this case failed top-one selection even though the action was present within five results.

FIRST RELEVANT #3P@5 20%

EXPECTED MINIMAL SET

npm.get_package
  1. 1npm.audit_package_versions
  2. 2npm.get_package_version
  3. 3npm.get_package
  4. 4npm.get_version_download_counts
  5. 5npm.get_package_download_trend

HACKERNEWS / 125 MS OBSERVED

“get top stories”

The expected action ranked first. Other story-list variants occupied the remaining results.

FIRST RELEVANT #1P@5 20%

EXPECTED MINIMAL SET

hackernews.get_top_stories
  1. 1hackernews.get_top_stories
  2. 2hackernews.get_new_stories
  3. 3hackernews.get_job_stories
  4. 4hackernews.get_ask_stories
  5. 5hackernews.get_show_stories

CROSSREF / 109 MS OBSERVED

“search scholarly works by title”

The general work search action ranked first. More specialized Crossref actions followed it.

FIRST RELEVANT #1P@5 20%

EXPECTED MINIMAL SET

crossref.list_works
  1. 1crossref.list_works
  2. 2crossref.list_scoped_works
  3. 3crossref.list_changed_works
  4. 4crossref.list_resources
  5. 5crossref.get_work_agency

What this snapshot supports

All four manually expected actions appeared within five results. Three appeared first; the npm case placed the smallest sufficient action third. The result supports only those statements for this dated, provider-filtered pass.

What it does not support

  • This is a four-case discovery snapshot, not a statistically powered benchmark.
  • Each case used a provider filter, so the result does not measure cross-catalog provider selection.
  • Only one pass was recorded; warm and cold conditions were not separated.
  • Gold sets were manually reviewed and can be challenged or revised with a dated correction.
  • Elapsed time includes client, network, and runtime overhead and should not be treated as a service-level objective.
  • No provider action was executed as part of this discovery dataset.