---
title: "Open Data Wasn't Missing. The Interface Was."
description: "A 12-domain benchmark and Sentinel-2 experiment show where the Registry of Open Data MCP server works, where search fails, and what the interface unlocks."
canonical_url: "https://artificialcuriositylabs.ai/posts/open-data-wasnt-missing-interface-was/"
md_url: "https://artificialcuriositylabs.ai/posts/open-data-wasnt-missing-interface-was.md"
published_at: "2026-07-29T00:00:00.000Z"
tags:
  - "mcp"
  - "agents"
  - "infrastructure"
  - "patterns"
---

Open data has an interface problem.

The [Registry of Open Data on AWS](https://registry.opendata.aws/) already
contained 1,153 active datasets when I tested it. The catalog spans satellite
imagery, weather, life sciences, genomics, LiDAR, census data, and web archives.
The hard part was not finding a file with public access. It was turning a
research question into the right collection, license, access path, and first
executable query.

The [Registry of Open Data MCP server](https://github.com/awslabs/mcp/tree/main/src/roda-mcp-server)
puts those qualification steps inside an agent conversation. The
[launch post](https://aws.amazon.com/blogs/opensource/introducing-mcp-server-for-registry-of-open-data-on-aws/)
describes discovery as the entry point. My test found a more precise boundary:
the server is a strong qualification interface behind a weak natural-language
search interface.

## What the server can do

Version `3.4.5` exposed 10 tools in my live session. They cover discovery,
qualification, bounded inspection, and handoff:

| Capability | Tools | What they return | Boundary |
|---|---|---|---|
| Search and browse | `search_datasets`, `list_datasets` | Keyword results plus tag, organization, and license filters | Search uses lexical matching, not semantic ranking |
| Focus discovery | `discover_by_organization`, `discover_by_license` | Datasets scoped to a provider or supported license family | Results depend on Registry metadata consistency |
| Inspect one dataset | `get_dataset_details` | Description, license, documentation, update frequency, and resources | Metadata describes access; it does not perform analysis |
| Follow catalog relationships | `find_related_datasets` | Other datasets with shared tags | Shared tags do not prove scientific relevance |
| Inspect the catalog | `get_knowledge_base_stats` | Dataset, tag, organization, resource, and license counts | Counts are a live catalog snapshot |
| Inspect public storage | `preview_dataset` | Up to 10 S3 object keys without downloading their contents | Multiple buckets require selection; private objects need credentials |
| Read a bounded sample | `sample_dataset` | Up to 100 KB from a selected public object; text display is capped at 2,000 characters | Sampling is inspection, not bulk data processing |
| Find domain handoffs | `search_stac_endpoints` | STAC APIs and catalogs referenced by Registry entries | The returned STAC service remains a separate interface |

The server runs locally over standard input and output. This is the MCP client
configuration shape I tested:

```json
{
  "mcpServers": {
    "roda": {
      "command": "uvx",
      "args": ["awslabs.roda-mcp-server@latest"],
      "env": {
        "FASTMCP_LOG_LEVEL": "ERROR"
      }
    }
  }
}
```

The equivalent direct launch command is:

```bash
uvx awslabs.roda-mcp-server@latest
```

Catalog search and metadata inspection did not require AWS credentials. Live
storage access still reflected each dataset's actual permissions.

## The 12-question test

I ran 12 target-retrieval tasks across wildfire, oceans, weather, agriculture,
genomics, single-cell biology, autonomous driving, terrain, web history,
biodiversity, flooding, and population data. Each task had a known-fit Registry
entry selected before the search.

Every task used the same sequence:

1. Search with the plain-language research question.
2. Search again with distinctive terms and a tag or organization filter.
3. Retrieve the target's metadata.
4. Check its license, documentation, and resources.
5. Run a live access preview.

This measured target retrieval, not universal relevance. Another result could
be useful and still count as a miss if the preselected target did not appear.

| Measure | Plain question | Constrained query |
|---|---:|---:|
| Known-fit target in top 3 | 2/12 | 11/12 |
| Known-fit target in top 20 | 3/12 | 12/12 |

All 12 targets returned license, resource, and documentation metadata. Ten
returned a live anonymous bucket preview. One required a bucket selection.
Common Crawl returned an access-denied result with credential instructions.

<details>
<summary>Complete benchmark results</summary>

| Domain | Plain rank | Constrained rank | Access result |
|---|---:|---:|---|
| Wildfire | Miss | 11 | Public preview |
| Oceans | Miss | 2 | Public preview |
| Weather | 11 | 1 | Public preview |
| Agriculture | 2 | 1 | Bucket selection required |
| Genomics | 1 | 1 | Public preview |
| Single-cell biology | Miss | 1 | Public preview |
| Autonomous driving | Miss | 1 | Public preview |
| Terrain | Miss | 2 | Public preview |
| Web history | Miss | 1 | Credentials required |
| Biodiversity | Miss | 2 | Public preview |
| Flooding | Miss | 2 | Public preview |
| Population | Miss | 1 | Public preview |

</details>

Across 48 per-case MCP calls, median latency was 50.2 milliseconds, average
latency was 99.5 milliseconds, and the slowest call took 854.0 milliseconds.
This was one run on July 29, 2026, not a reliability study.

The retrieval gap was not subtle. Plain questions matched between 421 and 1,153
datasets, and nine known-fit targets disappeared beyond the first 20 results.
The server removes only a few generic terms, accepts a record when any remaining
term appears as a substring, and then diversifies the result set by provider. It
does not calculate a relevance score.

Constrained queries changed the result. Distinctive dataset terms plus a tag or
organization filter recovered all 12 targets. The agent host currently has to
do the query planning that natural-language search implies.

## The handoff test

Finding metadata is not the same as using data. I ran a second experiment to
test the handoff from MCP qualification to an actual analysis:

```text
research question
  -> RODA MCP discovery and access qualification
  -> Earth Search STAC scene selection
  -> bounded Cloud-Optimized GeoTIFF reads
  -> NDVI change
  -> numerical and visual evidence
```

The target was [Sentinel-2 Cloud-Optimized GeoTIFFs](https://registry.opendata.aws/sentinel-2-l2a-cogs/).
A broad MCP query for that name matched 304 datasets and missed the target in
the first 20. The constrained query `sentinel s2 l2a cogs`, filtered to
`Element 84`, returned three records and ranked the target second.

The detail call then supplied the [Earth Search documentation](https://github.com/Element84/earth-search),
Sentinel access terms, public S3 and notification resource types, and the STAC
collection links. The preview correctly stopped at another boundary: the entry
has four public buckets, so the server asked for a bucket selection rather than
guessing.

That metadata was enough to leave MCP and use the domain interface.

## A bounded wildfire screen

The [CAL FIRE incident record](https://www.fire.ca.gov/incidents/2021/6/24/lava-fire)
places the 2021 Lava Fire near Mount Shasta at `41.459, -122.329`, reports a
June 24 start, and records 26,409 acres burned. I defined a fixed rectangle
around that published location. It is an area of interest, not a reconstructed
fire perimeter.

The script queried the current Earth Search `sentinel-2-c1-l2a` collection for
two windows:

- Pre-fire: June 1 through June 23, 2021
- Post-fire: July 15 through August 15, 2021

For each window, it selected the lowest-cloud scene that fully covered the same
area. It read red, near-infrared, scene-classification, and true-color assets
directly from their Cloud-Optimized GeoTIFFs. The scene-classification layer
removed cloud, shadow, snow, water, and invalid pixels. No complete source scene
was written to disk.

Each 10-meter source raster was 10,980 by 10,980 pixels. The geographic window
was 2,125 by 1,938 source pixels, or 3.42% of the tile's pixel footprint. The
analysis resampled that window to 20 meters and computed:

```text
NDVI = (near infrared - red) / (near infrared + red)
```

The selected dates were June 18 and July 18. Across 948,704 valid overlapping
pixels, mean NDVI moved from `0.4266` to `0.3549`. Mean change was `-0.0716`;
median change was `-0.0107`. The lower tenth percentile reached `-0.3377`.

The screening threshold matters more than the average. Pixels with a drop of at
least `0.2` covered 17.74% of the valid area, or 67.3152 square kilometers at
the analysis resolution.

![Pre-fire and post-fire Sentinel-2 imagery with NDVI change near the Lava Fire incident location](https://artificialcuriositylabs.ai/assets/roda-mcp/lava-fire-ndvi-change.png)

In the change panel, red marks lower post-fire NDVI and blue marks higher
post-fire NDVI. The strongest red region aligns visually with the scar in the
post-fire image. That is evidence that the handoff works. It is not a
burn-severity result. The rectangle includes land outside the final fire
boundary, and NDVI also responds to seasonality, smoke, terrain shadows, and
land management.

The experiment is reproducible from the repository:

```bash
uv run scripts/roda-sentinel-ndvi-experiment.py
```

The script reruns MCP discovery, queries live STAC metadata, selects scenes,
performs bounded raster reads, and regenerates the JSON and image artifacts.

## What the server unlocks

The server does not make open data public. It turns catalog work into agent
actions: search, filter, inspect a license, enumerate resources, and test an
access boundary. That removes the manual transition between a research question
and a known technical entry point.

The Sentinel experiment shows why that matters. MCP did not calculate NDVI.
It identified the collection, exposed its terms and resources, and pointed to
the interface that could. Earth Search selected scenes. Rasterio read bounded
COG windows. NumPy performed the calculation. Each layer stayed inside its
actual capability.

This pattern generalizes beyond satellite data:

- Registry discovery can hand ocean observations to a NetCDF or Zarr workflow.
- It can hand web archives to Common Crawl's index.
- It can hand single-cell data to a domain API.
- It can hand point clouds to a LiDAR processing engine.

The unlock is not one universal data tool. It is a reliable routing layer from
intent to the correct specialized tool.

## What's missing

Search ranking is the immediate gap. Broad natural-language prompts can match
nearly the whole catalog because common words survive tokenization and any-term
substring matching. Structured filters repair retrieval, but only after the
host agent extracts the right terms.

Access preview also stops at storage. It can show that a bucket is public,
requires credentials, or needs a selection. It does not inspect scientific
fitness, choose an analysis method, or validate a result.

The open question is where semantic selection belongs. It could move into the
MCP server, remain in the host agent over a larger candidate set, or become a
separate domain-specific discovery layer. The current server proves the value
of the qualification boundary. The benchmark also shows that the host still
does the most important retrieval work.