---
name: zenodo-rest-api-ingestion
description: "How to get Zenodo REST API data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load Zenodo records for a search into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/zenodo-rest-api
Provider: https://developers.zenodo.org/
Checked against the provider's documentation: 2026-11-03

## Access
- Public records search needs no key: Zenodo publishes guest limits (60 requests per minute, 2,000 per hour), and on 2026-11-04 a search of /api/records without a token returned HTTP 200 with records.
- A free account and access token give higher limits (100 per minute, 5,000 per hour) and are needed for authenticated and deposit operations; set ZENODO_TOKEN and the snippet sends it as a Bearer header. Zenodo's developer docs also say broadly that all API access requires an access token.

## Request
```bash
curl 'https://zenodo.org/api/records?q=%22working%20paper%22%20AND%20economics&size=25&sort=mostrecent'
```

## Format
JSON with hits.total and hits.hits; each record has id, doi, metadata (title, publication_date, resource_type, creators, license, description) and links (self_html, files).

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""Zenodo published records -> pandas -> Parquet. Works without a key; sends ZENODO_TOKEN if set (optional, for higher rate limits)."""
import os
import requests
import pandas as pd

headers = {"Authorization": f"Bearer {os.environ['ZENODO_TOKEN']}"} if os.environ.get("ZENODO_TOKEN") else {}
params = {"q": "\"working paper\" AND economics", "size": 25, "sort": "mostrecent"}
r = requests.get("https://zenodo.org/api/records", params=params, headers=headers, timeout=60)
r.raise_for_status()
js = r.json()
rows = [{
    "record_id": h["id"],
    "doi": h.get("doi"),
    "title": h["metadata"].get("title"),
    "publication_date": h["metadata"].get("publication_date"),
    "resource_type": (h["metadata"].get("resource_type") or {}).get("type"),
    "license": (h["metadata"].get("license") or {}).get("id"),
    "url": h["links"].get("self_html"),
} for h in js["hits"]["hits"]]
df = pd.DataFrame(rows)
df.to_parquet("zenodo_records.parquet", index=False)
print(len(df), "records on this page of", js["hits"]["total"], "matches; X-RateLimit-Limit:", r.headers.get("X-RateLimit-Limit"))
```
Run by us on 2026-11-03 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 25 records on this page of 2460 matches; X-RateLimit-Limit: 30

## Limits
- Search REST endpoints: 30 requests per minute; global limit 60 per minute (guest) or 100 per minute (authenticated).
- Check X-RateLimit-Limit, X-RateLimit-Remaining and X-RateLimit-Reset on every response.

## Quirks
- description fields are HTML; strip tags before text processing.
- license is per record and can be missing for closed-access records.

## Terms and attribution
- Metadata is CC0 except email addresses.
- Files are used under the licence they were deposited with; check each record's license.
- Terms page: https://about.zenodo.org/policies/

## Sources
- https://developers.zenodo.org/: "Global limit for guest users 60 requests per minute, 2000 requests per hour"
- https://about.zenodo.org/policies/: "Metadata is licensed under CC0, except for email addresses."
- https://about.zenodo.org/policies/: "Use and re-use is subject to the license under which the data objects were deposited."

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.