---
name: doaj-api-ingestion
description: "How to get DOAJ (Directory of Open Access Journals) API data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load DOAJ article records for a search into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/doaj-api
Provider: https://doaj.org/docs/api/
Checked against the provider's documentation: 2026-11-03

## Access
- No key to search; API keys are mainly for publishers who submit data.
- Base URL https://doaj.org/api/; search routes are /search/articles/{query} and /search/journals/{query} with page and pageSize.

## Request
```bash
curl 'https://doaj.org/api/search/articles/bibjson.title:%22systemic%20risk%22?page=1&pageSize=50'
```

## Format
JSON with total, page, pageSize and results; each article has id and bibjson with title, year, journal (title, ISSNs, publisher), identifier (type doi, eissn, pissn), abstract, keywords and link (type fulltext).

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""DOAJ article metadata -> pandas -> Parquet. No key for search; 2 requests a second."""
import requests
import pandas as pd

r = requests.get("https://doaj.org/api/search/articles/bibjson.title:%22systemic%20risk%22",
                 params={"page": 1, "pageSize": 50}, timeout=60)
r.raise_for_status()
js = r.json()
rows = []
for a in js["results"]:
    b = a["bibjson"]
    rows.append({
        "doaj_id": a["id"],
        "title": b.get("title"),
        "journal": (b.get("journal") or {}).get("title"),
        "year": b.get("year"),
        "doi": next((i.get("id") for i in b.get("identifier", []) if i.get("type", "").lower() == "doi"), None),
        "fulltext": next((l.get("url") for l in b.get("link", []) if l.get("type") == "fulltext"), None),
    })
df = pd.DataFrame(rows)
df.to_parquet("doaj_articles.parquet", index=False)
print(len(df), "articles on this page of", js["total"], "matches")
```
Run by us on 2026-11-03 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 50 articles on this page of 395 matches

## Limits
- Two requests per second on all API routes; bursts of up to five requests per user are queued.
- Not all fields are available on all records.

## Quirks
- The query goes in the URL path, so URL-encode quotes and spaces.
- Short field names (title, doi, issn, publisher, abstract) cannot take the .exact suffix; use the full bibjson path for exact matches.

## Terms and attribution
- Article metadata copyright is waived under CC0; DOAJ website content is CC BY-SA 4.0.
- Data dumps are granted case by case under the licences that accompany the data.
- Terms page: https://doaj.org/docs/api/

## Sources
- https://doaj.org/docs/api/: "There is a rate limit of two requests per second on all API routes."
- https://doaj.org/docs/api/: "API keys are usually only available to publishers who submit data to DOAJ."
- https://doaj.org/docs/api/: "Copyrights and related rights for article metadata waived via CC0 1.0 Universal (CC0) Public Domain Dedication."

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.