---
name: faostat-api-bulk-ingestion
description: "How to get FAO FAOSTAT API and bulk downloads data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load a FAOSTAT consumer-price bulk file into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/faostat-api-bulk
Provider: https://www.fao.org/faostat/en/
Checked against the provider's documentation: 2026-11-01

## Access
- No account or key is needed for the checked bulk catalog and ZIP file.
- The API developer portal documents programmatic JSON and CSV access; the API host was intermittent during this check.

## Request
```bash
curl -O 'https://bulks-faostat.fao.org/production/ConsumerPriceIndices_E_All_Data_(Normalized).zip'
```

## Format
ZIP containing normalized CSV observations plus area, item, element and flag lookup files. Observation columns include area, item, element, month, year, unit, value, flag and note.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""FAOSTAT normalized consumer-price bulk file -> pandas -> Parquet."""
import io
import zipfile
import requests
import pandas as pd

URL = "https://bulks-faostat.fao.org/production/ConsumerPriceIndices_E_All_Data_(Normalized).zip"
HEADERS = {"User-Agent": "Mozilla/5.0 SignalLabCheck/1.0"}
r = requests.get(URL, headers=HEADERS, timeout=120)
r.raise_for_status()
with zipfile.ZipFile(io.BytesIO(r.content)) as zf:
    name = "ConsumerPriceIndices_E_All_Data_(Normalized).csv"
    df = pd.read_csv(zf.open(name), low_memory=False)
df.to_parquet("faostat_consumer_prices.parquet", index=False)
print(len(df), "rows", df["Area"].nunique(), "areas", int(df["Year"].min()), "to", int(df["Year"].max()))
```
Run by us on 2026-11-01 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 266292 rows 247 areas 2000 to 2026

## Limits
- Rate limit: Not stated.
- The checked normalized consumer-price ZIP is about 2.1 MB compressed and expands to about 41.3 MB across five CSV files.
- Bulk files contain complete domains; select columns and rows after download.

## Quirks
- Domain release dates differ; use the bulk catalog DateUpdate field rather than assuming one FAOSTAT-wide refresh date.
- Keep Flag and Note because they identify estimates, imputations and base-year details.
- Area Code (M49) contains a leading apostrophe in the CSV and is safest to retain as text.

## Terms and attribution
- FAOSTAT data are CC BY 4.0 unless a dataset's metadata or page says otherwise.
- Use FAO's required database citation and check metadata for third-party exceptions.
- Terms page: https://www.fao.org/contact-us/terms/db-terms-of-use/en

## Sources
- https://www.fao.org/statistics/highlights-archive/highlights-detail/faostat-launches-a-new-api-developer-portal-to-make-data-access-easier/en: "Access all FAOSTAT domains through a single, consistent interface"
- https://www.fao.org/statistics/highlights-archive/highlights-detail/faostat-launches-a-new-api-developer-portal-to-make-data-access-easier/en: "Filter data by area, item, element, year and more dimensions"
- https://www.fao.org/contact-us/terms/db-terms-of-use/en: "all datasets disseminated through FAO corporate statistical databases (see examples in Annex 1) are licensed under the Creative Commons Attribution-4.0 International licence"

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.