---
name: nyfed-gscpi-ingestion
description: "How to get New York Fed GSCPI data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load the latest GSCPI vintage into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/nyfed-gscpi
Provider: https://www.newyorkfed.org/research/policy/gscpi
Checked against the provider's documentation: 2026-11-01

## Access
- No account or key. The interactive chart CSV and workbook are public downloads.
- The CSV is a vintage matrix: rows are observation months and columns are release vintages.

## Request
```bash
curl -O 'https://www.newyorkfed.org/medialibrary/research/interactives/data/gscpi/gscpi_interactive_data.csv'
```

## Format
CSV vintage matrix with Date as the observation month and one index-value column per monthly release vintage. The last column is the latest available vintage.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""New York Fed current GSCPI monthly vintage -> pandas -> Parquet."""
import io
import requests
import pandas as pd

URL = "https://www.newyorkfed.org/medialibrary/research/interactives/data/gscpi/gscpi_interactive_data.csv"
r = requests.get(URL, timeout=60)
r.raise_for_status()
wide = pd.read_csv(io.StringIO(r.text))
latest_vintage = wide.columns[-1]
df = wide[["Date", latest_vintage]].rename(columns={latest_vintage: "gscpi"})
df["date"] = pd.to_datetime(df.pop("Date"), format="%d-%b-%Y", utc=True)
df["gscpi"] = pd.to_numeric(df["gscpi"], errors="coerce")
df = df.dropna(subset=["gscpi"]).sort_values("date")
df.to_parquet("nyfed_gscpi.parquet", index=False)
print(len(df), "monthly observations in vintage", latest_vintage,
      df["date"].min().date(), "to", df["date"].max().date(),
      "latest", round(df.iloc[-1]["gscpi"], 2))
```
Run by us on 2026-11-01 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 349 monthly observations in vintage Oct-26 1997-09-30 to 2026-09-30 latest 1.28

## Limits
- Download rate limit: Not stated.
- The series is monthly and normally updates on the fourth business day of each month.
- The checked Oct-26 vintage contains 349 non-null observations from September 1997 through September 2026.

## Quirks
- The CSV is a revision triangle rather than a two-column series; selecting the last column loads the latest vintage, while melting all columns preserves the revision history.
- Observation dates are month-end labels, and the release vintage is usually the following month.
- The page carries a limited-data warning caused by suspended inputs, so preserve that status with recent releases.
- The GSCPI is a standardized pressure index, not a freight price, shipment count or official New York Fed estimate.

## Terms and attribution
- The New York Fed permits use, copying and distribution for personal or business purposes under its Terms of Use.
- Copies must retain notices and source identifiers; modifications must be clearly labelled and not attributed to the New York Fed.
- Redistribution must retain the same permissions, conditions and restrictions.
- Terms page: https://www.newyorkfed.org/privacy/termsofuse

## Sources
- https://www.newyorkfed.org/research/policy/gscpi: "Our index tracks the state of global supply chains using data from the transportation and manufacturing sectors."
- https://www.newyorkfed.org/research/policy/gscpi: "We update the GSCPI at 10:00 a.m. on the fourth business day of each month."
- https://www.newyorkfed.org/privacy/termsofuse: "Download, store, and use Content in any format or media,"

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.