---
name: ny-fed-markets-data-api-ingestion
description: "How to get New York Fed Markets Data API data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load recent New York Fed SOFR data into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/ny-fed-markets-data-api
Provider: https://markets.newyorkfed.org/static/docs/markets-api.html
Checked against the provider's documentation: 2026-11-01

## Access
- No account or key is required for the checked public Markets Data API endpoint.
- Use of API data is subject to the New York Fed Terms of Use and its attribution conditions.

## Request
```bash
curl 'https://markets.newyorkfed.org/api/rates/secured/sofr/last/5.json'
```

## Format
JSON with effective date, rate type, percent rate, distribution percentiles, transaction volume in billions and a revision indicator.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""Five recent New York Fed SOFR observations -> pandas -> Parquet."""
import requests
import pandas as pd

URL = "https://markets.newyorkfed.org/api/rates/secured/sofr/last/5.json"
r = requests.get(URL, timeout=30)
r.raise_for_status()
df = pd.DataFrame(r.json()["refRates"])
df["effectiveDate"] = pd.to_datetime(df["effectiveDate"])
df.to_parquet("nyfed_sofr.parquet", index=False)
print(len(df), "SOFR observations", df["effectiveDate"].min().date(), "to",
      df["effectiveDate"].max().date(), "latest rate", df.iloc[0]["percentRate"])
```
Run by us on 2026-11-01 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 5 SOFR observations 2026-10-23 to 2026-10-29 latest rate 3.94

## Limits
- Rate limit: Not stated.
- No unified response cap is stated in the API documentation.
- Formats and parameters vary by endpoint.

## Quirks
- The last-N endpoint returns newest first; sort by effectiveDate before time-series calculations.
- Keep revisionIndicator because published reference-rate observations may be corrected.
- Beta endpoints are for testing and are not regularly maintained.

## Terms and attribution
- The provider permits automated access and business use subject to its Terms of Use.
- Copies and distributions must retain source identifiers and the prescribed attribution; modifications must be labelled.
- Terms page: https://www.newyorkfed.org/privacy/termsofuse

## Sources
- https://markets.newyorkfed.org/static/docs/markets-api.yml: "The supported formats are JSON, XML, PDF, CSV, and Microsoft Excel."
- https://markets.newyorkfed.org/static/docs/markets-api.yml: "Data provided in beta are for testing purposes only and are not regularly maintained."
- https://www.newyorkfed.org/privacy/termsofuse: "Download, store, and use Content in any format or media,"
- https://www.newyorkfed.org/privacy/termsofuse: "Access the Content, manually or through an automated process  or device"

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.