---
name: abuse-ch-threat-feeds-ingestion
description: "How to get abuse.ch threat feeds data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load recent ThreatFox IOCs into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/abuse-ch-threat-feeds
Provider: https://abuse.ch/
Checked against the provider's documentation: 2026-11-01

## Access
- Create a free abuse.ch Auth-Key and place it in ABUSE_CH_AUTH_KEY.
- This example fetches only ThreatFox IOC metadata; it does not connect to any listed indicator.

## Request
```bash
curl -H "Auth-Key: $ABUSE_CH_AUTH_KEY" -H 'Content-Type: application/json' -X POST 'https://threatfox-api.abuse.ch/api/v1/' -d '{"query":"get_iocs","days":1}'
```

## Format
JSON response with IOC value and type, threat type, malware family, confidence, first and last seen times, reporter, tags and reference fields.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""ThreatFox IOCs first seen in the past day -> pandas -> Parquet."""
import os
import requests
import pandas as pd

URL = "https://threatfox-api.abuse.ch/api/v1/"
key = os.environ["ABUSE_CH_AUTH_KEY"]
r = requests.post(URL, headers={"Auth-Key": key},
                  json={"query": "get_iocs", "days": 1}, timeout=60)
r.raise_for_status()
payload = r.json()
if payload.get("query_status") != "ok":
    raise RuntimeError(payload.get("query_status", "ThreatFox query failed"))
df = pd.json_normalize(payload["data"], sep="_")
for column in ("first_seen", "last_seen"):
    if column in df:
        df[column] = pd.to_datetime(df[column], utc=True, errors="coerce")
df.to_parquet("threatfox_recent_iocs.parquet", index=False)
print(len(df), "IOCs", "from", df["first_seen"].min().isoformat())
```
Not run: it needs a free API key.

## Limits
- Recent-IOC queries accept 1 to 7 days; the default is 3 days.
- Tag and malware-family queries return at most 1,000 results.
- Community query-volume limits are not stated numerically.

## Quirks
- URLhaus and ThreatFox have different access terms from the CC0 Feodo Tracker and SSLBL feeds.
- IOC status and infrastructure ownership can change; retain first_seen, last_seen and confidence fields.
- Do not visit or fetch an IOC URL or host. Only query the provider API or download its feed.

## Terms and attribution
- Authenticated URLhaus and ThreatFox community use is free for not-for-profit purposes within fair-use volumes; commercial use may require a paid subscription.
- Feodo Tracker and SSLBL are free without stated limitations under CC0 principles for commercial and non-commercial use.
- Terms page: https://abuse.ch/terms-of-use/

## Sources
- https://threatfox.abuse.ch/api/: "If you don't have one you can get one for free here:"
- https://urlhaus.abuse.ch/api/: "The dumps are generated every 5 minutes. Please do not fetch it more often than every 5 minutes."
- https://feodotracker.abuse.ch/blocklist/: "and is available in the plain-text and JSON format."
- https://abuse.ch/terms-of-use/: "Feodo Tracker, SSL Blacklist and YARAify are all currently free to use without limitations according to Creative Commons principles (CC0) for both commercial and non-commercial purposes."

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.