---
name: ipsum-ingestion
description: "How to get IPsum Threat IP Feed data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load the IPsum threat-IP feed into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/ipsum
Provider: https://github.com/stamparm/ipsum
Checked against the provider's documentation: 2026-11-01

## Access
- No account or key. The full feed and threshold files are public raw GitHub downloads.

## Request
```bash
curl 'https://raw.githubusercontent.com/stamparm/ipsum/master/ipsum.txt'
```

## Format
Tab-separated text with comment headers, one IP address per data row, and an integer count of the public source lists containing that IP.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""IPsum full threat-IP feed -> pandas -> Parquet. No key."""
import io
import requests
import pandas as pd

URL = "https://raw.githubusercontent.com/stamparm/ipsum/master/ipsum.txt"
r = requests.get(URL, timeout=120)
r.raise_for_status()
df = pd.read_csv(io.StringIO(r.text), sep="\t", comment="#",
                 names=["ip", "source_count"], dtype={"ip": "string"})
df["source_count"] = pd.to_numeric(df["source_count"], downcast="integer")
df.to_parquet("ipsum.parquet", index=False)
print(len(df), "IP addresses", "source count", int(df["source_count"].min()), "to",
      int(df["source_count"].max()), int((df["source_count"] >= 3).sum()), "at level 3+")
```
Run by us on 2026-11-01 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 139773 IP addresses source count 1 to 12 18258 at level 3+

## Limits
- Rate limit: Not stated.
- The repository is refreshed every 24 hours.
- Preprocessed level files cover thresholds from at least one through at least eight source lists.

## Quirks
- Lines beginning with # are metadata and must be skipped before parsing the two tab-separated fields.
- source_count measures occurrence across source lists, not event frequency, severity or probability.
- Do not connect to listed hosts; use the downloaded feed only for defensive matching.

## Terms and attribution
- The repository uses the Unlicense and dedicates the work to the public domain.
- The work is supplied as-is without warranty.
- Terms page: https://raw.githubusercontent.com/stamparm/ipsum/master/LICENSE

## Sources
- https://raw.githubusercontent.com/stamparm/ipsum/master/README.md: "All lists are automatically retrieved and parsed on a daily (every 24 hours) basis and the final result is pushed to this repository."
- https://raw.githubusercontent.com/stamparm/ipsum/master/README.md: "Higher counts generally mean higher confidence and fewer false positives when blocking inbound traffic."
- https://raw.githubusercontent.com/stamparm/ipsum/master/LICENSE: "This is free and unencumbered software released into the public domain."

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.