---
name: circl-hashlookup-bgp-ranking-ingestion
description: "How to get CIRCL hashlookup and BGP Ranking data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Query CIRCL hashlookup and BGP Ranking into Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/circl-hashlookup-bgp-ranking
Provider: https://www.circl.lu/services/
Checked against the provider's documentation: 2026-11-01

## Access
- No account or key. Both checked JSON API endpoints are public.
- Use hashlookup /info for database status and POST a JSON object containing an asn string to BGP Ranking /json/asn.

## Request
```bash
curl 'https://hashlookup.circl.lu/info'
curl -X POST -H 'Content-Type: application/json' -d '{"asn":"13335"}' 'https://bgpranking.circl.lu/json/asn'
```

## Format
JSON objects. hashlookup /info returns dataset version and key-count fields; BGP /json/asn returns meta.asn plus response fields for AS description and current ranking.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""CIRCL hashlookup status and one BGP Ranking query -> pandas -> Parquet."""
import requests
import pandas as pd

HASH_INFO = "https://hashlookup.circl.lu/info"
BGP_ASN = "https://bgpranking.circl.lu/json/asn"
h = requests.get(HASH_INFO, timeout=60)
h.raise_for_status()
b = requests.post(BGP_ASN, json={"asn": "13335"}, timeout=60)
b.raise_for_status()
hash_info = pd.DataFrame([h.json()])
bgp = pd.json_normalize(b.json())
hash_info.to_parquet("circl_hashlookup_info.parquet", index=False)
bgp.to_parquet("circl_bgp_asn_13335.parquet", index=False)
print("hashlookup keys", len(hash_info.columns),
      "BGP rows", len(bgp), "BGP columns", len(bgp.columns))
```
Run by us on 2026-11-01 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: hashlookup keys 2 BGP rows 1 BGP columns 5

## Limits
- Rate limit: Not stated.
- hashlookup has no stated service-level commitment.
- The optional local SHA-1 Bloom filter is about 700 MB, omits metadata and updates monthly.

## Quirks
- A hash match indicates a known file, not a malicious-file verdict; inspect source and hashlookup:trust context.
- BGP responses are nested; pandas.json_normalize produces dotted column names.
- A null position or zero rank may reflect unavailable ranking coverage and should not be converted into a clean-network label.

## Terms and attribution
- CIRCL's open-data page applies CC-BY-4.0 unless a resource states otherwise.
- The hashlookup OpenAPI metadata says CC-BY and responses retain upstream source fields; preserve attribution and provenance.
- Terms page: https://www.circl.lu/opendata/

## Sources
- https://www.circl.lu/services/hashlookup/: "CIRCL hashlookup is a public API for looking up file hashes in known-file databases."
- https://www.circl.lu/services/hashlookup/: "It is free to use and provided on a best-effort basis."
- https://www.circl.lu/services/bgp-ranking/: "analysts identify malicious activity associated with an autonomous system (AS)"
- https://www.circl.lu/opendata/: "Creative Commons Attribution 4.0 International licence"

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.