---
name: misp-galaxy-ingestion
description: "How to get MISP Galaxy Clusters data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load a MISP Galaxy threat-actor cluster into Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/misp-galaxy
Provider: https://github.com/MISP/misp-galaxy
Checked against the provider's documentation: 2026-11-01

## Access
- No account or key. Raw cluster JSON files are public downloads from the MISP Galaxy repository.

## Request
```bash
curl -O 'https://raw.githubusercontent.com/MISP/misp-galaxy/main/clusters/threat-actor.json'
```

## Format
JSON cluster metadata plus a values array; each value can include a UUID, value, description and heterogeneous nested meta fields such as aliases, country, references and classifications.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""MISP Galaxy threat-actor cluster -> pandas -> Parquet. No key."""
import json
import requests
import pandas as pd

URL = "https://raw.githubusercontent.com/MISP/misp-galaxy/main/clusters/threat-actor.json"
r = requests.get(URL, timeout=60)
r.raise_for_status()
cluster = r.json()
df = pd.json_normalize(cluster["values"])
for column in df.columns:
    if df[column].map(lambda value: isinstance(value, (list, dict))).any():
        df[column] = df[column].map(
            lambda value: json.dumps(value, sort_keys=True)
            if isinstance(value, (list, dict)) else value
        )
df.insert(0, "cluster_uuid", cluster["uuid"])
df.insert(1, "cluster_version", cluster["version"])
df.to_parquet("misp_threat_actor_cluster.parquet", index=False)
print(len(df), "threat-actor values", "cluster version", cluster["version"])
```
Run by us on 2026-11-01 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 1115 threat-actor values cluster version 358

## Limits
- Rate limit: Not stated for raw file downloads.
- There is no fixed update schedule or completeness statement.
- The checked repository directory contained 136 cluster JSON files.

## Quirks
- Cluster meta fields are heterogeneous and frequently contain lists or dictionaries; serialize or normalize them before Parquet output.
- Use the cluster and value UUIDs as stable join keys and preserve the cluster version for reproducibility.
- Cluster content is reference knowledge, not evidence that a listed actor, tool or technique is currently active.

## Terms and attribution
- The JSON files are dual-licensed under CC0-1.0 or the repository's BSD-style terms.
- Choosing the BSD-style option requires retention of its copyright notice, conditions and disclaimer on redistribution.
- Terms page: https://raw.githubusercontent.com/MISP/misp-galaxy/main/LICENSE.md

## Sources
- https://raw.githubusercontent.com/MISP/misp-galaxy/main/README.md: "A cluster can be composed of one or more elements. Elements are expressed as key-values."
- https://raw.githubusercontent.com/MISP/misp-galaxy/main/README.md: "Threat Actors, Tools, Ransomware, ATT&CK matrixes"
- https://raw.githubusercontent.com/MISP/misp-galaxy/main/LICENSE.md: "The MISP galaxy (JSON files) are dual-licensed under:"
- https://raw.githubusercontent.com/MISP/misp-galaxy/main/LICENSE.md: "CC0 1.0 Universal"

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.