---
name: neo23x0-signature-base-ingestion
description: "How to get Neo23x0 Signature-Base data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load the Signature-Base rule index into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/neo23x0-signature-base
Provider: https://github.com/Neo23x0/signature-base
Checked against the provider's documentation: 2026-11-01

## Access
- No account or key. Raw repository files are public downloads.
- Detection Rule License 1.1 applies except where a YARA rule explicitly states another licence.

## Request
```bash
curl -O 'https://raw.githubusercontent.com/Neo23x0/signature-base/master/sig-base-rules.csv'
```

## Format
Semicolon-delimited rule index. The ingestion keeps the rule name separate and preserves the remainder of each record because some free-text fields contain extra semicolons.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""Signature-Base rule index -> pandas -> Parquet. No key."""
import requests
import pandas as pd

URL = "https://raw.githubusercontent.com/Neo23x0/signature-base/master/sig-base-rules.csv"
r = requests.get(URL, headers={"User-Agent": "SignalLabCheck/1.0"}, timeout=30)
r.raise_for_status()
rows = []
for line in r.text.splitlines():
    if line.strip():
        rule, _, record = line.partition(";")
        rows.append({"rule": rule, "record": record})
df = pd.DataFrame(rows)
df.to_parquet("signature_base_rule_index.parquet", index=False)
print(len(df), "rule index rows", df["rule"].nunique(), "distinct rule names")
```
Run by us on 2026-11-01 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 3216 rule index rows 3216 distinct rule names

## Limits
- Rate limit: Not stated.
- The public repository is smaller than the separate commercial rule set advertised by the provider.
- Rule-specific YARA metadata can override the repository-level licence.

## Quirks
- The semicolon-delimited index has no header and some free-text records contain extra semicolons; this example preserves the unparsed remainder.
- Some rules use external variables supplied by LOKI or THOR Lite and can fail in other YARA tools unless those variables are defined.
- Do not connect to IOC URLs, domains or hosts; ingest the IOC text only.

## Terms and attribution
- Detection Rule License 1.1 permits broad reuse, including commercial use, subject to retained attribution, source-link and licence information when rules are shared.
- Messages produced from matches must retain supplied rule-author identification.
- A YARA rule can declare a different licence in its metadata.
- Terms page: https://raw.githubusercontent.com/Neo23x0/signature-base/master/LICENSE

## Sources
- https://raw.githubusercontent.com/Neo23x0/signature-base/master/README.md: "Signature-Base is the YARA signature and IOC database for our scanners"
- https://raw.githubusercontent.com/Neo23x0/signature-base/master/README.md: "All signatures and IOC files in this repository, except the YARA rules that explicitly indicate a different license (see "license" meta data), are licensed under the [Detection Rule License (DRL) 1.1](https://raw.githubusercontent.com/Neo23x0/signature-base/master/LICENSE)."
- https://raw.githubusercontent.com/Neo23x0/signature-base/master/LICENSE: "Permission is hereby granted, free of charge, to any person obtaining a copy"

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.