---
name: cer-pipeline-throughput-ingestion
description: "How to get CER Pipeline Throughput and Capacity data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load CER Keystone pipeline data into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/cer-pipeline-throughput
Provider: https://open.canada.ca/data/en/dataset/dc343c43-a592-4a27-8ee7-c77df56afb34
Checked against the provider's documentation: 2026-11-01

## Access
- No account or key. CER publishes separate CSV files for individual pipelines and data dictionaries.

## Request
```bash
curl -O 'https://www.cer-rec.gc.ca/open/energy/throughput-capacity/keystone-throughput-and-capacity.csv'
```

## Format
CSV observations with date, operator, pipeline, key point, coordinates, flow direction, trade type, product, throughput, committed and uncommitted volumes, nameplate and available capacity, and variance reason.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""CER Keystone monthly throughput and capacity -> pandas -> Parquet. No key."""
import io
import requests
import pandas as pd

URL = "https://www.cer-rec.gc.ca/open/energy/throughput-capacity/keystone-throughput-and-capacity.csv"
r = requests.get(URL, timeout=60)
r.raise_for_status()
df = pd.read_csv(io.BytesIO(r.content), encoding="utf-8-sig")
df["Date"] = pd.to_datetime(df["Date"], utc=True)
df.to_parquet("cer_keystone_throughput.parquet", index=False)
print(len(df), "rows", df["Date"].min().date().isoformat(), "to",
      df["Date"].max().date().isoformat(), len(df["Key Point"].dropna().unique()), "key points")
```
Run by us on 2026-11-01 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 522 rows 2010-07-01 to 2026-06-01 1 key points

## Limits
- Rate limit: Not stated.
- The catalog is split into separate pipeline CSV files rather than one combined data file.
- Files are updated quarterly and may be refreshed as needed.

## Quirks
- The full collection begins in 2006, but individual pipeline histories differ; the checked Keystone file begins in July 2010.
- Oil and liquids records are monthly, while natural-gas records are daily.
- Capacity fields can be blank, and available capacity can differ from nameplate capacity; preserve the variance-reason field.

## Terms and attribution
- Open Government Licence – Canada permits lawful reuse, including commercial use.
- Acknowledge the source and, where possible, link to the licence.
- Terms page: https://open.canada.ca/en/open-government-licence-canada

## Sources
- https://open.canada.ca/data/en/dataset/dc343c43-a592-4a27-8ee7-c77df56afb34: "Oil and liquids pipelines report monthly data and natural gas pipelines report daily data."
- https://open.canada.ca/data/en/dataset/dc343c43-a592-4a27-8ee7-c77df56afb34: "The dataset covers 2006 to present. Files are updated quarterly but may be refreshed as needed."
- https://www.cer-rec.gc.ca/open/energy/throughput-capacity/oil-pipeline-data-dictionary.csv: "Date,Date data refers to"
- https://open.canada.ca/en/open-government-licence-canada: "Contains information licensed under the Open Government Licence – Canada."

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.