---
name: boem-offshore-data-ingestion
description: "How to get BOEM Offshore Energy Data Center data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Load BOEM pipeline masters into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/boem-offshore-data
Provider: https://www.data.boem.gov/Main/HtmlPage.aspx?page=downloadable-files
Checked against the provider's documentation: 2026-11-01

## Access
- No account or key for the checked delimited pipeline-master ZIP.
- The ZIP contains a headerless, quoted, comma-delimited text file; BOEM publishes the field order on a separate record-layout page.

## Request
```bash
curl -A 'Mozilla/5.0 (X11; Linux x86_64) SignalLabCheck/1.0' -O 'https://www.data.boem.gov/Pipeline/Files/pplmastdelimit.zip'
```

## Format
ZIP containing one headerless comma-delimited TXT file. Its 42 fields describe pipeline segment identifiers, origin and destination areas, blocks and leases, approval and construction dates, status, size, product, operator, water depth and allowable pressure.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""BOEM pipeline master delimited ZIP -> pandas -> Parquet."""
import io
import zipfile
import requests
import pandas as pd

URL = "https://www.data.boem.gov/Pipeline/Files/pplmastdelimit.zip"
HEADERS = {"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) SignalLabCheck/1.0"}
COLUMNS = ["segment_num", "segment_length", "orig_id_name", "orig_ar_code",
           "orig_blk_num", "orig_lse_num", "dest_id_name", "dest_ar_code",
           "dest_blk_num", "dest_lse_num", "aban_aprv_dt", "aban_date",
           "approved_date", "auth_code", "bd_ppl_sdv_fl", "bur_dsgn_fl",
           "cat_life_tm", "dep_flag", "ppl_const_date", "lk_detec_fl",
           "last_rev_date", "init_hs_dt", "fed_st_lgth", "status_code",
           "ppl_size_code", "row_number", "recv_maop_prss", "recv_seg_num",
           "prpsl_con_dt", "prod_code", "sys_desig_code", "ppl_row_prmte",
           "facil_operator", "min_wtr_dpth", "max_wtr_dpth", "prot_number",
           "maop_prss", "cathodic_code", "bidir_flag", "bd_ppl_fsv_fl",
           "aprv_code", "aban_type"]
r = requests.get(URL, headers=HEADERS, timeout=120)
r.raise_for_status()
with zipfile.ZipFile(io.BytesIO(r.content)) as archive:
    member = archive.namelist()[0]
    df = pd.read_csv(archive.open(member), header=None, names=COLUMNS,
                     dtype="string", skipinitialspace=True)
df = df.apply(lambda column: column.str.strip())
df.to_parquet("boem_pipeline_masters.parquet", index=False)
print(len(df), "pipeline segments", len(df.columns), "fields",
      df["last_rev_date"].max(), "latest revision date")
```
Run by us on 2026-11-01 (python 3.13.5 | pandas 3.1.0 | pyarrow 26.0.0 | requests 2.34.2). Result: 21314 pipeline segments 42 fields 20261030 latest revision date

## Limits
- Rate limit: Not stated.
- Each subject and region is distributed as a separate file family.
- The checked pipeline-master ZIP is about 1.0 MB compressed.

## Quirks
- The delimited file has no header row; keep the published record layout with the data version.
- Values are padded with spaces and dates use compact YYYYMMDD text; stripping as strings preserves identifiers and leading zeroes.
- BOEM says the working database can contain errors and is not for legal or navigational use.

## Terms and attribution
- BOEM says federal-agency materials are generally public domain but cautions that some site materials may have separate restrictions.
- The Data Center disclaimer says the working data can contain errors and is not a legal or navigational record.
- Terms page: https://www.boem.gov/about-boem/copyright-restrictions-and-permissions

## Sources
- https://www.data.boem.gov/Main/HtmlPage.aspx?page=FAQ: "A summary file provides a list of all data fields that are contained in the file selected to be downloaded."
- https://www.data.boem.gov/Main/HtmlPage.aspx?page=Disclaimer: "Some errors may exist in this data and we are constantly working to find and eliminate them."
- https://www.boem.gov/about-boem/copyright-restrictions-and-permissions: "Generally, materials produced by federal agencies are in the public domain and may be reproduced without permission."

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.