---
name: google-safe-browsing-api-ingestion
description: "How to get Google Safe Browsing API data into a pandas DataFrame and Parquet: access, the request, the format, limits, quirks and terms."
---

# Check a URL with Safe Browsing into pandas and Parquet

Source entry: https://signal-lab-jet.vercel.app/datasets/google-safe-browsing-api
Provider: https://developers.google.com/safe-browsing/reference
Checked against the provider's documentation: 2026-11-01

## Access
- Create a Google account and Cloud project, enable the Safe Browsing API, and create a free API key.
- Store the key in GOOGLE_SAFE_BROWSING_API_KEY; do not place it in source code or output.

## Request
```bash
curl -G --data-urlencode 'key='"$GOOGLE_SAFE_BROWSING_API_KEY" --data-urlencode 'urls=testsafebrowsing.appspot.com/' 'https://safebrowsing.googleapis.com/v5/urls:search'
```

## Format
JSON containing an unordered threats array with matched URL and threat types plus a cacheDuration; an empty threats array is a successful negative result.

## Load into pandas and Parquet
Needs Python 3 with requests, pandas and pyarrow.
```python
"""Check Google's Safe Browsing v5 test URL -> pandas -> Parquet. Needs a free API key."""
import json
import os
import requests
import pandas as pd

URL = "https://safebrowsing.googleapis.com/v5/urls:search"
params = {"key": os.environ["GOOGLE_SAFE_BROWSING_API_KEY"],
          "urls": "testsafebrowsing.appspot.com/"}
r = requests.get(URL, params=params, timeout=60)
r.raise_for_status()
payload = r.json()
rows = [{"url": item.get("url"),
         "threat_types": json.dumps(item.get("threatTypes", [])),
         "cache_duration": payload.get("cacheDuration")}
        for item in payload.get("threats", [])]
df = pd.DataFrame(rows, columns=["url", "threat_types", "cache_duration"])
df.to_parquet("google_safe_browsing.parquet", index=False)
print(len(df), "threat matches for the Safe Browsing test URL")
```
Not run: it needs a free API key.

## Limits
- Rate limit: project-specific default quota shown in Google Developer Console; no universal numeric limit is stated.
- urls.search accepts no more than 50 URLs per request.
- Commercial use is prohibited without a separate agreement; Google directs commercial users to the paid Web Risk API.

## Quirks
- An HTTP 200 response with an empty threats field is a valid negative lookup, not an error.
- urls.search sends raw URLs to Google; hashes.search sends four-byte SHA-256 prefixes but requires correct URL canonicalization and local full-hash matching.
- Respect cacheDuration, and do not treat a listed URL as unsafe using information older than 30 minutes.
- Do not visit or fetch submitted or returned URLs or hosts; only query the Google endpoint.

## Terms and attribution
- Safe Browsing is free of charge but restricted to non-commercial use unless Google grants a separate agreement.
- User-facing warnings must explain service limitations and use the required Google attribution; Google may use and share URLs submitted through SearchUrls.
- Terms page: https://developers.google.com/safe-browsing/terms

## Sources
- https://developers.google.com/safe-browsing/reference: "The Safe Browsing API is for non-commercial use only."
- https://developers.google.com/safe-browsing/reference/rest/v5/urls/search: "Clients MUST NOT send more than 50 URLs."
- https://developers.google.com/safe-browsing/v4/get-started: "You need an API key to access the Safe Browsing APIs."
- https://developers.google.com/safe-browsing/terms: "You may not treat a URL from Google's list as an unsafe web resource"

## Help improve this skill
Missing a source, or see something to improve? Call the MCP tool suggest_source or feedback (or POST /api/suggest or /api/feedback). Reviewed ideas: https://signal-lab-jet.vercel.app/agent-suggestions

Information for educational purposes only. Free tiers, limits and licenses belong to the providers and may change; check the provider's own terms before using the data.