google-ad-scraper
Scrape competitor ads from Google Ads by domain. Returns ad creatives, formats, and campaign details. Use for competitive ad research and messaging analysis.
Install
npx skills add https://github.com/gooseworks-ai/goose-skills/tree/main/skills/ads/capabilities/google-ad-scraper
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install gooseworks-ai-goose-skills@llmmart
git clone https://github.com/gooseworks-ai/goose-skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole gooseworks-ai/goose-skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Google Ads Scraper
Scrape ads from Google Ads using the Apify burbn/google-ads-search actor. Search by domain to get ad creatives, formats, and campaign details.
Quick Start
Requires APIFY_API_TOKEN env var (or --token flag).
# Search by domain (recommended)
python3 skills/google-ad-scraper/scripts/search_google_ads.py \
--domain "hubspot.com"
# Search by company name (resolves to domain via transparency center)
python3 skills/google-ad-scraper/scripts/search_google_ads.py \
--company "Nike"
# Limit results
python3 skills/google-ad-scraper/scripts/search_google_ads.py \
--domain "hubspot.com" --max-ads 30
# Human-readable summary
python3 skills/google-ad-scraper/scripts/search_google_ads.py \
--domain "stripe.com" --output summary
How It Works
- Domain Input: Pass the target company's domain directly via
--domain - Company Name Resolution (optional): If only
--companyis provided, the script searches Google Ads Transparency Center using Apify's web-scraper (Puppeteer) to resolve the company name to advertiser info - Ad Scraping: Calls the Apify
burbn/google-ads-searchactor with{"domain": "...", "maxItems": N} - Output: Returns ads as JSON or human-readable summary
CLI Reference
| Flag | Default | Description |
|---|---|---|
--domain |
none | Company domain (e.g. hubspot.com) — recommended |
--company |
none | Company name (resolved to domain via transparency center) |
--max-ads |
50 | Maximum number of ads to return |
--output |
json | Output format: json or summary |
--token |
env var | Apify token (prefer APIFY_API_TOKEN env var) |
--timeout |
300 | Max seconds to wait for Apify run |
At least one of --company or --domain is required.
Output Fields
Each ad in the output contains:
{
"advertiserId": "AR13129532367502835713",
"advertiserName": "Nike, Inc.",
"creativeId": "CR12345678901234567890",
"originalUrl": "https://www.nike.com/",
"imageUrl": "https://...",
"variantFormat": "TEXT",
"variantContent": "Shop the latest Nike shoes...",
"variants": [...],
"variantCount": 3,
"startDate": "2026-01-15"
}
Output fields:
| Field | Description |
|---|---|
advertiserId |
Google Ads advertiser ID |
advertiserName |
Company/advertiser display name |
creativeId |
Unique ID for the ad creative |
originalUrl |
Destination URL the ad links to |
imageUrl |
URL of the ad image (if applicable) |
variantFormat |
Ad format (TEXT, IMAGE, VIDEO, etc.) |
variantContent |
Ad copy/text content |
variants |
Array of ad variants |
variantCount |
Number of variants for this creative |
startDate |
Date the ad first appeared |
Cost
- Ad scraping: Varies by actor pricing, typically a few cents per domain
- Company name resolution (optional): ~$0.05 (one web-scraper page)
Common Workflows
1. Competitor Ad Research
python3 skills/google-ad-scraper/scripts/search_google_ads.py \
--domain "competitor.com" --max-ads 100 --output summary
2. Compare Multiple Competitors
# Run for each competitor domain
for domain in "competitor1.com" "competitor2.com" "competitor3.com"; do
python3 skills/google-ad-scraper/scripts/search_google_ads.py \
--domain "$domain" --max-ads 50
done
Limitations
- Company name resolution uses Puppeteer-based web scraping of Google's SPA. It may occasionally fail — use
--domainfor best results. - Ad coverage: Google only shows ads from verified advertisers. Some smaller advertisers may not appear.
- Historical data: Primarily shows recently active ads.
Files (goose-skills)
-
scripts
-
search_google_ads.py 11.1 KB
#!/usr/bin/env python3 """ Search Google Ads by domain using Apify. Scrapes ad creatives, formats, and campaign details via the burbn/google-ads-search actor. Usage: python3 search_google_ads.py --domain "hubspot.com" python3 search_google_ads.py --company "Nike" python3 search_google_ads.py --domain "nike.com" --max-ads 30 """ import json import os import sys import argparse import requests import time as time_mod import re from urllib.parse import quote ACTOR_ID = "burbn~google-ads-search" GOOSEWORKS_API_BASE = os.environ.get("GOOSEWORKS_API_BASE", "https://api.gooseworks.ai") GOOSEWORKS_API_KEY = os.environ.get("GOOSEWORKS_API_KEY") if GOOSEWORKS_API_KEY: BASE_URL = f"{GOOSEWORKS_API_BASE}/v1/proxy/apify" else: BASE_URL = "https://api.apify.com/v2" # Google Ads Transparency Center base URL (used for advertiser ID resolution) GADS_BASE = "https://adstransparency.google.com" def get_token(cli_token=None): """Get API token from CLI arg, GOOSEWORKS_API_KEY, or APIFY_API_TOKEN env var.""" token = cli_token or GOOSEWORKS_API_KEY or os.environ.get("APIFY_API_TOKEN") if not token: print("Error: Set GOOSEWORKS_API_KEY or APIFY_API_TOKEN env var.", file=sys.stderr) sys.exit(1) return token def resolve_advertiser_id(company=None, domain=None, token=None, timeout=120): """ Resolve a company name to a domain via Google Ads Transparency Center. This is an optional helper for when only a company name is provided. It uses Apify's web-scraper to search the transparency center and extract advertiser info including the domain. Args: company: Company name to search domain: Company domain (e.g. "nike.com") — if provided, returns immediately token: Apify API token timeout: Max seconds to wait Returns: dict with advertiser_id, advertiser_name, and domain, or None """ if domain: return {"domain": domain} search_term = company if not search_term: return None print(f"Searching Google Ads Transparency Center for: {search_term}", file=sys.stderr) scraper_actor = "apify~web-scraper" search_url = f"{GADS_BASE}/?region=anywhere&text={quote(search_term)}" print(f"Attempting to resolve company name to advertiser info...", file=sys.stderr) run_input = { "startUrls": [{"url": search_url}], "pageFunction": """async function pageFunction(context) { const { page, request } = context; // Wait for advertiser results to load await page.waitForSelector('advertiser-row, .advertiser-name, [data-advertiser-id], a[href*="/advertiser/"]', { timeout: 15000 }).catch(() => {}); await new Promise(r => setTimeout(r, 3000)); // Extract advertiser links and info const advertisers = await page.evaluate(() => { const results = []; const links = document.querySelectorAll('a[href*="/advertiser/"]'); links.forEach(link => { const href = link.getAttribute('href') || ''; const match = href.match(/\\/advertiser\\/(AR\\d+)/); if (match) { const name = link.textContent.trim() || ''; results.push({ advertiser_id: match[1], advertiser_name: name, url: 'https://adstransparency.google.com' + href, }); } }); const seen = new Set(); return results.filter(r => { if (seen.has(r.advertiser_id)) return false; seen.add(r.advertiser_id); return true; }); }); return advertisers; }""", "proxyConfiguration": {"useApifyProxy": True}, "maxRequestsPerCrawl": 1, } resp = requests.post( f"{BASE_URL}/acts/{scraper_actor}/runs", json=run_input, params={"token": token}, ) resp.raise_for_status() run_data = resp.json() run_id = run_data["data"]["id"] print(f"Advertiser lookup run started (ID: {run_id})", file=sys.stderr) # Poll for completion deadline = time_mod.time() + timeout while time_mod.time() < deadline: status_resp = requests.get( f"{BASE_URL}/acts/{scraper_actor}/runs/{run_id}", params={"token": token}, ) status_resp.raise_for_status() status_data = status_resp.json() status = status_data["data"]["status"] if status == "SUCCEEDED": break elif status in ("FAILED", "ABORTED", "TIMED-OUT"): print(f"Advertiser lookup {status}. Try providing --domain directly.", file=sys.stderr) return None time_mod.sleep(3) else: print("Advertiser lookup timed out. Try providing --domain directly.", file=sys.stderr) return None # Fetch results dataset_id = status_data["data"]["defaultDatasetId"] dataset_resp = requests.get( f"{BASE_URL}/datasets/{dataset_id}/items", params={"token": token, "format": "json"}, ) dataset_resp.raise_for_status() results = dataset_resp.json() advertisers = [] for item in results: if isinstance(item, list): advertisers.extend(item) elif isinstance(item, dict): if "advertiser_id" in item: advertisers.append(item) elif isinstance(item.get("result"), list): advertisers.extend(item["result"]) if advertisers: print(f"Found {len(advertisers)} advertiser(s):", file=sys.stderr) for adv in advertisers[:5]: print(f" - {adv.get('advertiser_name', 'Unknown')}: {adv.get('advertiser_id', 'N/A')}", file=sys.stderr) return advertisers[0] else: print("No advertisers found. Try providing --domain directly.", file=sys.stderr) return None def run_ad_scraper(token, domain, max_ads=50, timeout=300): """ Run the burbn/google-ads-search actor. Args: token: Apify API token domain: Domain to search ads for (e.g. "hubspot.com") max_ads: Maximum ads to return timeout: Max seconds to wait Returns: List of ad dicts from the actor's dataset """ run_input = { "domain": domain, "maxItems": max_ads, } print(f"Starting Google Ads scraper for domain: {domain}...", file=sys.stderr) resp = requests.post( f"{BASE_URL}/acts/{ACTOR_ID}/runs", json=run_input, params={"token": token}, ) resp.raise_for_status() run_data = resp.json() run_id = run_data["data"]["id"] print(f"Run started (ID: {run_id})", file=sys.stderr) # Poll for completion deadline = time_mod.time() + timeout while time_mod.time() < deadline: status_resp = requests.get( f"{BASE_URL}/acts/{ACTOR_ID}/runs/{run_id}", params={"token": token}, ) status_resp.raise_for_status() status_data = status_resp.json() status = status_data["data"]["status"] if status == "SUCCEEDED": print("Scraping complete.", file=sys.stderr) break elif status in ("FAILED", "ABORTED", "TIMED-OUT"): print(f"Actor run {status}.", file=sys.stderr) raise RuntimeError(f"Actor run {status}: {json.dumps(status_data['data'], indent=2)}") print(f"Status: {status}...", file=sys.stderr) time_mod.sleep(5) else: raise TimeoutError(f"Actor run did not complete within {timeout}s") # Fetch dataset items dataset_id = status_data["data"]["defaultDatasetId"] dataset_resp = requests.get( f"{BASE_URL}/datasets/{dataset_id}/items", params={"token": token, "format": "json"}, ) dataset_resp.raise_for_status() ads = dataset_resp.json() print(f"Fetched {len(ads)} ads.", file=sys.stderr) return ads def format_summary(ads): """Format ads as a human-readable summary.""" lines = [] lines.append(f"{'#':<4} {'Advertiser':<25} {'Format':<12} {'Start Date':<12} {'Creative URL (preview)'}") lines.append("-" * 110) for i, ad in enumerate(ads, 1): advertiser = str(ad.get("advertiserName") or ad.get("advertiserId") or "Unknown")[:24] fmt = str(ad.get("variantFormat") or "")[:11] start_date = str(ad.get("startDate") or "")[:11] url = ad.get("originalUrl") or ad.get("imageUrl") or "" url_preview = str(url)[:45] if url else "" lines.append(f"{i:<4} {advertiser:<25} {fmt:<12} {start_date:<12} {url_preview}") lines.append(f"\nTotal: {len(ads)} ads") return "\n".join(lines) def main(): parser = argparse.ArgumentParser( description="Search Google Ads by domain using Apify burbn/google-ads-search actor", formatter_class=argparse.RawDescriptionHelpFormatter, epilog=""" Examples: # Search by domain (recommended) %(prog)s --domain "hubspot.com" # Search by company name (resolves to domain via transparency center) %(prog)s --company "Nike" # Limit results %(prog)s --domain "hubspot.com" --max-ads 30 # Human-readable summary %(prog)s --domain "stripe.com" --output summary """, ) parser.add_argument("--company", help="Company name to search for (resolved to domain via transparency center)") parser.add_argument("--domain", help="Company domain (e.g. hubspot.com) — recommended, most direct") parser.add_argument("--max-ads", type=int, default=50, help="Max number of ads to return (default: 50)") parser.add_argument("--output", choices=["json", "summary"], default="json", help="Output format (default: json)") parser.add_argument("--token", help="Apify API token (or set APIFY_API_TOKEN env var)") parser.add_argument("--timeout", type=int, default=300, help="Max seconds to wait for Apify run (default: 300)") args = parser.parse_args() if not args.company and not args.domain: parser.error("At least one of --company or --domain is required") token = get_token(args.token) # Resolve domain domain = args.domain if not domain: # Try to resolve company name to domain result = resolve_advertiser_id( company=args.company, token=token, ) if result and result.get("domain"): domain = result["domain"] print(f"Resolved to domain: {domain}", file=sys.stderr) else: print("Could not resolve company name to domain.", file=sys.stderr) print("Tips:", file=sys.stderr) print(" 1. Use --domain directly (e.g. --domain nike.com)", file=sys.stderr) print(" 2. The domain is what appears in the company's ad URLs", file=sys.stderr) sys.exit(1) # Run the ad scraper ads = run_ad_scraper( token=token, domain=domain, max_ads=args.max_ads, timeout=args.timeout, ) # Output if args.output == "summary": print(format_summary(ads)) else: print(json.dumps(ads, indent=2)) if __name__ == "__main__": main()
-
-
SKILL.md 3.8 KB
--- name: google-ad-scraper description: Scrape competitor ads from Google Ads by domain. Returns ad creatives, formats, and campaign details. Use for competitive ad research and messaging analysis. --- # Google Ads Scraper Scrape ads from Google Ads using the Apify `burbn/google-ads-search` actor. Search by domain to get ad creatives, formats, and campaign details. ## Quick Start Requires `APIFY_API_TOKEN` env var (or `--token` flag). ```bash # Search by domain (recommended) python3 skills/google-ad-scraper/scripts/search_google_ads.py \ --domain "hubspot.com" # Search by company name (resolves to domain via transparency center) python3 skills/google-ad-scraper/scripts/search_google_ads.py \ --company "Nike" # Limit results python3 skills/google-ad-scraper/scripts/search_google_ads.py \ --domain "hubspot.com" --max-ads 30 # Human-readable summary python3 skills/google-ad-scraper/scripts/search_google_ads.py \ --domain "stripe.com" --output summary ``` ## How It Works 1. **Domain Input**: Pass the target company's domain directly via `--domain` 2. **Company Name Resolution** (optional): If only `--company` is provided, the script searches Google Ads Transparency Center using Apify's web-scraper (Puppeteer) to resolve the company name to advertiser info 3. **Ad Scraping**: Calls the Apify `burbn/google-ads-search` actor with `{"domain": "...", "maxItems": N}` 4. **Output**: Returns ads as JSON or human-readable summary ## CLI Reference | Flag | Default | Description | |------|---------|-------------| | `--domain` | none | Company domain (e.g. hubspot.com) — recommended | | `--company` | none | Company name (resolved to domain via transparency center) | | `--max-ads` | 50 | Maximum number of ads to return | | `--output` | json | Output format: `json` or `summary` | | `--token` | env var | Apify token (prefer `APIFY_API_TOKEN` env var) | | `--timeout` | 300 | Max seconds to wait for Apify run | At least one of `--company` or `--domain` is required. ## Output Fields Each ad in the output contains: ```json { "advertiserId": "AR13129532367502835713", "advertiserName": "Nike, Inc.", "creativeId": "CR12345678901234567890", "originalUrl": "https://www.nike.com/", "imageUrl": "https://...", "variantFormat": "TEXT", "variantContent": "Shop the latest Nike shoes...", "variants": [...], "variantCount": 3, "startDate": "2026-01-15" } ``` **Output fields:** | Field | Description | |-------|-------------| | `advertiserId` | Google Ads advertiser ID | | `advertiserName` | Company/advertiser display name | | `creativeId` | Unique ID for the ad creative | | `originalUrl` | Destination URL the ad links to | | `imageUrl` | URL of the ad image (if applicable) | | `variantFormat` | Ad format (TEXT, IMAGE, VIDEO, etc.) | | `variantContent` | Ad copy/text content | | `variants` | Array of ad variants | | `variantCount` | Number of variants for this creative | | `startDate` | Date the ad first appeared | ## Cost - Ad scraping: Varies by actor pricing, typically a few cents per domain - Company name resolution (optional): ~$0.05 (one web-scraper page) ## Common Workflows ### 1. Competitor Ad Research ```bash python3 skills/google-ad-scraper/scripts/search_google_ads.py \ --domain "competitor.com" --max-ads 100 --output summary ``` ### 2. Compare Multiple Competitors ```bash # Run for each competitor domain for domain in "competitor1.com" "competitor2.com" "competitor3.com"; do python3 skills/google-ad-scraper/scripts/search_google_ads.py \ --domain "$domain" --max-ads 50 done ``` ## Limitations - **Company name resolution** uses Puppeteer-based web scraping of Google's SPA. It may occasionally fail — use `--domain` for best results. - **Ad coverage**: Google only shows ads from verified advertisers. Some smaller advertisers may not appear. - **Historical data**: Primarily shows recently active ads. -
skill.meta.json 252 B
{ "slug": "google-ad-scraper", "category": "capabilities", "tags": [ "ads" ], "installation": { "base_command": "npx goose-skills install google-ad-scraper", "supports": [ "claude", "cursor", "codex" ] } }
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.