Claude Skill

google-ad-scraper

Scrape competitor ads from Google Ads by domain. Returns ad creatives, formats, and campaign details. Use for competitive ad research and messaging analysis.

LLM Mart · 0 points · 5 views 0 listing impressions 0 install-command copies
Virus-scanned Reviewed automatically before listing.

Full trust report

Download gooseworks-ai-goose-skills-skills_ads_capabilities_google-ad-scraper-e1592ee.zip · 5 KB
Part of gooseworks-ai/goose-skills — 44 skills

Install

skills CLI npx skills add https://github.com/gooseworks-ai/goose-skills/tree/main/skills/ads/capabilities/google-ad-scraper
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install gooseworks-ai-goose-skills@llmmart
Git git clone https://github.com/gooseworks-ai/goose-skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole gooseworks-ai/goose-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

Google Ads Scraper

Scrape ads from Google Ads using the Apify burbn/google-ads-search actor. Search by domain to get ad creatives, formats, and campaign details.

Quick Start

Requires APIFY_API_TOKEN env var (or --token flag).

# Search by domain (recommended)
python3 skills/google-ad-scraper/scripts/search_google_ads.py \
  --domain "hubspot.com"

# Search by company name (resolves to domain via transparency center)
python3 skills/google-ad-scraper/scripts/search_google_ads.py \
  --company "Nike"

# Limit results
python3 skills/google-ad-scraper/scripts/search_google_ads.py \
  --domain "hubspot.com" --max-ads 30

# Human-readable summary
python3 skills/google-ad-scraper/scripts/search_google_ads.py \
  --domain "stripe.com" --output summary

How It Works

  1. Domain Input: Pass the target company's domain directly via --domain
  2. Company Name Resolution (optional): If only --company is provided, the script searches Google Ads Transparency Center using Apify's web-scraper (Puppeteer) to resolve the company name to advertiser info
  3. Ad Scraping: Calls the Apify burbn/google-ads-search actor with {"domain": "...", "maxItems": N}
  4. Output: Returns ads as JSON or human-readable summary

CLI Reference

Flag Default Description
--domain none Company domain (e.g. hubspot.com) — recommended
--company none Company name (resolved to domain via transparency center)
--max-ads 50 Maximum number of ads to return
--output json Output format: json or summary
--token env var Apify token (prefer APIFY_API_TOKEN env var)
--timeout 300 Max seconds to wait for Apify run

At least one of --company or --domain is required.

Output Fields

Each ad in the output contains:

{
  "advertiserId": "AR13129532367502835713",
  "advertiserName": "Nike, Inc.",
  "creativeId": "CR12345678901234567890",
  "originalUrl": "https://www.nike.com/",
  "imageUrl": "https://...",
  "variantFormat": "TEXT",
  "variantContent": "Shop the latest Nike shoes...",
  "variants": [...],
  "variantCount": 3,
  "startDate": "2026-01-15"
}

Output fields:

Field Description
advertiserId Google Ads advertiser ID
advertiserName Company/advertiser display name
creativeId Unique ID for the ad creative
originalUrl Destination URL the ad links to
imageUrl URL of the ad image (if applicable)
variantFormat Ad format (TEXT, IMAGE, VIDEO, etc.)
variantContent Ad copy/text content
variants Array of ad variants
variantCount Number of variants for this creative
startDate Date the ad first appeared

Cost

  • Ad scraping: Varies by actor pricing, typically a few cents per domain
  • Company name resolution (optional): ~$0.05 (one web-scraper page)

Common Workflows

1. Competitor Ad Research

python3 skills/google-ad-scraper/scripts/search_google_ads.py \
  --domain "competitor.com" --max-ads 100 --output summary

2. Compare Multiple Competitors

# Run for each competitor domain
for domain in "competitor1.com" "competitor2.com" "competitor3.com"; do
  python3 skills/google-ad-scraper/scripts/search_google_ads.py \
    --domain "$domain" --max-ads 50
done

Limitations

  • Company name resolution uses Puppeteer-based web scraping of Google's SPA. It may occasionally fail — use --domain for best results.
  • Ad coverage: Google only shows ads from verified advertisers. Some smaller advertisers may not appear.
  • Historical data: Primarily shows recently active ads.
Files (goose-skills)
  • scripts
    • search_google_ads.py 11.1 KB
      #!/usr/bin/env python3
      """
      Search Google Ads by domain using Apify.
      Scrapes ad creatives, formats, and campaign details via the burbn/google-ads-search actor.
      
      Usage:
        python3 search_google_ads.py --domain "hubspot.com"
        python3 search_google_ads.py --company "Nike"
        python3 search_google_ads.py --domain "nike.com" --max-ads 30
      """
      
      import json
      import os
      import sys
      import argparse
      import requests
      import time as time_mod
      import re
      from urllib.parse import quote
      
      
      ACTOR_ID = "burbn~google-ads-search"
      
      GOOSEWORKS_API_BASE = os.environ.get("GOOSEWORKS_API_BASE", "https://api.gooseworks.ai")
      GOOSEWORKS_API_KEY = os.environ.get("GOOSEWORKS_API_KEY")
      
      if GOOSEWORKS_API_KEY:
          BASE_URL = f"{GOOSEWORKS_API_BASE}/v1/proxy/apify"
      else:
          BASE_URL = "https://api.apify.com/v2"
      
      # Google Ads Transparency Center base URL (used for advertiser ID resolution)
      GADS_BASE = "https://adstransparency.google.com"
      
      
      def get_token(cli_token=None):
          """Get API token from CLI arg, GOOSEWORKS_API_KEY, or APIFY_API_TOKEN env var."""
          token = cli_token or GOOSEWORKS_API_KEY or os.environ.get("APIFY_API_TOKEN")
          if not token:
              print("Error: Set GOOSEWORKS_API_KEY or APIFY_API_TOKEN env var.", file=sys.stderr)
              sys.exit(1)
          return token
      
      
      def resolve_advertiser_id(company=None, domain=None, token=None, timeout=120):
          """
          Resolve a company name to a domain via Google Ads Transparency Center.
      
          This is an optional helper for when only a company name is provided.
          It uses Apify's web-scraper to search the transparency center and extract
          advertiser info including the domain.
      
          Args:
              company: Company name to search
              domain: Company domain (e.g. "nike.com") — if provided, returns immediately
              token: Apify API token
              timeout: Max seconds to wait
      
          Returns:
              dict with advertiser_id, advertiser_name, and domain, or None
          """
          if domain:
              return {"domain": domain}
      
          search_term = company
          if not search_term:
              return None
      
          print(f"Searching Google Ads Transparency Center for: {search_term}", file=sys.stderr)
      
          scraper_actor = "apify~web-scraper"
          search_url = f"{GADS_BASE}/?region=anywhere&text={quote(search_term)}"
      
          print(f"Attempting to resolve company name to advertiser info...", file=sys.stderr)
      
          run_input = {
              "startUrls": [{"url": search_url}],
              "pageFunction": """async function pageFunction(context) {
                  const { page, request } = context;
                  // Wait for advertiser results to load
                  await page.waitForSelector('advertiser-row, .advertiser-name, [data-advertiser-id], a[href*="/advertiser/"]', { timeout: 15000 }).catch(() => {});
                  await new Promise(r => setTimeout(r, 3000));
      
                  // Extract advertiser links and info
                  const advertisers = await page.evaluate(() => {
                      const results = [];
                      const links = document.querySelectorAll('a[href*="/advertiser/"]');
                      links.forEach(link => {
                          const href = link.getAttribute('href') || '';
                          const match = href.match(/\\/advertiser\\/(AR\\d+)/);
                          if (match) {
                              const name = link.textContent.trim() || '';
                              results.push({
                                  advertiser_id: match[1],
                                  advertiser_name: name,
                                  url: 'https://adstransparency.google.com' + href,
                              });
                          }
                      });
                      const seen = new Set();
                      return results.filter(r => {
                          if (seen.has(r.advertiser_id)) return false;
                          seen.add(r.advertiser_id);
                          return true;
                      });
                  });
      
                  return advertisers;
              }""",
              "proxyConfiguration": {"useApifyProxy": True},
              "maxRequestsPerCrawl": 1,
          }
      
          resp = requests.post(
              f"{BASE_URL}/acts/{scraper_actor}/runs",
              json=run_input,
              params={"token": token},
          )
          resp.raise_for_status()
          run_data = resp.json()
          run_id = run_data["data"]["id"]
          print(f"Advertiser lookup run started (ID: {run_id})", file=sys.stderr)
      
          # Poll for completion
          deadline = time_mod.time() + timeout
          while time_mod.time() < deadline:
              status_resp = requests.get(
                  f"{BASE_URL}/acts/{scraper_actor}/runs/{run_id}",
                  params={"token": token},
              )
              status_resp.raise_for_status()
              status_data = status_resp.json()
              status = status_data["data"]["status"]
      
              if status == "SUCCEEDED":
                  break
              elif status in ("FAILED", "ABORTED", "TIMED-OUT"):
                  print(f"Advertiser lookup {status}. Try providing --domain directly.", file=sys.stderr)
                  return None
      
              time_mod.sleep(3)
          else:
              print("Advertiser lookup timed out. Try providing --domain directly.", file=sys.stderr)
              return None
      
          # Fetch results
          dataset_id = status_data["data"]["defaultDatasetId"]
          dataset_resp = requests.get(
              f"{BASE_URL}/datasets/{dataset_id}/items",
              params={"token": token, "format": "json"},
          )
          dataset_resp.raise_for_status()
          results = dataset_resp.json()
      
          advertisers = []
          for item in results:
              if isinstance(item, list):
                  advertisers.extend(item)
              elif isinstance(item, dict):
                  if "advertiser_id" in item:
                      advertisers.append(item)
                  elif isinstance(item.get("result"), list):
                      advertisers.extend(item["result"])
      
          if advertisers:
              print(f"Found {len(advertisers)} advertiser(s):", file=sys.stderr)
              for adv in advertisers[:5]:
                  print(f"  - {adv.get('advertiser_name', 'Unknown')}: {adv.get('advertiser_id', 'N/A')}", file=sys.stderr)
              return advertisers[0]
          else:
              print("No advertisers found. Try providing --domain directly.", file=sys.stderr)
              return None
      
      
      def run_ad_scraper(token, domain, max_ads=50, timeout=300):
          """
          Run the burbn/google-ads-search actor.
      
          Args:
              token: Apify API token
              domain: Domain to search ads for (e.g. "hubspot.com")
              max_ads: Maximum ads to return
              timeout: Max seconds to wait
      
          Returns:
              List of ad dicts from the actor's dataset
          """
          run_input = {
              "domain": domain,
              "maxItems": max_ads,
          }
      
          print(f"Starting Google Ads scraper for domain: {domain}...", file=sys.stderr)
          resp = requests.post(
              f"{BASE_URL}/acts/{ACTOR_ID}/runs",
              json=run_input,
              params={"token": token},
          )
          resp.raise_for_status()
          run_data = resp.json()
          run_id = run_data["data"]["id"]
          print(f"Run started (ID: {run_id})", file=sys.stderr)
      
          # Poll for completion
          deadline = time_mod.time() + timeout
          while time_mod.time() < deadline:
              status_resp = requests.get(
                  f"{BASE_URL}/acts/{ACTOR_ID}/runs/{run_id}",
                  params={"token": token},
              )
              status_resp.raise_for_status()
              status_data = status_resp.json()
              status = status_data["data"]["status"]
      
              if status == "SUCCEEDED":
                  print("Scraping complete.", file=sys.stderr)
                  break
              elif status in ("FAILED", "ABORTED", "TIMED-OUT"):
                  print(f"Actor run {status}.", file=sys.stderr)
                  raise RuntimeError(f"Actor run {status}: {json.dumps(status_data['data'], indent=2)}")
      
              print(f"Status: {status}...", file=sys.stderr)
              time_mod.sleep(5)
          else:
              raise TimeoutError(f"Actor run did not complete within {timeout}s")
      
          # Fetch dataset items
          dataset_id = status_data["data"]["defaultDatasetId"]
          dataset_resp = requests.get(
              f"{BASE_URL}/datasets/{dataset_id}/items",
              params={"token": token, "format": "json"},
          )
          dataset_resp.raise_for_status()
          ads = dataset_resp.json()
          print(f"Fetched {len(ads)} ads.", file=sys.stderr)
          return ads
      
      
      def format_summary(ads):
          """Format ads as a human-readable summary."""
          lines = []
          lines.append(f"{'#':<4} {'Advertiser':<25} {'Format':<12} {'Start Date':<12} {'Creative URL (preview)'}")
          lines.append("-" * 110)
          for i, ad in enumerate(ads, 1):
              advertiser = str(ad.get("advertiserName") or ad.get("advertiserId") or "Unknown")[:24]
              fmt = str(ad.get("variantFormat") or "")[:11]
              start_date = str(ad.get("startDate") or "")[:11]
      
              url = ad.get("originalUrl") or ad.get("imageUrl") or ""
              url_preview = str(url)[:45] if url else ""
      
              lines.append(f"{i:<4} {advertiser:<25} {fmt:<12} {start_date:<12} {url_preview}")
      
          lines.append(f"\nTotal: {len(ads)} ads")
          return "\n".join(lines)
      
      
      def main():
          parser = argparse.ArgumentParser(
              description="Search Google Ads by domain using Apify burbn/google-ads-search actor",
              formatter_class=argparse.RawDescriptionHelpFormatter,
              epilog="""
      Examples:
        # Search by domain (recommended)
        %(prog)s --domain "hubspot.com"
      
        # Search by company name (resolves to domain via transparency center)
        %(prog)s --company "Nike"
      
        # Limit results
        %(prog)s --domain "hubspot.com" --max-ads 30
      
        # Human-readable summary
        %(prog)s --domain "stripe.com" --output summary
      """,
          )
      
          parser.add_argument("--company", help="Company name to search for (resolved to domain via transparency center)")
          parser.add_argument("--domain", help="Company domain (e.g. hubspot.com) — recommended, most direct")
          parser.add_argument("--max-ads", type=int, default=50,
                              help="Max number of ads to return (default: 50)")
          parser.add_argument("--output", choices=["json", "summary"], default="json",
                              help="Output format (default: json)")
          parser.add_argument("--token", help="Apify API token (or set APIFY_API_TOKEN env var)")
          parser.add_argument("--timeout", type=int, default=300,
                              help="Max seconds to wait for Apify run (default: 300)")
      
          args = parser.parse_args()
      
          if not args.company and not args.domain:
              parser.error("At least one of --company or --domain is required")
      
          token = get_token(args.token)
      
          # Resolve domain
          domain = args.domain
          if not domain:
              # Try to resolve company name to domain
              result = resolve_advertiser_id(
                  company=args.company,
                  token=token,
              )
              if result and result.get("domain"):
                  domain = result["domain"]
                  print(f"Resolved to domain: {domain}", file=sys.stderr)
              else:
                  print("Could not resolve company name to domain.", file=sys.stderr)
                  print("Tips:", file=sys.stderr)
                  print("  1. Use --domain directly (e.g. --domain nike.com)", file=sys.stderr)
                  print("  2. The domain is what appears in the company's ad URLs", file=sys.stderr)
                  sys.exit(1)
      
          # Run the ad scraper
          ads = run_ad_scraper(
              token=token,
              domain=domain,
              max_ads=args.max_ads,
              timeout=args.timeout,
          )
      
          # Output
          if args.output == "summary":
              print(format_summary(ads))
          else:
              print(json.dumps(ads, indent=2))
      
      
      if __name__ == "__main__":
          main()
      
  • SKILL.md 3.8 KB
    ---
    name: google-ad-scraper
    description: Scrape competitor ads from Google Ads by domain. Returns ad creatives, formats, and campaign details. Use for competitive ad research and messaging analysis.
    ---
    
    # Google Ads Scraper
    
    Scrape ads from Google Ads using the Apify `burbn/google-ads-search` actor. Search by domain to get ad creatives, formats, and campaign details.
    
    ## Quick Start
    
    Requires `APIFY_API_TOKEN` env var (or `--token` flag).
    
    ```bash
    # Search by domain (recommended)
    python3 skills/google-ad-scraper/scripts/search_google_ads.py \
      --domain "hubspot.com"
    
    # Search by company name (resolves to domain via transparency center)
    python3 skills/google-ad-scraper/scripts/search_google_ads.py \
      --company "Nike"
    
    # Limit results
    python3 skills/google-ad-scraper/scripts/search_google_ads.py \
      --domain "hubspot.com" --max-ads 30
    
    # Human-readable summary
    python3 skills/google-ad-scraper/scripts/search_google_ads.py \
      --domain "stripe.com" --output summary
    ```
    
    ## How It Works
    
    1. **Domain Input**: Pass the target company's domain directly via `--domain`
    2. **Company Name Resolution** (optional): If only `--company` is provided, the script searches Google Ads Transparency Center using Apify's web-scraper (Puppeteer) to resolve the company name to advertiser info
    3. **Ad Scraping**: Calls the Apify `burbn/google-ads-search` actor with `{"domain": "...", "maxItems": N}`
    4. **Output**: Returns ads as JSON or human-readable summary
    
    ## CLI Reference
    
    | Flag | Default | Description |
    |------|---------|-------------|
    | `--domain` | none | Company domain (e.g. hubspot.com) — recommended |
    | `--company` | none | Company name (resolved to domain via transparency center) |
    | `--max-ads` | 50 | Maximum number of ads to return |
    | `--output` | json | Output format: `json` or `summary` |
    | `--token` | env var | Apify token (prefer `APIFY_API_TOKEN` env var) |
    | `--timeout` | 300 | Max seconds to wait for Apify run |
    
    At least one of `--company` or `--domain` is required.
    
    ## Output Fields
    
    Each ad in the output contains:
    
    ```json
    {
      "advertiserId": "AR13129532367502835713",
      "advertiserName": "Nike, Inc.",
      "creativeId": "CR12345678901234567890",
      "originalUrl": "https://www.nike.com/",
      "imageUrl": "https://...",
      "variantFormat": "TEXT",
      "variantContent": "Shop the latest Nike shoes...",
      "variants": [...],
      "variantCount": 3,
      "startDate": "2026-01-15"
    }
    ```
    
    **Output fields:**
    
    | Field | Description |
    |-------|-------------|
    | `advertiserId` | Google Ads advertiser ID |
    | `advertiserName` | Company/advertiser display name |
    | `creativeId` | Unique ID for the ad creative |
    | `originalUrl` | Destination URL the ad links to |
    | `imageUrl` | URL of the ad image (if applicable) |
    | `variantFormat` | Ad format (TEXT, IMAGE, VIDEO, etc.) |
    | `variantContent` | Ad copy/text content |
    | `variants` | Array of ad variants |
    | `variantCount` | Number of variants for this creative |
    | `startDate` | Date the ad first appeared |
    
    ## Cost
    
    - Ad scraping: Varies by actor pricing, typically a few cents per domain
    - Company name resolution (optional): ~$0.05 (one web-scraper page)
    
    ## Common Workflows
    
    ### 1. Competitor Ad Research
    
    ```bash
    python3 skills/google-ad-scraper/scripts/search_google_ads.py \
      --domain "competitor.com" --max-ads 100 --output summary
    ```
    
    ### 2. Compare Multiple Competitors
    
    ```bash
    # Run for each competitor domain
    for domain in "competitor1.com" "competitor2.com" "competitor3.com"; do
      python3 skills/google-ad-scraper/scripts/search_google_ads.py \
        --domain "$domain" --max-ads 50
    done
    ```
    
    ## Limitations
    
    - **Company name resolution** uses Puppeteer-based web scraping of Google's SPA. It may occasionally fail — use `--domain` for best results.
    - **Ad coverage**: Google only shows ads from verified advertisers. Some smaller advertisers may not appear.
    - **Historical data**: Primarily shows recently active ads.
    
  • skill.meta.json 252 B
    {
      "slug": "google-ad-scraper",
      "category": "capabilities",
      "tags": [
        "ads"
      ],
      "installation": {
        "base_command": "npx goose-skills install google-ad-scraper",
        "supports": [
          "claude",
          "cursor",
          "codex"
        ]
      }
    }
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.

Related