Scrape Google Shopping Results with Python and Playwright

To scrape Google Shopping results reliably, use Playwright to render an authorized Shopping page, extract each product card, preserve displayed evidence, and write validated data to CSV. This guide covers prices, sellers, ratings, links, pagination, locale controls, and validation.

The workflow was verified on a controlled, redacted fixture rather than through automated live extraction from Google. It validates the browser, parser, and CSV pipeline, but it does not grant permission or guarantee a current live layout.

Key Takeaways

• Use an authorized browser workflow with a controlled query and locale.
• Preserve raw product and price evidence before normalization.
• Stop when the expected surface, consent state, or validation check fails.

Before You Start: Permission, Scope, and Stop Conditions

Technical access is not the same as permission. Before using a Google Shopping scraper, review the Google Terms of Service, machine-readable instructions, contracts, applicable law, and your organization’s data-governance rules. Google’s spam policies identify unauthorized machine-generated traffic to Search as a policy violation.

Treat Google Shopping scraping as controlled data collection with documented ownership, retention, and stop conditions.

Stop the workflow instead of trying to work around the page when you encounter any of the following:

• a CAPTCHA or unusual-traffic message;
• a consent flow that has not been completed legitimately;
• a login or authentication request;
• an explicit denial or policy restriction;
• repeated error pages or an unrecognized result surface.

The tutorial does not include login automation, CAPTCHA solving, stealth plugins, access-control circumvention, checkout data, or personal information. A proxy changes the network route; it does not create authorization or make prohibited automation acceptable.

Prerequisites and Verification Environment

The test project separates a verification target from a result that was actually proven. The following environment was used on August 21, 2026:

Create and activate a virtual environment, then install the pinned package.

Windows PowerShell

py -3.12 -m venv .venv
.\.venv\Scripts\python.exe -m pip install playwright==1.62.0

macOS or Linux

python3 -m venv .venv
source .venv/bin/activate
python -m pip install playwright==1.62.0

These are the standard PyPI installation commands. On the verification machine, the normal network transfer stalled, so the same official Playwright 1.62.0 wheel and its declared dependencies were installed locally before testing. The parser, browser run, and tests below were then executed successfully; this article does not claim that the network installation itself completed on that machine.

The Windows verification used an installed stable Chrome channel. If you choose Playwright’s bundled Chromium instead, follow the current Playwright browser installation instructions and remove channel="chrome" from the launch call. That variation was not the environment tested here.

The complete project accompanying this article uses the following layout:

google-shopping-scraper/
├── README.md
├── requirements.txt
├── shopping_scraper.py
├── price_parser.py
├── selectors.py
├── fixtures/
│   ├── google_shopping_minimal_redacted.html
│   ├── google_shopping_page_2.html
│   ├── unusual_traffic.html
│   ├── consent.html
│   ├── empty_results.html
│   └── README.md
├── tests/
│   ├── test_parser.py
│   ├── test_price_parser.py
│   ├── test_url_identity.py
│   ├── test_failure_paths.py
│   └── test_validation.py
└── artifacts/
    └── google_shopping_results.csv

Understand Google Shopping Results Before Parsing

A Shopping-tab grid and a Shopping module inside a standard search page are not interchangeable. During manual review on August 21, 2026, a Google Shopping query using the older tbm=shop form redirected to a URL containing udm=28. The inspected page exposed an Ads region with a visible “Sponsored products” heading. That observation is a point-in-time finding, not a permanent URL or DOM contract.

Use module_position to describe a product’s order inside the module you actually parsed. Do not present it as a universal Google Shopping rank. Results can vary by query, country, language, device, time, personalization, inventory, and advertising.

Advertising status needs the same restraint. The extractor stores:

• placement_type: sponsored, organic, or unknown;
• placement_label_raw: the visible label when one is found;
• source_module: the region or grid that contained the card.

A missing ad label is not proof that a record is organic, so the safe fallback is unknown.

Price fields also require evidence. Preserve price_text_raw exactly as displayed. Only populate price_amount when one tested rule can interpret the text without guessing. A value such as $99.99 may be parseable under a controlled U.S. locale; From $49.99, $12.50/mo, or $63.98 at checkout describes a different commercial meaning and should not be silently converted into the same field.

Use a schema that separates observed values from interpretations:

This prevents missing values from becoming false zeroes. It also keeps parser changes auditable because the original displayed evidence survives later normalization updates.

Figure 1. A manually reviewed Google Shopping results page captured on August 21, 2026. It shows the current udm=28 surface and visible product evidence. No automated product-card extraction was performed. Source: Google.

Figure 2. A real Playwright screenshot of the controlled fixture used for verification. All records are fictional, and the page explicitly states that it is not a Google interface.

Scrape Google Shopping Results with Playwright

For broader browser-rendering patterns in Python, see Scraping dynamic web pages with Python. Apply those techniques only to pages and collection workflows you are authorized to test.

Step 1: Centralize the selectors

Google does not publish the result-card CSS classes as a stable scraping API. Keep locator candidates in one file so that selector drift does not force changes throughout the application. The fixture selectors come first; the reviewed live-surface fallbacks come second.

# selectors.py
CARD_SELECTOR = "[data-shopping-card], .mnr-c.pla-unit"
TITLE_SELECTORS = (
    '[data-field="title"]',
    '[role="heading"][aria-level="3"]',
)
CURRENT_PRICE_SELECTORS = (
    "[data-price-current]",
    ".VbBaOe",
)
OLD_PRICE_SELECTORS = (
    "[data-price-old]",
    ".tWaJ3e",
)
SELLER_SELECTORS = (
    "[data-seller]",
    '[aria-label^="From "]',
)
PRODUCT_LINK_SELECTORS = (
    "a[data-product-url]",
    "a.pla-unit-single-clickable-target",
)
REVIEW_SUMMARY_SELECTORS = (
    "[data-review-summary]",
    '[role="img"][aria-label^="Rated "]',
)
NEXT_PAGE_SELECTORS = (
    "a[data-next-page]",
    'a[aria-label="Next page"]',
)

Figure 3. The selector adapter used by the tested project. Generated class names are isolated as replaceable fallbacks, not treated as durable contracts.

The first selectors make the fixture deterministic. The second selectors reflect the manually inspected surface on the verification date. Recheck them before any authorized live run.

Step 2: Build a controlled Shopping query

Do not concatenate raw search input into a URL. Use urlencode and freeze the query, country, and language alongside the output:

from urllib.parse import urlencode

def build_live_url(query: str, language: str, country: str) -> str:
    return "https://www.google.com/search?" + urlencode({
        "q": query,
        "udm": "28",
        "hl": language,
        "gl": country.lower(),
    })

hl influences interface language and gl supplies a country hint, but neither guarantees a local-shopper view. The browser locale, time zone, cookies, device profile, network location, and Google’s own systems can still affect the response. If the expected Shopping surface is unavailable, fail clearly instead of switching silently to a different search module.

Step 3: Launch a clean context and fail safely

The verified browser setup uses explicit context values and a bounded navigation timeout:

with sync_playwright() as playwright:
    browser = playwright.chromium.launch(channel="chrome", headless=True)
    context = browser.new_context(
        locale="en-US",
        timezone_id="America/New_York",
        viewport={"width": 1440, "height": 900},
    )
    page = context.new_page()
    page.goto(target, wait_until="domcontentloaded", timeout=30_000)

If an authorized regional test needs proxy routing, configure it at browser launch and keep credentials outside source control. Rola IP’s Python proxy integration guide shows the connection pattern; it does not replace the permission checks described above.

Before parsing cards, inspect visible page text for known invalid states. The tested project stops on unusual-traffic, CAPTCHA, unresolved consent, and sign-in markers. It also exits with a nonzero status when validated records fall below MIN_VALID_RECORDS. A challenge page must never become a successful empty CSV file.

The command-line interface enforces the same boundary. --fixture is reproducible by default. Live navigation requires the separate --authorized-live acknowledgement flag; without it, the program refuses the request with exit code 2. That flag only records the operator’s acknowledgement. Code cannot determine whether a contract, site policy, or law actually authorizes the run, so permission must be documented outside the program.

Step 4: Extract typed product records

For each matched card, require a title and displayed price. Read optional seller, shipping, old price, and product-link evidence without replacing missing values with empty numeric defaults. The extractor then assigns position within its source module:

cards = page.locator(CARD_SELECTOR)
positions: dict[str, int] = {}
results = []

for index in range(cards.count()):
    card = cards.nth(index)
    title = first_text(card, TITLE_SELECTORS)
    raw_price = first_text(card, CURRENT_PRICE_SELECTORS)
    if not title or not raw_price:
        continue

    placement_type, placement_label, source_module = placement_for(card)
    positions[source_module] = positions.get(source_module, 0) + 1
    parsed = parse_price(raw_price, currency_hint="USD")
    # Build ShoppingResult with raw evidence and nullable derived fields.

Ratings and review counts follow the same evidence-first rule. The extractor retains displayed strings in rating_text_raw and review_count_text_raw. It does not turn abbreviations such as 32K into an exact integer because the page has not supplied that precision.

Canonicalize product URLs conservatively before using them as identity evidence. Removing every query parameter can merge distinct variants or seller offers. The tested implementation strips only known tracking parameters, sorts the remaining parameters, removes fragments, and preserves identity-critical values such as variant=blue:

TRACKING_PARAMS = {"gclid", "dclid", "fbclid", "msclkid", "gbraid", "wbraid"}

def canonicalize_url(url: str | None) -> str | None:
    if not url:
        return None
    parts = urlsplit(url)
    if parts.scheme not in {"http", "https"} or not parts.netloc:
        return None
    query = [
        (key, value)
        for key, value in parse_qsl(parts.query, keep_blank_values=True)
        if not key.lower().startswith("utm_")
        and key.lower() not in TRACKING_PARAMS
    ]
    query.sort(key=lambda pair: (pair[0].lower(), pair[1]))
    return urlunsplit((parts.scheme.lower(), parts.netloc.lower(),
                       parts.path.rstrip("/"), urlencode(query, doseq=True), ""))

The deterministic identity hierarchy is:

1. a cleaned product URL or exposed stable product ID;
2. otherwise, normalized title + seller + variant evidence + raw price;
3. separate records when preserved variant evidence conflicts or confidence is low.

The output includes identity_key and match_confidence. It never guesses a GTIN, EAN, MPN, or model number from the title.

Step 5: Normalize only prices you can prove

The price parser uses Decimal rather than binary floating-point numbers. Its deliberately narrow MVP supports a single displayed price and tested separator formats. It classifies special formats without inventing a comparable amount:

if re.search(r"\b(from|starting at)\b", lowered):
    return ParsedPrice(None, "from", currency_code,
                       currency_source, "unsupported_kind")
if re.search(r"(/\s*mo\b|per month|monthly)", lowered):
    return ParsedPrice(None, "monthly", currency_code,
                       currency_source, "unsupported_kind")
if re.search(r"\d[\d.,\s]*\s*[-–]\s*[$€£¥￥]?\s*\d", raw):
    return ParsedPrice(None, "range", currency_code,
                       currency_source, "unsupported_kind")

The parser records currency provenance as visible, controlled_locale_inference, or unknown. An ambiguous $ without a controlled currency hint remains null. An old price is captured only when it appears in a distinct visible field; the code does not infer a discount from two unrelated numbers.

Step 6: Export CSV and prove the run succeeded

Run the controlled workflow from the project directory:

.\.venv\Scripts\python.exe shopping_scraper.py `
  --fixture fixtures\google_shopping_minimal_redacted.html `
  --query "wireless earbuds" `
  --country US `
  --language en `
  --currency USD `
  --min-valid-records 3 `
  --max-pages 2 `
  --output artifacts\google_shopping_results.csv

The verified result was:

status=success mode=controlled-fixture records=7 output=artifacts\google_shopping_results.csv min_valid_records=3

Success means more than process exit code 0. The validator confirms that the recognized Shopping surface is present, no challenge marker is visible, the record count meets the frozen threshold, every accepted row contains both a title and raw price, and each module_position is unique within its source module. It also requires country, language, and capture time before storage. If any of those checks fail, the program returns a nonzero exit and does not describe an empty file as a successful scrape. Establish the record threshold manually for the exact frozen query and surface; do not copy the fixture’s value of three into every production workflow.

Fifteen tests passed. They cover real headless-Chrome fixture extraction, two-page continuation, variant-safe URL identity, explicit and inferred currency handling, unsupported From and monthly prices, ambiguous dollars, duplicate module positions, the minimum-record failure gate, unusual-traffic and consent pages, an empty result surface, and refusal of unacknowledged live mode.

Figure 4. A rendered transcript of the actual verification logs for the 15 tests and two-page scraper run. The paths contain no credentials, account data, cookies, or live IP addresses.

Figure 5. The generated CSV contains seven records across two controlled pages, preserves rating and review-count evidence, and leaves unsupported “from” and monthly amounts null.

Add Pagination and Build Price History

Do not assume that start=20 will always retrieve another Shopping page. A layout may expose a next link, a load-more control, continuous scrolling, or no continuation mechanism. Inspect the authorized target surface, then implement only the behavior that is actually present.

The tested fixture exposes an ordinary same-host next link. extract_pages() follows it only up to --max-pages, records page_number, rejects a link that leaves the current host, stops when no next link exists, and fails when either a URL or collected identity set repeats. The two-page test returns seven unique records and proves that variant=blue and variant=black remain separate identities. This validates the continuation algorithm, not Google’s current live pagination mechanism.

Use hard ceilings for pages, records, elapsed time, and consecutive empty results in any authorized implementation. Stop when content repeats, a challenge appears, the module changes, or validation fails. Never keep requesting pages merely because the expected button disappeared. When proxy routing is part of an authorized workflow, apply documented proxy throttling rules as an additional operational limit.

For price history, append observations instead of overwriting them. Compare records only when country, currency, seller scope, product identity, and variant confidence agree. Keep shipping separate from the displayed product price. Minimum, median, and maximum statistics are meaningful only after those dimensions match.

This is also where a conservative deduplication key matters. Two sellers offering the same model are separate offers; two titles that look similar may describe different variants. Exact GTIN/EAN and offer matching should be treated as a separate advanced workflow. For a broader implementation context, see Rola IP’s guide to proxies for price monitoring.

Collect Comparable Results by Country and Language

Record every input that could affect the snapshot:

Only call two market snapshots comparable when these inputs are controlled and recorded. A proxy can help route an authorized regional QA or price-monitoring job, but it cannot stabilize selectors or guarantee the same inventory a resident would see. When that routing is legitimately required, Rola IP is one web scraping proxy infrastructure option. Keep one sticky session within a single query snapshot, rotate only between independent authorized batches, and verify the exit location with a what is my IP check before collection.

When to Use a Google Shopping Results API Instead

A Google Shopping results API usually means a third-party service that returns structured Shopping SERP data. Google’s Merchant API is different: it manages an authorized merchant’s own Merchant Center resources and is not a public competitor-results API.

Evaluate a third-party API by permitted use and retention terms, country/language/device controls, sponsored labeling, seller and shipping coverage, freshness, cache behavior, schema versioning, retries, request IDs, and cost per validated usable result. An API shifts browser maintenance to a provider; it does not automatically resolve data rights or contractual obligations.

Reliability, Maintenance, and Troubleshooting

Monitor more than success rate. Useful signals include valid record count, missing-price rate, placement_type=unknown rate, duplicate rate, currency mismatch, and parser failures. A sudden drop in records on a recognized Shopping surface is selector drift, not a successful “zero products” result.

Limits, Data Quality, and Compliance

Displayed Shopping prices can vary by location and may not be the final checkout price. A card does not prove inventory, authenticity, seller authorization, delivered cost, or stable ranking. Advertising, personalization, history, device, and capture time may change the result set.

Respect applicable terms, machine-readable instructions, privacy obligations, contracts, and intellectual-property rights. Do not collect account, checkout, payment, or non-public personal data. Preserve placement_type=unknown whenever advertising status cannot be verified. This tutorial provides technical guidance, not legal advice.

Validate One Query Before Scaling

The reliable path is deliberately narrow: one authorized surface, one frozen query, explicit locale inputs, raw evidence, conservative price parsing, and a CSV written only after validation. Prove that workflow first. Then add continuation logic, history, regional routing, or an API only when each additional layer has its own limits and tests.