A production engineering reference for teams that scrape, match, normalize, and act on competitor pricing at scale.
Anti-bot-resilient ingestion · canonical catalog matching · currency / tax / promo normalization · automated repricing · Python throughout.

Read it at www.price-monitoring.org →

Competitive price intelligence is an engineering discipline, not a scraping script. Turning a retailer's shifting HTML into a clean, matched, comparable price — and then into a defensible repricing decision — takes stateful pipelines, deterministic data contracts, probabilistic matching, and hard compliance boundaries. This repository is the complete source behind www.price-monitoring.org, a deep, practical reference for that work.

Every page is written for e-commerce analysts, pricing strategists, Python scraping developers, and retail tech teams who need workflows that survive retailer DOM churn, promotional noise, and catalog fragmentation. Expect runnable Python, hand-authored architecture diagrams, threshold tables, named failure modes, and compliance guardrails you can wire straight into production — not surface-level overviews.

The material is organised into four deeply interlinked workstreams that mirror the real data flow — from raw retailer payloads all the way to an automated price on a channel.

• Scraping & Data Ingestion Workflows — stateful orchestration, tiered fetch routing (static → API → headless), async Scrapy pipelines, headless browsers for dynamic pricing, anti-bot resilience, infinite-scroll traversal, and a consolidated compliance & legal boundaries layer (robots.txt, ToS risk tiers, GDPR).
• Core Architecture & Catalog Matching Fundamentals — the canonical product schema, GTIN/UPC identifier governance, fuzzy matching (edit distance, TF-IDF vs. transformer embeddings), cross-platform taxonomy mapping, and rule-based price hierarchy routing.
• Data Normalization & Promo Parsing Pipelines — currency conversion and rate caching, tax / shipping / landed-cost normalization, promotional discount decomposition, unit-price standardization, and statistical outlier detection.
• Competitive Price Intelligence & Repricing — competitive price indexing and positioning, automated repricing rule engines with floors and guardrails, price-elasticity modeling, MAP-violation and price-war detection, anomaly alerting, and scheduled reporting.

• Production-shaped, runnable code. Python with type hints, Decimal money handling, idempotent stages, quarantine-over-drop error handling, and streaming-first memory discipline.
• Failure modes up front. Every guide names the subtle bugs — GTIN collisions, currency drift, DOM mutations, repricing oscillation, stale competitor data, MAP breaches — before they cost you a pipeline run or a margin.
• Compliance baked in. robots.txt adherence, ToS risk tiering, GDPR-safe schemas, MAP/antitrust guardrails, and audit trails treated as first-class engineering concerns.
• A tight reference graph. Concepts link to the pages that define them, so you can go as deep as a topic needs without losing the thread.

A static site generated with Eleventy, hand-authored inline SVG diagrams, KaTeX for pricing math, and highlight.js for syntax highlighting — deployed on Cloudflare. The source in this repository is the complete content and templates behind www.price-monitoring.org.

→ Start reading at www.price-monitoring.org

More from the same project lives on GitHub at github.com/price-monitoring.