One of the recurring questions I hear from data privacy professionals is:
“How many pages should I audit to properly assess a website’s compliance with data collection regulations?”
It’s a fair question. On the one hand, checking only a homepage is far too narrow — it will not reveal how different templates, e-commerce flows, or campaign landing pages deploy cookies and trackers. On the other hand, attempting to crawl every page of a large enterprise site is not only impractical, but also unnecessary.
To address this, I ran a structured study using the PAGE Inspector, auditing four enterprise websites across different sectors (healthcare, travel, technology, and retail) with a combined total of more than 4.4 million pages indexed by Google.
Key Findings
- 25–50 pages: Enough for a quick health check on whether a site respects “Reject All” consent choices.
- 100 pages: Sufficient to uncover ~90% of all cookies and trackers on most websites.
- 500 pages: For high-risk sectors (e.g. healthcare or finance), this provides added confidence in inventory completeness.
- Auditing all pages (>4M in this study) would take more than 50 days of continuous crawling — a huge burden with little practical benefit.
Why This Matters
Traditional large-scale audits are slow, expensive, and often outdated by the time they complete. The evidence from this study shows that smaller, targeted audits can achieve the same level of insight, and more importantly, can be run regularly to catch issues as they arise.
For organisations, this means:
- Lower audit costs
- Faster compliance insights
- A more proactive approach to governance and risk management
Download the Full Study
The full report, “Crawl Size and its Impact on Identifying Cookies and Trackers”, details the methodology, data, and practical recommendations for privacy professionals.
(no registration required!)
Brian Clifton is a measurement and data privacy strategist, author, and founder of Verified Data — the auditing platform for data quality and governance. Formerly Google’s Head of Web Analytics for EMEA, Brian has spent over two decades helping organisations build trust in their digital data. He is the author of the best-selling books Successful Analytics and the series Advanced Web Metrics with Google Analytics. He is a certified member of the European Association of Data Protection Professionals.
0 Comments