Google Analytics Accuracy – Comparing Google Analytics, Yahoo Web Analytics and Nielsen SiteCensus
Last year I wrote an whitepaper on web analytics accuracy. The intention of this was to be a reference guide to all the accuracy issues that on-site web measurement tools face, and how you can mitigate the error bars. Apart from updating the article recently, I wanted to illustrate how close (or not) different vendor tools on the same website can be when it comes to counting the basics – visits, pageviews, time on site and visitors.
To do this, I have looked at two very different web sites with two tools collecting web visitor data side by side:
- Site A – This blog, running Google Analytics and Yahoo Web Analytics. According to Google, there are 188 pages in the Google Index and traffic is approximately 10,000 visits/month
- Site B – A retail site that runs Nielsen SiteCensus and Google Analytics (site to remain anonymous). According to Google, there 12,808 pages in the Google Index and traffic is approximately 1 million visits/month
These are obviously two very different web sites with two different objectives…
Site A is relatively small in content and visit volume, with the purpose of driving engagement i.e. readership and their interaction via comments, ratings, click throughs etc. For this site, I had complete control of the deployment of both web analytics tools. This enabled me to have a best practice deployment of both Google Analytics and Yahoo Web Analytics.
Site B is approximately 100x larger in terms of content and traffic. Its main objective is to drive sales via online transactions. For this site, I had complete control of the Google Analytics implementation, with no control of the SiteCensus implementation as this was before my time. However, as the tool was professionally installed, I am assuming it was a best practice deployment.
Both tools use first party cookies for visitor tracking.
I analysed the reports for August, September, October 2008 and took the average of the reported differences between the two tools for the following 5 metrics:
- Visits – also known as the number of sessions
- Pageviews – also known as the number of page impressions
- Pages/visit – the average number of pageviews per visit
- Time on site – the average time a visit lasts
- Unique visitors
Comparing each of the three months separately was preferred in order to mitigate any outliers of simply comparing a longer 90 day period. A monthly interval also reduced the effects of cookie deletion for tracking unique visitors – the longer the time period, the greater the chance the original ID cookie being lost or deleted and therefore the inflation of unique visitors. The approach was validated by selecting weekly comparisons at random. These showed almost identical differences as the monthly comparisons. The results are shown in Tables 1 and 2.