A comment on my Hit Tracking with PHP and MySQL post raised some interesting questions about what a consumer of Web analytics data needs to consider when selecting providers for their sites.
Most folks are familiar with the model of Web analytics vendors: You place their JS tag on your page which makes a call back to their centralized system with all of the data that can be collected about the visitor who has just come to your site.
There are three items that you, the consumer of this third-party service, need to get straight answers about.

Performance of the Tag

Visitors to your site do not know which components you are responsible for and which you have farmed out to vendors and services. In their mind, all content is your content. This means that the performance of the Web analytics tag, which if not placed correctly on the page can affect the rendering of the content, is a critical factor in selecting a vendor.
To my knowledge, none of the Web analytics vendors used by most people blog and social media firms (StatCounter, GetClicky, Google Analytics, Omniture, etc.) freely share response time and error rate metrics of their tag infrastructure with the world as measured by a third party.
Is the tag delivered from a central location, or does the provider use a CDN? Is the data delivered asynchronously or synchronously?
The download performance of the tag is not the only concern. Ask them to provide data on the average processing time that they see when a client parses and processes the code. How does this processing time vary from browser to browser, between OSs? What steps are they taking to ensure that the tag does not affect the perceived performance of your page?
Ask for this data. Ask that the measurement data be collected by an impartial third party. Demand that this data be freely available to you before you make a decision on purchasing their service.

Size of the Tag

All of these services rely on a JS tag to collect and deliver the data to their data warehouse. Ask the provider to tell you how large this tag is, and what steps they have taken to reduce its size so that it has a limited effect on the download of your content.
Have they minified the tag? Is it delivered using HTTP compression to further reduce the size? Are ways to reduce the size of the tag always under considerations?

Data Storage

Analyzing data takes a couple of forms: daily operational views to spot changes or issues; and long-term trending to find larger patterns and major shifts in your visitors. As a result, data needs to be available with a substantial degree of detail while still providing aggregated data that allows larger patterns to be easily discovered.
How long does your provider store detailed data? What is the data expiry policy? Can you extract the data and import it into your own database?

Summary

Web analytics is key to determining what works and doesn’t work on your site. It tells who, where and how people are accessing your content. But without a Web analytics partner/vendor that provides performance and support, you may be left more in the cold than if you were just looking at your raw Web logs.