Back in November, I mentioned that I was working on the idea of new ways to benchmark the success of online businesses in today’s more mature operational environment. I am still working on the base ideas, but a colleague of mine has helped me coalesce some ideas, and they are now forming the foundation of the concepts my company will begin using internally to more deeply understand the various Web performance benchmarks we monitor.
For those who use the existing Web performance benchmarks to determine the success and failure of your online business, you understand how thin the veneer is on these benchmarks. They do not provide true insight into the operational success of an online business, and they are more likely to sow the seeds of distrust between IT and Business operations in the long-term by creating an artificial standard which becomes the goal.
If an online business truly wants to achieve and maintain exemplary Web performance numbers, it has to start with a strong foundation, and build on it. Why? The team I work with spends a lot of time trying to understand and reverse engineer the broken processes, designs, and architectures that were laid out in order to get big fast. After 3-4 years of technical starvation and underfunding, these online businesses are beginning to show strain; the temporary fix has become the permanently broken process.
The rush into the Web analytics space in the last few weeks is a key sign that companies now see value in and want to exploit the vast quantities of data that they collect on their traffic daily. Web analytics is an astoundingly complex field, but most people boil it up to a single concept: How many Unique Page Views did I get?
Unique Page Views is an outdated Web server analytics metric. It does not tell me anything about the business, other than it has a lot of traffic. Back in the “eyeballs are everything” period, this would have been a big deal. Now, I say so what, and start asking:
- From where
- Dialup? Broadband?
- How many were able to successfully complete their transactions?
- What paths are most visited
- Average spend by connection type?
- Average spend by hour?
- etc.
Like Unique Page Views, the average Web performance and availability of a Web page or transaction does not accurately represent the overall health of any online business. Within the large populations of data that exist at the Web measurement firms, there is a wealth of data that could be used to clearly expose more important benchmarking statistics.
If you are from an online business, you already understand that the average performance over an artificially-defined period of time is a very inaccurate way to measure the success of the online business. However, it is the accepted standard in the field. Underlying those aggregated values, there are clearly-defined statistical methods which can be used to extract even more meaningful information from the mass of measurement data.
I would discuss more of the ideas and concepts that we are working on, but I know that I do get visitors from our competitors, so I will have to keep our ideas under wraps for right now.
But I want to hear yours. What does your online business use as a benchmark for success? Standard avergae performance and availability? Or something more complex that examines the performance data as a complete population, as opposed to an aggregated summary value? Does your firm tie business goals and objectives into the performance benchmarks so that people across the company can understand how the business is succeeding, and how delivering a good, bad, and downright awful online performance experience can affect the bottom line?
This is an exciting time to have access to large amounts of data on the health of the Internet.
