Category: Effective Web Performance

Service Level Agreements in Web Performance

Service Level Agreements (SLAs) appear to finally be maturing in the realm of Web performance. Both of the Web performance companies that I have worked for have understood their importance, but convincing the market of the importance of these metrics has been a challenge up until recently.

In the bandwidth and networking industries, SLAs have been a key component of contracts for many years. However, as Doug Kaye outlined in his book Strategies for Web Hosting and Managed Services, SLAs can also be useless.

The key to determining a useful Web performance SLA rests on some clear business concepts: relevance and enforceability. Many papers have been written on how to calculate SLAs, but that leaves companies still staggering with the understanding that they need SLAs, but don’t understand them.

Relevance

Relevance is a key SLA metric because an SLA defined by someone else may have no meaning to the types of metrics your business measures itself on. Whether the SLA is based on performance, availability or a weighted virtual metric designed specifically by the parties bound by the agreement, it has to mean something, and be meaningful.

The classic SLA is average performance of X seconds and availability of Y% over period Z. This is not particularly useful to businesses, as they have defines business metrics that they already use.

Take for example a stock trading company. in most cases, they are curious, but not concerned with their Web performance and availability between 17:00 and 08:00 Eastern Time. But when the markets are open, these metrics are critical to the business.

Now, try and take your stock-trading metric and overlay it at Amazon or eBay. Doesn’t fit. So, in a classic consultative fashion, SLAs have to be developed by asking what is useful to the client.

  • Who is to be involved in the SLA process?
  • How do SLAs for Internal Groups differ from those for External vendors?
  • Will this be pure technical measurement? Will business data be factored in?

Asking and answering these questions makes the SLA definition process relevant to the Web performance objectives set by the organization.

Enforceability

The idea that an SLA with no teeth could exist is almost funny. But if you examine the majority of SLAs that are contracted between businesses in the Web performance space today, you will find that they are so vaguely defined and meaningless to the business objectives that actually enforcing the penalty clauses is next to impossible.

As real world experience shows, it is extremely difficult for most companies enforce SLAs. If the relevance objectives discussed above are hammered out so that the targets are clear and precise, then enforcement becomes a snap. The relevance objective often fails, because the SLA is imposed by one party on another; or an SLA is included in a contract as a feature, but when something goes wrong, escape path is clear for the “violating” party.

If an organization would like to try and develop a process to define enforceable SLAs, start with the internal business units. These are easier to develop, as everyone has a shared business objective, and all disputes can be arbitrated by internal executives or team leaders.

Once the internal teams understand and are able to live with the metrics used to measure the SLAs, then this can be extended to vendors. The important part of this extension is that third-party SLA measurement organizations will need to become involved in this process.

Some would say that I am tooting my own horn by advocating the use of these third-party measurement organizations, as I have worked for two of the leaders in this area. The need for a neutral third-party is crucial in this scenario; it would be like watching a soccer match (football for the enlightened among you) without the mediating influence of the referee.


If your organization is now considering implementing SLAs, then it is crucial that these agreements are relevant and enforceable. That way, both parties understand and will strive to meet easily measured and agreed upon goals, and understand that there are penalties for not delivering performance excellence.

Web Performance Evangelism Run Amok

I wanted to point you to an evangelist of the good kind that Scoble found — “Obi-Wan”, the Prowler Knight. They come in all shapes and sizes.

One of the directors in our company keeps saying how impressed he was by a certain product evangelist he saw at a conference a few years ago. He sings high praises about my potential to do the same. I know I can — spent the last five years delving into the how-tos of Web performance, and have a bit of an opinionated streak to help me along.

Today, I am going to evangelize on Web performance.

The issue is that everyone has specific questions and nobody wants to think about the actual big picture. The biggest question an online has to ask is: “How do we make it fast, reliable, scalable, efficient and economic?”

Easy, right? Well, no actually. Big players in the online commerce world still have problems with this. Why? Why can’t they get it right?

Over the last few days, I have posted a couple of screenshots showing that Amazon, the online retailing poster child, has had 3 distinct and length outages. This is unheard of from them. However, they should be in seasonal lockdown at the moment. So I looked at some data I have access to last night. I know when the problem started, but don’t know the root cause. It is frightening that in the span of a single day, the internet leader is in the uncomfortable position of scrambling to decipher and resolve their problem during the busiest time of the year.

This doesn’t surprise me anymore. I just shrug my shoulders and say, loudly, for the umpteenth time that if someone had asked the right questions, followed the correct process, and accurately analyzed the data none of this would be happening.

I have said it before: I have tried. Look at a retailer like Amazon, and you must also look at Target — Target is completely wedded to the Amazon Infrastructure. Was Target part of the analysis of the data so that they could approve the system state freeze? The answer is likely no, and you know what? Target is likely going to collect a ton of paybacks from SLA infringements as a result of the Amazon outages.

At the beginning, I asked what does it take to achieve Web performance excellence. The answer is time and dedication. Online businesses have to either dedicate themselves to this, or sign on to partners who can.

Some big firms think that the traditional IT consulting firms can do it. What is their expertise in Web performance? How do they plan to validate and verify that the improvement plan they have outlined is actually meeting your business objectives? How will they help you manage your content, customer-tracking and ad providers?

Big IT consulting firms: Can you validate and verify that the performance improvements that you have implemented are economical? Are they efficiently resolving the issue? Who resolves problems?

How many consultant, engineers, developers and business managers does it take to fix a bad Web page?

Answer: I don’t know. Do you?

In the end, Web performance is no longer about response times and success rates. It is no longer about usability. It is no longer about hit tracking, processor utilization, SANs, and distributed content. We performance boils down to a single question:

“How do we make it fast, reliable, scalable, efficient and economic?”

Web Performance Benchmarks

Web performance benchmarks have been a part of the industry since the beginning. However, it is not 1999 anymore. What do these benchmarks really mean?

Simple, aggregated performance and success rate values based on a limited dataset for a finite time period give a very tiny perspective into the world of Web performance. The operation of a multi-billion dollar enterprise should not hinge on the ranking a firm receives in these comparative exercises.

The question I pose to the Web performance industry is this: If you could walk away from the heritage of benchmarking that we have inherited, what would you replace it with? How could you justify your IT spend on Web infrastructure to your business leaders? How could you demonstrate that your Web infrastructure was a benefit and not a drain to the business?

I don’t have the silver bullet to these questions, but I do have some very primitive new ideas running around in my head, trying to evolve. I would love to here what other Web performance IT and business leaders have to say on this concept.

Copyright © 2025 Performance Zen

Theme by Anders NorenUp ↑