Category: The Web

Real User Measurement – A tool for the whole business

2022-03-29 / / 0 Comments

The latest trend A key tool in web performance measurement is the drive to implement the use of Real User Measurement (RUM) in a web performance measurement strategy. As someone who cut their teeth on synthetic measurements using distributed robots and repeatable scripts, it took me a long time to see the light of RUM, but I am now a complete convert – I understand that the richness and completeness of RUM provides data that I was blocked from seeing with synthetic data.

[UPDATE: I work for Akamai focusing on the mPulse RUM tool.]

The key for organizations is realizing that RUM is complementary to Synthetic Measurements. The two work together when identifying and solving tricky external web performance issues that can be missed by using a single measurement perspective.

The best way to adopt RUM is to use the dimensions already in place to segment and analyze visitors in traditional web analytics tools. The time and effort used in this effort can inform RUM configuration by determining:

  • Unique customer populations – registered users, loyalty program levels, etc
  • Geography
  • Browser and Device
  • Pages and site categories visited
  • Etc.

This information needs to bleed through so that it can be linked directly to the components of the infrastructure and codebase that were used when the customer made their visit. Limiting this data pool to the identification and solving of infrastructure, application, and operations issues isolates the information from a potentially huge population of hungry RUM consumers – the business side of any organization.

The Business users who fed their web analytics data into the setup of RUM need to see the benefit of their efforts. Sharing RUM with the teams that use web analytics and aligning the two strategies, companies can directly tie detailed performance data to existing customer analytics. With this combination, they can begin to truly understand the effects of A/B testing, marketing campaigns, and performance changes on business success and health. But business users need a different language to understand the data that web performance professionals consume so naturally.

I don’t know what the language is, but developing it means taking the data into business teams and seeing how it works for them. What companies will find is that the data used by one group won’t be the same as for the other, but there will be enough shared characteristics to allow the group to share a dialect of performance when speaking to each other.

This new audience presents the challenge of clearly presenting the data in a form that is easily consumed by business teams alongside existing analytics data. Providing yet another tool or interface will not drive adoption. Adoption will be driven be attaching RUM to the multi-billion dollar analytics industry so that the value of these critical metrics is easily understood by and made actionable to the business side of any organization.

So, as the proponents of RUM in web performance, the question we need to ask is not “Should we do this?”, but rather “Why aren’t we doing this already?”.

Web Performance – At What Cost? Trends for 2013

2012-11-27 / / 0 Comments
image courtesy of Corey Seeman – http://www.flickr.com/photos/cseeman/

As we moved through the traditional start of the holiday shopping season (Thanksgiving / Black Friday / Cyber Monday), it is clear that most sites were prepared for what was coming. No big names went down, no performance slowdowns rose to the headlines, and online revenue – both web and mobile – appears to have increased over 2011.

But when you these companies do their year-end review, they need to take a step back and ask: “Could we have done it better?”

While performance events were few and far between (if they occurred at all), companies will need to examine the cost of scaling their sites for performance. When planning for the peak performance period, companies will need to asses whether simply scaling-up to handle increased traffic and sales could have been managed more effectively, by implementing sites that were not only fast, but  also efficient.

Joshua Bixby (here) noted that web page size has increased 20% in the last 6 months, an indication that efficiency is not always at the top of mind when new web content is presented to visitors. In order to deliver ever more complex web content, companies are spending more on services such as CDNs and cloud services to deliver their own content, while incorporating ever increasing numbers of third-party items into their pages to supply additional content and services (analytics, performance, customer service, Help Desk, and many more) that they have outsourced.

Increasing page size, outside acceleration and cloud services, and third-party services – a potent mix that companies need to asses critically, with an eye to understanding what all of these mean for the performance experienced by their visitors and customers. Add in the increasing importance of the mobile internet, with its variable connection speeds and service quality, and things become even more interesting.

In 2013, I see companies assessing these three trends with a focus on making sites perform the same (or better!) at the same (or lower!) cost than they did in 2012.
Over the next 12 months, I will be watching the performance industry news to see if those companies that have been successful at making their sites perform under the heaviest loads increasingly focus not just on speed and availability, but on efficient delivery of their entire site at a lower cost with the best user experience possible.

The key strategics questions that online businesses will be asking in 2013 will be:

  • Have we optimized our content? This does not mean make it faster, this means make it better and more efficient. It is almost absurdly easy to make a big, inefficient site fast, but it is harder to step back and “edit” the site in a way that you deliver the same content with less work – think Chevy Volt, not Cadillac Escalade.
  • Are we in control of our third-party services? Managing what services get placed on your site is only the first step. Understanding where the content you have added comes from and whether it is optimized for the heaviest shared loads will also become important checklist items for companies.
  • Can we deliver the design and functionality our customers want at a lower cost? This is the hardest one to be successful at, as each company is different. But Devops teams should be prepared to be accountable for not just cool, but also for the cost of creating, deploying, and managing a site.

HTTP Compression – Have you checked ALL your browsers?

2012-09-25 / / 0 Comments

Apache has been my web server of choice for more than a decade. It was one of the first things I learned to compile and manage properly on linux, so I have a great affinity for it. However, there are still a few gotchas that are out there that make me grateful that I still know my way around the httpd.conf file.

HTTP compression is something I have advocated for a long time (just Googled my name and compression – I wrote some of that stuff?) as just basic common sense.
Make Stuff Smaller. Go Faster. Cost Less Bandwidth. Lower CDN Charges. [Ok, I can’t be sure of the last one.]

But, browsers haven’t always played nice. At least up until about 2008. After then, I can be pretty safe in saying that even the most brain-damaged web and mobile browsers could handle pretty much any compressed content we threw at them.

Oh, Apache! But where were you? There is an old rule that is still out there, buried deep in the httpd.conf file that can shoot you. I actually caught it yesterday when looking at a site using IE8 and Firefox 8 measurement agents at work. Firefox was about 570K while IE was nearly 980K. Turns out that server was not compressing CSS and JS files sent to IE due to this little gem:

 BrowserMatch \bMSIE !no-gzip gzip-only-text/html

This was in response to some issues with HTTP Compression in IE 5 and early versions of IE6 – remember them? – and was appropriate then. Guess what? If you still have this buried in your Apache configuration (or any web server or hardware device that does compression for you), break out the chisels: it’s likely your httpd.conf file hasn’t been touched since the stone age.

Take. It. Out. NOW!

Your site shouldn’t see traffic from any browsers that don’t support compression (unless they’re robots and then, oh well!) so having rules that might accidentally deny compression might cause troubles. Turn the old security ACL rule around for HTTP compression:

Allow everything, then explicitly disable compression.

That should help prevent any accidents. Or higher bandwidth bills due to IE traffic.

OCSP and the GoDaddy Event

2012-09-18 / / 0 Comments

The GoDaddy DNS event (which I wrote about here) has been the subject of many a post-mortem and water-cooler conversation in the web performance world for the last week. In addition to the many well-publicized issues that have been discussed, there was one more, hidden effect that most folks may not have noticed – unless you use Firefox.

Firefox uses OCSP lookups to validate the certificate of SSL certificates. If you go to a new site and connect using SSL, Firefox has a process to check the validity of SSL cert. The results are of the lookup cached and stored for some time (I have heard 3 days, this could be incorrect) before checking again.

Before the security wonks in the audience get upset, realize I’m not an OCSP or SSL expert, and would love some comments and feedback that help the rest of us understand exactly how this works. What I do know is that anyone who came to a site the relied on an SSL cert provided and/or signed by GoDaddy at some point in its cert validation path discovered a nasty side-effect of this really great idea when the GoDaddy DNS outage occurred: If you can’t reach the cert signer, the performance of your site will be significantly delayed.

Remember this: It was GoDaddy this time; next time, it could be your cert signing authority.

How did this happen? Performing an OCSP lookup requires a opening a new TCP connection so that an HTTP request can be made to the OCSP provider. A new TCP connection requires a DNS lookup. If you can’t perform a successful DNS lookup to find the IP address of the OCSP host…well, I think you can guess the rest.
Unlike other third-party outages, these are not ones that can be shrugged off. These are ones that will affect page rendering by blocking the downloading the mobile or web application content you present to customers.

I am not someone who can comment on the effectiveness of OCSP lookups in increasing web and mobile security. OCSP lookup for Firefox are simply one more indication of how complex the design and management of modern online applications is.

Learning from the near-disaster state and preventing it from happening again is more important that a disaster post-mortem. The signs of potential complexity collapse exist throughout your applications, if you take the time to look. And while something like OCSP may like like a minor inconvenience, when it affects a discernible portion of your Firefox users, it becomes a very large mouse scaring a very jumpy elephant.

Web Performance: Your opinion is only somewhat relevant

2012-04-26 / / 0 Comments

Context is everything. Where you stand when reading or watching something shapes the way you experience it. Just as Einstein explained to us in the Train/Platform Thought Experiment, the position of the observer dictates how the event is described and recorded.

There is no difference with web performance. When a company develops an online application and presents it to customers (it doesn’t matter if they are outside/retail or inside/partner/employee), the perspective of the team that approved, created, tested, and released the application becomes, as a VP at a previous company explained to me, “interesting, but irrelevant”.

Step away from the world of online application performance for a minute, and put yourself in the shoes of the customer; become a consumer. How do you feel when a site, application, or mobile app is slow to give you what you want? I’ll give you some idea:

The stress levels of volunteers who took part in the study rose significantly when they were confronted with a poor online shopping experience, proving the existence of ‘Web Stress’. Brain wave analysis from the experiment revealed that participants had to concentrate up to 50% more when using badly performing websites, while EOG technology* and behavioural analysis of the subjects also revealed greater agitation and stress in these periods. (“Web Stress: A Wake Up Call for European Business”, emphasis mine)

I know it comes from a competitor, but it is true. It applies to me; it applies to you. And web performance professionals need to step away from the screens for a minute and put themselves in the shoes of the people standing on the platform.

Everyday, your online applications change, grow, fail, falter, and evolve – the train is always moving. To the people on the platform, all they see is your train and how it’s moving compared to the other trains they have watched go by. You worked hard on your train, polishing the brass, adding new cars, even upgrading the engine. To you, the train is a magnificent achievement that everyone should admire, especially now that the new engine makes it so much faster!

The customer on the platform is measuring how your updated train is moving compared to the MAGLEV bullet train on the super-conducting rail next to you and asking “How come this train is so slow?”

The complexity of a modern web site is astounding, and improving performance by 0.4 seconds is often a feat worthy of applause…among web performance professionals. From the perspective of your customers, that 0.4 second improvement is still not enough.

Web performance is a numbers game. As an industry, we have been focused on one set of numbers for too long. The customer experience, not the stopwatch, has to drive your company to the next level of performance maturity. To do that, you have to step off your online application train and take a cold hard look at what you deliver to your customers, alongside them down on the platform.

The Three Pillars of Web Performance

2012-01-24 / / 0 Comments

Had a great conversation with a colleague today. She and I were bouncing around some ideas, and I listed my top 3 topics in Web performance as “Speed, Revenue, and Experience”. She was quick to correct me.

“No, not revenue, conversions”.

She was right. Just last week, I talked about how critical it is to convert visitors into customers. Doing this in some businesses doesn’t mean that there is any revenue, but the goal remains the same.

Speed is the one everything thinks is the same as Web Performance. It’s not. It’s the don’t be that guy measure of Web Performance, the one that can be easily quantified and put on display. But performance for an online application is so much more than raw speed.

Experience is the hardest of the three to measure, because what it is depends on who you ask. Is it design, flow, ease of use, clarity, or none of these things? But a fast application can still make people cranky. There are online applications that are clearly designed to make the customer do things the way the vendor demands and these are the ones that make you go “Why am I here?”.

Now, can all the metrics that measure Web Performance be distilled to Speed, Conversions, and Experience? If you stepped away from the very product specific terms the Web Performance industry uses every day, what would describe the final, bottled, and served essence of Web Performance?

Customer Experience: Standing on your own four legs

2012-01-20 / / 0 Comments

Tables. They’re pretty ubiquitous. You might even be using one right now (although in the modern mobile world, you may not. LAMP POST!).

A strong business is like a table, supported by four legs.

  • The Business. The reason that resources and people have been gathered together. There is a vision of what the group wants to do and what success looks like.
  • The Design. Don’t think style; think Design/Build. This is where the group takes the business idea and determines how they will make it happen, where the stores will be, what a datacenter looks like, who they will partner with.
  • The Presentation. How the Business and the Design are shown to people. How the shelves are stocked, the landing pages look, the advertising is placed, how the business looks to potential customers.
  • The Delivery. This is the critical part of how the business uses the systems they have designed and the presentation they have crafted to deliver something of value to the potential customer.

Without any one of these, an organization will fail to meet the most critical goal it has set to be successful: an experience that turns a visitor or browser into a customer.

All the Business and MBA grads in the audience are yawning, and slapping their Venti non-fat, no-whip, decaf soy lattés down on the table. This message isn’t for you. Well, it is, but you can stand up and give your chair to one of the people behind you.

Now that I have Dev, QA, and Operations sitting with me (remember, the Business guys are still in the back of the room, tapping away on their Blackberries), tell me what you think of this conceptual table. How does the Table of Customer Experience relate to you?

Ok, put down the Red Bulls and Monsters and listen: Everything that Dev, QA, or Operations does has an effect on the experience (negative or positive) of the potential customer. If one of the table legs is broken (or even shorter than the others), the rippling shockwaves will eventually affect the entire operation.

So, if I were to ask the member so of your organization how their daily activities supported the online application in each of these four areas, do you think they could answer?

Grab a white board. This is going to be a long day.

Picture courtesy of sashafatcat

The Nomenclature Problem (or "What's in a name?")

2012-01-20 / / 1 Comment

Someone walks into your store. They say hello, poke around the racks, ask a few questions. Then they walk out.

Now, if I asked you, how would you describe that person?
Customer? Visitor? Yes?

I have been asking this question in preparation for some session for a group of motivated partners and employees in Singapore and Bangalore. As I prepare the presenter slides (not the dense textbook slides the participants get – thank you Garr Reynolds!), I keep correcting the words, typing customer to describe a visitor who is not.

When you and your teams discuss deep topics like conversion rates and transaction abandonment (WAKE UP! NO MEDITATION!) does the group classify non-buying, real people as  customers or visitors?

The label customer should be reserved for those visitors who complete the transaction and provide the revenue/information to the company whose online application they are interacting with. This means that the customer is the visitor who has bought into the entire online application experience.

A visitor becomes a customer only when they are happy with:

  • The Business
  • The Design
  • The Presentation
  • The Delivery

Where in the four areas has your application let the company down before?
If you asked a random visitor why they haven’t become a customer, what do you think the typical answer would be right now? Next week? A year from now?
Then ask your parents (or your spouse, if you’re brave) to use your application. You must show incredible restraint during this exercise (I suggest a remote operated camera and 6,000 miles of separation) to stop yourself from leaping in and telling them what to do,  shaping their experience and guiding them to your expected and desired outcome.

Can they do it? Would your parents or spouse become a customer?
When you look at your online applications tomorrow, use beginners mind to truly look at what you are doing in the four key areas. If you find yourself shaking your head and saying that this doesn’t make sense, put yourself in the visitors’ shoes.
You may ask yourself if the application you provide to support your business is truly improving the visitor experience.  What you have a strong chance of finding is that your application is designed for customers at the expense of visitors.

When a visitor doesn’t complete the tasks you defined for them to reach the goal of becoming one of your customers, what do you call them?

And do you know what to do next?

Does Traditional Web Performance Still Matter?

2011-12-28 / / 6 Comments

More than two years ago, I created a post that was frank in its statement that Web performance measurement isn’t just a technology issue, it’s a business issue.

As we approach 2012, a new question is driving how I examine the world I work in: Does traditional Web performance still matter?

Seems drastic and will raise the ire of more than a few folks I know, but it is a valid point of discussion. The entire Web performance industry needs to look around and determine how they got where they are and what the world will look like in 5 years.

The “Web” as it was defined when I started in the industry was simple – browser and page driven, with a growing focus on delivering services to visitors. Now, there is no definition of “Web” that can encompass everything that can be used when talking to companies. And in many cases, if asked, companies may not fully understand how customers interact with their online properties on a daily basis.

I used to be able to say what determined fast Web performance. Now, the simple answer is irrelevant, replaced with the reality of “It depends”. Fast is completely dependent on what is being done, when and where is it happening, how things being done, and who is driving the way it is is done.

I am issuing a challenge to the entire Web performance industry: Step back and and ask yourself if we are asking and answering the right questions for the companies we work with.

If we don’t find out now, in 5 years it won’t matter.

The Joy of the Platform

2011-07-26 / / 0 Comments

In the last few months, I have found myself uttering the word platform on an almost daily basis. As I was flying home last night, I began to consider what that actually means.

In the world I work in, customers bought a product or a tool. The purchase is driven by a desire to solve a problem or prevent a problem from appearing in the first place. It was a point solution, a single point of entry into an organization and added a very limited amount of value to a siloed compartment of an organization for a limited period of time, before the next shiny toy came along that purported to do the same thing, only better.

Economies of scale be damned, full inefficiencies ahead!

Companies that sell platforms, or have begun to to consider doing more than just paying lip-service to the word, look at the world with a different filter. The customer is seen as a holistic entity, as complex as any patient who comes to a doctor for treatment. If two people come to a doctor with the flu, they don’t always get the same treatment, as the one patient may be sent home for rest and the other rushed to hospital because their compromised immune system means the flu will kill them without specialist care.

The best platforms are those that are focused on one to three key aspects of customers business or way of doing business and provide a unified way to perform these 1-3 functions. The customer should not be forced to go to completely different places to use each tool on the platform.

Platforms have unified flows, and customers can expect that using different parts of the platform will be easy to learn, as they all work the same way. An example of a bad platform is Microsoft Office. When you go to the File Menu in a Microsoft Office product, you know that regardless which product in the suite you are using, the same items will be there. Where Microsoft Office fails as a platform is in the way that the rest of the menus and actions are not unified, with Powerpoint behaving differently from Word, which are both different from Excel. Microsoft Office is a the case study of history is getting in the way of ease of use, of standalone products loosely linked, like cheap knock-offs of Lego™.
Platforms are truly extensible. If a customer needs an additional component of the platform, it can be enabled (after the appropriate business negotiations) in minutes.

Platforms need to allow simplicity when needed and complexity where required. While a 10-person company and a 10,000-person company have different needs, the same platform should be able to support these needs. Salesforce.com is a classic example of this – in their world, they don’t care what the size of your company is.

And, platforms have to guided by product management teams, to have a shared vision. Product management has to enforce a strong adherence to the core values of focus, unity, extensibility, and complex simple complexity. A product management team that lacks the leadership to drive these values will produce a broken platform

How does your platform compare against this checklist?

Copyright © 2024 Performance Zen

Theme by Anders Noren — Up ↑