New Server – New Fun!

This is the amazing “new” server that this blog is hosted on. Using a dynamic DNS service and Ubuntu on a 9 year-old MacBook, I have taken control of my blog again, trying to revive it from the untouched archive it has been for the last few years.

[Oh, and if you have arrived here from a Newest Industry link, don’t panic. Just search for the article you want – they’re all here.]

I’ve just recently changed companies (through an acquisition), so I will have new web performance items to hook into as I get a chance to work with new customers.

All raise your glass – PerformanceZen is back.

Performance Trends for ???? – Smarter Systems

IF-repair by Yo Mostro (Flickr)Most of the trending items that I have discussed in the last two weeks are things that can be done today, problems that we are aware of and know need to be resolved. One item on my trend list, the appearance of smarter performance measurement systems, is something the WPO industry may have to wait a few years to appear.

A smarter performance measurement system is one that can learn what, when, and from where items need to be monitored by analyzing the behavior of your customers/employees and your systems. A hypothetical scenario of a smarter performance measurement system at work would be in the connection between RUM and synthetic monitoring. All of the professionals in WPO claim that these must be used together, but the actual configuration relies on humans to deliver the advantages that come from these systems. If RUM/analytics know where your customers are, what they do, and when they do it, then why can’t these same systems deploy (maybe even create and deploy!) synthetic tests to those regions automatically to capture detailed diagnostic data?

Why do measurement systems rely on us to manually configure the defaults for measurements? Why can’t we take a survey when we start with a system (and then every month or so after that) that helps the system determine the what/when/where/why/how of data and information we are looking to collect and have the system create a set of test deployment defaults and information displays that match our requirements?

The list of questions goes on, but they don’t have to. Measurement systems have, for too long, been built to rely on expert humans to configure and interpret results. Now we have a chance to step back and ask “If we built a performance measurement system for the a non-expert, what would it look like?”

More data isn’t the goal of performance measurement systems – more information is what we want.

Performance Trends 2013 – Employees are Customers too

Peter Levine from Andreesen Horowitz wrote an article on The Renaissance of Enterprise Computing yesterday that finally sprouted the seed of an idea that has been dormant at the back of my brain for a few months. While the ideas of enterprise computing and web/mobile performance seem disconnected, they’re not.

When companies begin to rely on outside services (Levine mentions Box, Google Docs, and others in his article) they have given part of their infrastructure over to an outside organization. And, when you do that, this means that any performance hiccups that affect us as consumers can have a very major effect on us as employees.

Even if your company decides to purchase and deploy an enterprise application within your own infrastructure or datacenters, the performance and experience that your employees experience when using it on their desktops or on their mobile devices can affect productivity and effectiveness in the workplace. An unmanaged (read unmonitored) solution can have shut down groups in the company for minutes or hours.

Think of the call-center. No matter the industry you’re in, what increase customer calls: slow performance or a poor experience with the web/mobile application. Now, if your employees rely on a variant of the same web application to answer questions in the call-center, have you actually improved the customer experience and increased employee productivity?

Some considerations when managing, designing, or buying an enterprise application in the coming  year:

  • What do your peers tell you about their experience implementing the solution or using an outside service – has it made employees more effective and efficient?
  • Are employees already using a “workaround” that makes them more effective and efficient? Why aren’t they using the internal or mandated solution?
  • Is performance and experience a driving factor in the lack of adoption of the mandated solution?
  • Do you have clear and insightful performance information that shows when employees are experiencing issues performing critical tasks? Can you clearly understand what the root cause is?
  • Are employees experiencing issues using the application in certain browsers or on certain mobile devices? How quickly can your design or your outside service respond to these issues?
  • Are you reviewing the chosen solution regularly to understand how usage is changing and how this could affect the performance of the application in the future?

Performance issues are not simply affecting the customers you serve. Your own employees use many of the same systems and applications in their day-to-day tasks, so a primary goal of managing these application should be to ensure that the applications deliver performance and experience that encourages employees to use them, no matter whether they are developed in-house or purchased as software or SaaS.

Web Performance Trends 2013 – Third Party Services

Every site has them. Whether they’re for analytics, advertising, customer support, or CDN services, third-party services are here to stay. However, for 2013, I believe that these services will face a level of scrutiny that many have avoided up until now.

Recent performance trends indicate that while web site content has been tested and scaled to meet even the highest levels of traffic, the third-party services that these sites have some to rely on (with a few exceptions) are not yet prepared to handle the largest volumes of traffic that occur when many of their customers experience a peak on the same day.

In 2013, I see web site owners asking their third-party service providers to provide verification that their systems be able to handle the highest volumes of traffic on their busiest days, with an additional amount of overhead – I suggest 20% – available for growth and to absorb “super-spikes”. Customer experience is built on the performance of the entire site, so leaving a one component of site delivery untested (and definitely unmonitored!) leaves companies exposed to brand and reputation degradation as well as performance degradation.

In your own organizations, make 2013 the year you:

  • Implement tight controls over how outside content is deployed and managed
  • Implement tight change control policies that clearly describe the process for adding third-party content to your site, including the measurement of performance impacts
  • Define clear SLAs and SLOs for your third-party content providers, including the performance levels at which their content will be disabled or removed from the site.

When speak to your third-party content and service providers about their plans for 2013, ask them to:

  • Explicitly detail how they handled traffic on their busiest days in 2012, and what they plan to do to effectively handle growth in 2013
  • Clearly demonstrate how they are invested in helping their customers deliver successful mobile sites and apps in 2013
  • Lay out how they will provide more transparent access to system performance metrics and what the goals of their performance strategy for 2013 are.

Take control of your third-party content. Don’t let it control you.

Effective Web Performance – Will A CDN Really Solve The Problem?

Content Delivery Networks (CDNs) have been available for over a decade, with companies leaning on these distributed networks to accelerate the delivery of their content, absorb the load of the peak traffic periods, and help deal with the problem of geography. During my career I have helped companies assess and evaluate CDN solutions so that they understood the performance improvement benefit they would receive for the money they were spending.

But a something bothered me during these evaluations. I felt that companies were rushing into the decision because it was the “right” thing to do, that their site could only get faster if they used a CDN. This rush often left question unasked and unanswered.

In this post, I want to provide 4 questions that you should ask before deploying a CDN, questions that will help you ensure that a CDN is the performance improvement solution you need right now.

Is the performance issues due to slow application components?

If the performance issue can be tracked down to application elements, then a CDN is likely not the immediate starting point for you and your team. If generating dynamic content or database lookups are causing the performance issue, this is on you, in your datacenter.

CDNs do not make slow applications run faster; CDNs move bits to customers faster more efficiently.

How are we measuring page response times?

If you are measuring response times using the load time of the entire page, then you may be inflating your response times by including all of the third-party content that has been included. You may want to set up parallel measurements that allow you to compare the full page with a page measurement that only includes the content you are directly responsible for managing. If you find that third-party content is slowing down your pages, then a CDN can’t help you.

Full page load times only give you one performance perspective. Modern performance measurement teams need to understand how long it takes for a page to be ready and usable for the customer – the perceived rendering or page ready time. What you could discover is that customers can do 90% of what they want on your page well before all of the content fully loads. If this is true, than the perceived load time for your company to determine that a CDN isn’t necessary right now.

Have we done everything to optimize our own performance?

Often companies choose the easy way (CDN) to solve a hard problem (improve performance). Unfortunately, the easy way can be more expensive than taking the time to design pages and content that ensure long-term and sustainable performance. A CDN could mask a bad design until it slows down so much that even the CDN can no longer help.

Measure your page and challenge the devops team to tweak the exiting design until they don’t think they can get any more performance out of it. Then, during your CDN evaluations, set up measurements that compare the performance of origin and CDN delivered pages. You might find that making the pages as fast as you possibly can before purchasing the services of a CDN make the difference between the origin and CDN times less dramatic than they would have been before optimization.

Can we effectively estimate if the CDN cost will be offset by an increase in revenue?

CDNs don’t come cheap, so choosing to deploy a CDN better pay for itself over time. Challenge the CDNs you are evaluating to present you with ROI calculations that show how their service more than pays for itself and case studies (or customers) that prove that the cost of the CDN was offset by a business metric that can be easily tracked.

Summary

Don’t mis-understand the questions I am posing here: I am still a strong advocate for the use of CDNs. However, the days of companies simply purchasing the services of a CDN because it’s the “right thing to do” are long over.

As companies evolve their perspective on web and mobile performance, they need to ensure they have done everything they can to make their own applications faster. Once the hard work of tuning and optimization is complete, the process of choosing a CDN must include deeper, more probing questions about the performance and business benefits that come along with the service.

 

Managing Performance Measurement: Who uses this stuff anyway?

Clogged Pipe - staale.skaland - FLICKROne of the least glamorous parts of managing performance measurement data is the time I have to take every month to wade through my measurements and decide which stay on and which get shut off. Since I’m the only person who uses my measurement account, this process usually takes less than 10 minutes, but can take longer if I’ve ignored it for too long.

With large organizations that are collecting data on multiple platforms, this process may be more involved. By the time you look at the account, the tests have likely accumulated for months and years, collecting data that no one looks at or cares about. They remain active only because no one owns the test and can ask to disable it.

What can you do to prevent this? Adding some measurement management tasks to your calendar will help prevent Performance Cruft from clogging your information pipes.

  1. Define who can create measurements. When you examine account permissions on your measurement systems, do you find that far more people than are necessary (YMMV on this number) have measurement creation privileges? If so, why? If someone should not have the ability to create new measurements, then take the permissions away. Defining a measurement change policy that spells out how measurements get added will help you reduce the amount of cruft in your measurement system.
  2. Create no measurement without an owner. This one is relatively easy – no new measurement gets added to or maintained on any measurement system without having one or more names attached to it. Making people take responsibility for the data being collected helps you with future validations and, if your system is set up this way, with assigning measurement cost to specific team budgets. It’s likely that management will make this doubly enforceable by assigning the cost of any measurement that has no owner to the performance team.
  3. Set measurement expiry dates. If a measurement will be absolutely critical during  only a specific time range, then only run the measurement for that time. There is no sense collecting data for any longer than is necessary as you have likely either stored or saved the data you need from that time for future analysis or comparisons.
  4. Validate measurement usage monthly or quarterly. Once names have been associated to measurements, the next step is to meet with all of the stakeholders monthly or quarterly to ensure that the measurements are still meaningful to their owners. Without a program of continuous follow-through, it will take little time for the system to get clogged again.
  5. Cull aggressively. If a measurement has no owner or is no longer meaningful to its owners, disable it immediately. Keep the data, but stop the collection. If it has no value to the organization, no one will miss it. If stopping the data leads to much screaming and yelling, assign the measurement to those people and reactivate.

Managing data collection is not the sexiest part of the web performance world, but admitting you have a data collection cruft problem is the first step along the path of effective measurement management.

Pain at Every Level – Web Performance in the Organization

People in every organization are happy (in an unhappy way) to tell you exactly what their level of Web performance pain is. They go into great detail on how every performance issue affects them and and why it makes every day an unpredictable and almost unmanageable challenge.

If you take the personal perspective of Web performance pain, the risk not finding the real problem, the true cause of the pain.

Talking to customers at all levels of organizations has shown that when you ask “where it hurts”, they can tell you exactly what they want you to work on. And once you solve that problem, you get another person from the same organization with a different pain coming to you, complaining that you have ignored them.

A whole-organization focus is required when working to solve a customers Web performance pain. And it starts by asking questions of everyone in a company, not just the one who came to you for the initial diagnosis. Different groups at different levels have different questions.

Here’s a (very basic) list of some of those that you should be prepared to answer as you work to diagnose a company’s Web performance issues.

C-Level

  • How am I doing against my competitors?
  • How does performance affect my revenue?
  • If I want to use the Web for more revenue, what do I need to do to make it work?
  • How does Mobile deliver what I need?
VP, Operations
  • How much will it cost me to deliver the necessary Web performance?
  • What is critical for me to deliver now, and what can I delay until the next budget cycle?
  • How do I ensure that Web performance issues don’t affect revenue?
  • Are my partners helping or hindering us?
  • How do I get Marketing to the table to understand the technology boundaries we have?
VP, Marketing
  • How do I effectively use the Web without alienating customers with slow performance?
  • How do I ensure that our design is delivered appropriately to both fixed-Web and mobile users?
  • What parts of the site are customers unsatisfied with due to performance?
  • Do my promotions scale to handle the surge in customers?
  • How do I get Operations to understand that delivering new experiences with leading-edge technology is critical for us to be successful?
Director, Operations
  • I spend most of my time on troubleshooting conference calls. How can I reduce this drain on my time and resources?
  • My team spends most of its time trying to correlate data between 5 different systems. Help!
  • The latest design is putting a massive strain on our infrastructure. Didn’t anyone test this on the production servers before it went live?
  • I know that we need to take a load of our servers, but I don’t know how to choose a CDN. What do I need to do?
Operations Staff, NOC
  • Man, I get a lot of alerts. How do I tell which ones I need to care about?
  • This sure looks like a problem. How do I show the appropriate folks that this issue is their responsibility?
  • Most of the time, the issues I investigate are with one third-party. Who is responsible for fixing this and does it really affect customers?
  • I get bonused on fast MTTR. How can I figure out what the problem is faster?

In the sections above, notice that none of the questions need to be answered with product descriptions. Companies are desperate to understand not how other companies deployed the latest Kazoo to solve their Waka-waka problem, but how they made life easier and more manageable.

Coming to the customer with an open mind and a listening ear is the new hallmark of Web performance.

Web Performance Concepts: Customer Anywhere

Companies are beginning to fully grasp the need to measure performance from all perspectives: backbone, last mile, mobile, etc. But this need is often driven by the operational perspective – “We need to know how our application/app is doing from all perspectives”.

While this is admirable, and better than not measuring at all, turning this perspective around will provide companies with a whole new perspective. Measure from all perspectives not just because you can, but because your customers demand performance from all perspectives.

The modern company needs to always keep in mind the concept of Customer Anywhere. The desire to visit your site, check a reservation, compare prices, produce coupons can now occur at the customer’s whim. Smartphones and mobile broadband have freed customers from the wires for the first time.

If I want to shop poolside, I want your site to be fast over 3G on my Blackberry. I don’t care what the excuse is: If it’s not fast, it’s not revenue.

Knowing how a site performs over the wire, in the browser, around the world made “Web” performance a lot harder. The old ways aren’t enough.

How does your “Web” performance strategy work with Customer Anywhere?

Effective Web Performance: The Wrong 80 Percent

Steve Souders is the current king of Web performance gurus. His mantra, which is sound and can be borne out by empirical evidence, is that 80% of performance issues occur between the Web server and the Web browser. He offers a fantastically detailed methodology for approaching these issues.

But fixing the 80% of performance issues that occur on the front-end of a Web site doesn’t fix the 80% of the problems that occur in the company that created the Web site.

Huh? Well, as Inigo Montoya would say, let me explain.

The front-end of a Web site is the final product of a process, (hopefully) shaped by a vision, developed by a company delivering a service or product. It’s the process, that 80% of Web site development that is not Web site development, that let a Web site with high response times and poor user experience get out the door in the first place.

Shouldn’t the main concern of any organization be to understand why the process for creating, managing, and measuring Web sites is such that after expending substantial effort and treasure to create a Web site, it has to be fixed because of performance issues detected only after the process is complete?

Souders’ 80% will fix the immediate problem, and the Web site will end up being measurably faster in a short period of time. The caveat to the technical fix is that unless you can step back and determine how a Web site that needed to be fixed was released in the first place, there is a strong likelihood that the old habits will appear again.

Yahoo! and Google are organizations that are fanatically focused on performance. So, in some respects, it’s understandable how someone (like Steve Souders) who comes out of a performance culture can see all issues as technical issues. I started out in a technical environment, and when I locked myself in that silo, every Web performance issue had a technical solution.

I’ve talked about culture and web performance before, but the message bears repeating. A web performance problem can be fixed with a technical solution. But patching the hole in the dike doesn’t stop you from eventually having to look at why the dike got a hole in the first place.

Solving performance Web problems starts with not tolerating them in the first place. Focusing on solving the technical 80% of Web performance leaves the other 80% of the problem, the culture and processes that originally created the performance issues, untouched.

Effective Web Performance: Measurement-First or CDN-First?

A hallway conversation this morning brought up a very interesting point about the relationship between Web performance measurements and Content Delivery Networks (CDNs). When choosing between a Web performance measurement solution and a CDN, which service should come first?

Companies facing dire and obvious Web performance issues will want immediate results, leading them to fall into the CDN-First camp. Deploying a CDN will have a positive effect on response times, increase user satisfaction, and may even increase customer conversions, in the short term.

In six months, deeper questions may start to be asked. A core question that will need to be answered by CDN-First organizations will be “Are we using the CDN effectively and efficiently?“.

A company that makes the leap to CDN deployment without assessing the overall performance environment of their Web site may be faced with a situation where they can’t tell if they need more, less, or different CDN strategies in order to continue to succeed.

As a result of the buyers remorse that can result from the leap directly to a CDN, I highly recommend the Measurement-First approach when selecting a CDN.

To help you become an advocate for the Measurement-First approach, come to the table during the CDN discussions and ask three questions. The answers will allow your organization to make the best and most appropriate CDN decision.

1. Is the CDN necessary?

In most cases, the answer to this is a resounding yes. But what can happen with a sudden shift to the CDN is that a organization overlooks those things that they can do themselves to gain some initial performance improvements.

Baselining the existing site before deploying a CDN will allow items and elements that need to be improved to be clearly identified. In some cases, an organization can fix some of these on their own to improve performance before investing in a CDN. In other cases, measuring the performance of a site may clearly indicate that third-party content is responsible for the performance issues, which would likely not be fixed by a CDN deployment.

Measurement-First policy helps clearly identify the geographies that have the worst performance before deploying the CDN. If performance in the US is acceptable, while performance in Europe or Asia-Pacific is intolerable, then the CDN deployment may initially be targeted to respond to the greatest pain first.

Understanding the current performance of your existing site can reduce the cost of the initial deployment and maximize the the long term effectiveness of the deployment.

2. Which CDN is best for us?

For a complex modern Web site, content comes in many different shapes, sizes, and formats. The thing is, so do CDNs. As I’ve discussed before, understanding what the CDNs vying for your business do and do well is as critical as the process of vetting their effectiveness compared to delivering the site yourself. The performance boost given to you site by a CDN may vary by region, leading your team to select one CDN for Europe and another for the Asia-Pacific region.

CDN performance can also vary based on the content you are asking them to accelerate. One CDN may be good at streaming media, while another may be better at static content (JS, CSS, Images, etc.), while yet another is better at accelerating the delivery of dynamic content.

Choose your CDN(s) based on what you need them to deliver. In some cases, one size does not fit all.

3. Is the CDN delivering?

This may look like a question for after the purchase has been completed and the solution deployed, but you will never know if the solution is working effectively unless you have a baseline of your performance before the deployment, and from your origin servers after deployment.

Measuring the performance of the CDNs under all conditions and from all perspectives (Datacenter, Last Mile, and from within the Browser) doesn’t stop with the selection of a CDN(s). It becomes even more critical once the CDN solution(s) is rolled into production in order to ensure that the level of service that was promised during the sales cycle is delivered once you become a customer.

Constantly validate the performance of the CDN-accelerated site with the performance of the non-accelerated origin site. Have regular meetings with, and channels of communication into, your CDN(s) to discuss not only existing performance, but how changes you and/or the CDN provider are planning may affect performance in the future.

Takeaway

CDNs are a critical component for any Web business that wants to scale and deliver services to a national or global audience. But selecting a CDN should come after you have a very strong understanding of the current performance of your own Web site.

After you have measured and identified the items you can do to improve your own performance, your team will have greater insight into the areas of your site where the services of a CDN(s) can have the greatest impact.

The Measurement-First approach to selecting a CDN will ensure that you select a set of services that exactly meets the unique performance challenges of your site.