Performance Zen

Author: spierzchala

  • Black Friday 2008: The pain, the horror, the suffering

    The GrabPERF Black Friday Dashboard is done for another year and there were two performance victims that suffered the most at the hands of the onslaught of bargain-hunters in the area of Web performance.

    Some caveats that I need to mention about the GrabPERF measurement methodology.

    1. Only the base HTML file of each site is measured.
    2. Only the base HTML of the homepage is measured. This means that any issues that arose in the shopping process were not captured.

    All of the sites in the GrabPERF Holiday Retail Measurement Index can be continually monitored on the GrabPERF Black Friday Dashboard. This page will be available until January 1 2009.

    That said, the two primary performance victims this year are HP Shopping and Sears. We focus here on those that did not do that well because sites who have met the Web performance challenge and survived to fight another year are not as interesting from a learning perspective.

    HP Shopping

    hp-shopping-blackfriday-2008

    HP Suffered the greatest response time problems, by effectively becoming unresponsive as of 09:00 EST. The greatest affect on overall response time came as a result of the First Byte time metric which is a solid proxy for measuring the server or application load, as it is the time between the initial client HTTP request and the server’s HTTP response.

    Factored into the poor performance analysis is the fact that GrabPERF only captures data for the base HTML object. If the performance seen here is carried over to the download of all of the graphical content on the page, I would be surprised if anyone was able to make any kind of purchases on the HP web site on Black Friday.

    Today, performance has returned to substantially lower levels, indicating that this application was simply not ready for the amount of traffic it received, or ran into a completely unexpected issue when the load increased.

    Recommendation for 2009: Load Test the application using this year’s traffic metrics as a baseline for validating the scalability of the application.

    Sears

    sears-shopping-blackfriday-2008

    Sears is a returning visitor from last year’s Black Friday measurements. Unfortunately, they return for exactly the same reason that they were on last year – scaling/capacity issues that appear as errors.

    And these are the worst kind of errors. As can be seen in the graphic below, the Sears Web site announced to the whole world that they had over-reached and that they could not handle the incoming volume of traffic.

    sears-error-image-blackfriday

    What is interesting is that Sears owns properties that survived the day very well, namely Lands End. The question that must be posed is why does the parent site fail so badly when the child sites handle the traffic without difficulty?

    Recommendation for 2009: Load testing for capacity, and meeting with the Lands End team to understand what they are doing to handle the load.

  • Why Terms Matter: Consultant v. SME v. Evangelist

    The term consultant is bandied about so much in this new economy that it has lost it’s meaning. Wikipedia defines a consultant as

    A consultant (from the Latin consultare means “to discuss” from which we also derive words such as consul and counsel) is a professional who provides advice in a particular area of expertise….

    A consultant is usually an expert or a professional in a specific field and has a wide knowledge of the subject matter. A consultant usually works for a consultancy firm or is self-employed, and engages with multiple and changing clients. Thus, clients have access to deeper levels of expertise than would be feasible for them to retain in-house, and to purchase only as much service from the outside consultant as desired.

    http://en.wikipedia.org/wiki/Consultant

    What this definition misses is that a good consultant, especially in a small firm, is not simply a person with specific subject-matter expertise and therefore a subject-matter expert (SME), a consultant is a jack-of-all-trades.
    A simple list of skills needed by a good consultant include:

    • Sales
    • Project Management
    • Product Management
    • Educator
    • Trainer
    • Mentor and Coach
    • Business Manager
    • Subject-Matter Expert

    In large consulting organizations, these functions are broken out into specific team members. In a small consultancy, everyone has to be able to manage all of these items.

    Then there is another leap: How does a consultant move to being an evangelist? These two roles are substantially different.

    While both are SMEs, an evangelist takes that one final step from being a functional expert who is able to make things happen and work in a product to a place where they can stand in front of any audience and make the product sing. It is not just able the abilty to do anymore; it is about the ability to show.

    Go through the list of people that you or your organization work with. Do you work with true consultants, SMEs, or evangelists? Which group is most effective in helping your organization get better? Are you using consultants as expert problem-solvers, or are you simply using them as staff augmentation?

    To draw on my experience, I am learning to be a better small-firm consultant. I have developed my skills as a SME and Evangelist over the last decade, but I have not had to be worried about any of the things listed above until the last two years when I started working in a more structured consulting/Professional Services environment.

    What has your experience been? Did you start as a SME and become a consultant?

    Or did you come out of B-school and then develop into a SME?

    How has your development as a consultant affected the clients you have worked with and experiences you have had?

  • UAL – Thank you for flying, but to hell with your Premier Status.

    Flying back from SFO after a long and frustrating week introduced me to a new rule that UAL gate staff have been asked to start enforcing. Apparently, my Premier status, which I realize is the lowest of the frequent-flyer levels, means even less now than it did in the past.
    Over my career, I had settled on UAL as my carrier of choice. Flying out of SFO for the first 4.5 years in the US meant that UAL was the primary choice to get anywhere. After a while, I became a devoted UAL fan when I realized that in this day of limited overhead bin room having Premier status got you the vaunted 1 on your boarding pass.
    I could accept that First-Class and 1K flyers got to board ahead of me – hell, they’re on a first-name basis with most of the flight crews. This didn’t bother me because I knew that I got to board next.
    Friday, that changed.
    Apparently, the rule is that Premier Executive now rates between the Red Doormat Club and the Premier status flyers.
    I have commented in the past about how people who travel a great deal assume too much from their airline frequent-flyer plans. I do not want to become one of these people. All I ask is that this single privilege I had grown accustomed to having be re-instated. I know my travel money doesn’t have a huge effect on your bottom-line, but I stuck with you through thick and thin.
    But now this is a really thick move, and my patience has grown thin.

  • Happy….?

    It’s November 14 2008. Charles Windsor is 60 today.
    Forty – Oh yeah?http://seesmic.com/embeds/wrapper.swf

  • GrabPERF: FiOS and BitTorrent – Don't Play Nice

    I fired up the Boston FIoS measurement location today after a couple of days off, and found that suddenly FIoS doesn’t like the BitTorrent.

    The line of purple dots all indicate measurements that reported an error code. All of those measurements come from Boston FiOS. See the real-time graph here.
    Accident? Design? That I cannot comment on. I simply report on what I see.

  • GrabPERF: Three New Measurement Locations

    In the last 24 hours, thanks to the help of some willing volunteers, GrabPERF has seen the addition of three new measurement locations:

    • Dallas, TX (USA)
    • Virginia (USA)
    • London, UK

    All of these location have been graciously provided by the team at e-planning.
    Thanks to all of you who volunteer your machines and bandwidth for this project.
    As always, we are looking for as more measurement locations. It would be great if we could get some data from the Asia-Pacific region.

  • Two Weeks with the MacBook

    My new MacBook arrived two weeks ago, and I felt that I had spent enough time with it to actually make some useful comments on the good, the bad, and the headbangingly frustrating.

    The Finder

    Dear Apple: Shoot the Finder development team. Thanks.

    I have switched to Path Finder as a Finder replacement. Truly the finder is one of the most debilitating pieces of software I have ever used. Nautilus on Gnome is a far superior file management system.

    Software, in general

    On the whole, I have found replacements for most of the Windows tools I use on a regular basis. But, as I am not made of money, I am using GIMP for Mac, and that is just clunky in the X11 environment.

    Living in the browser makes my life much more tolerable than those who require the Windows environment. I haven’t got the money to buy Parallels or VMWare Fusion right now, so I am using RDC to connect to my Windows box. Slap Windows in Space #3 Fullscreen, and no one would know the difference.

    Haven’t found a good Mind Map tool. And BBEdit is also muchos dineros. So Smultron is the text editor.

    Usability

    I rate this very high. Other than adjusting to the lack of certain keys (DEL, Pg up/dn, etc), the transition has been seamless. The trackpad is a dream and I miss being able to throw stuff around on my Dell laptops’ trackpads like I can using the one on the MacBook. I do find I leave apps hanging, as I am still adjusting to CMD-q closing the app.

    Dashboard. What can I say? It’s what I need – high-level data at a glance, including the Prem Tables!

    Overall

    After four years waiting for a MacBook, I can say that it has been worth the wait. Solid, dependable, and slowly becoming my primary computer.
    The only concern that I have is the aluminum case. I have an aluminum sensitivity, and if my hands start to peel and otherwise be in bad shape, I will have to determine a solution to that issue.

  • House – Personal Ad

    Have I mentioned that Greg House is one of my mentors?

    Hugh Laurie
    more lol celebs!

  • Moving from Windows – My First Week With Ubuntu (Hardy Heron)

    For the last week, I have been using Ubuntu 8.04 (Hardy Heron) on my personal laptop. I can say that the experience has been mostly transparent for me, even with the need for a complete re-build last night after an attempt to install a complex theme replacement.

    I can say that it has been transparent because I have been using Linux desktops in one form or another on an intermittent basis since 1999. When business was slow in the Fall and Winter of 2001/2002, I was the Guinea Pig in my organization to see if Linux could be a corporate replacement for Windows for all desktops and laptops.

    So, when I say that the process has been transparent, you will have to realize that I have been a technical user of these desktop interfaces for a number of years. But I can say that since my first positive experiences with the Red Hat Fedora and the Ximian Gnome replacement interface, things have come a very long way.

    Ubuntu 8.04 is the first real interface that seems to work predictably, efficiently, and effectively with external devices and programs that are business friendly. This is especially the case if most of the tools are Web-based, as Firefox and Opera work seamlessly. OpenOffice 2.4 can open DOCX files, and media players support most of the files I want to watch/listen to.

    It prints to the home network printer.

    It accesses the home file server.

    I can share and synchronize files among my computers using DropBox.
    Some caveats to my positive experience.

    • I work mainly on the Web
    • I do not play games
    • I have been using Linux in various forms and editions since 1999.

    If you have technically savvy friend, or really want to push and expand your knowledge of computers and highly configurable operating systems, I would definitely suggest giving Ubuntu a try on the extra computer you have lying around. My laptop is at least 3.5 years old, and not anywhere near as fast as my work laptop running XP. However, with Linux, the two are comparable in speed and performance.

    Go on. Try it. I know you want to.

  • Web Performance: Nice Display. Now Show Me the Data.

    Today’s Web interfaces are all about the Flash (literally). Smooth charting, cool effects, callouts to references — ways to try and simplify complex data collections.
    Problem-solving and diagnosis requires a far deeper dive than the flashiest interface could ever provide, because it comes down to the numbers. The actual measurements that make up the flashy chart. If you look at the data used by a professional trader and a someone at home looking at stock charts, there is a substantial difference.

    When you get down to that level of analysis, the interface becomes irrelevant. Any analyst worth her or his salary (or salt – same thing) can tell you more from a spreadsheet full of relevant numbers than they can from any pretty graphic. This is true in any field.

    When do traders or Web performance analysts use pretty charts? When they have to explain complex issues to non-technical or non-specialist audiences. When these analysts work on solving the sticky problems faced in the everyday world, they always fall back on the numbers.

    Web performance data consists of the same few components, regardless of which company is providing the data. In effect, beyond a few key pieces of information about how the measurement data is captured, all Web performance data is the same.

    Just because the components that make up the data are the same does not guarantee that the data from two different providers is of the same quality. In an imaginary system, Web performance data from all the major providers could flow into a centralized repository and be transformed using an XSLT or some other mangler so that it would be indistinguishable in most cases to tell which firm was the source.

    But a skilled analyst would quickly learn to recognize the data that can be trusted. That would be the data that quickly and accurately represented the issues he was trying to diagnose. The data that flowed with the known patterns of the Web site.

    The data that helped him do his job more effectively.

    In the end, a pretty interface can go a long way to hide the quality of the data that is being represented. A shiny gloss on poor data does not make it better data. It is critical that the data that underlies that pretty chart is able to live up to the quality demands of the people who use it every day.

    Selling the interface is selling the brand. Trust in the data builds the reputation.
    Which one sold you when you chose your Web performance measurement provider?