Performance Zen

Author: spierzchala

  • Pain at Every Level – Web Performance in the Organization

    People in every organization are happy (in an unhappy way) to tell you exactly what their level of Web performance pain is. They go into great detail on how every performance issue affects them and and why it makes every day an unpredictable and almost unmanageable challenge.

    If you take the personal perspective of Web performance pain, the risk not finding the real problem, the true cause of the pain.

    Talking to customers at all levels of organizations has shown that when you ask “where it hurts”, they can tell you exactly what they want you to work on. And once you solve that problem, you get another person from the same organization with a different pain coming to you, complaining that you have ignored them.

    A whole-organization focus is required when working to solve a customers Web performance pain. And it starts by asking questions of everyone in a company, not just the one who came to you for the initial diagnosis. Different groups at different levels have different questions.

    Here’s a (very basic) list of some of those that you should be prepared to answer as you work to diagnose a company’s Web performance issues.

    C-Level

    • How am I doing against my competitors?
    • How does performance affect my revenue?
    • If I want to use the Web for more revenue, what do I need to do to make it work?
    • How does Mobile deliver what I need?

    VP, Operations

    • How much will it cost me to deliver the necessary Web performance?
    • What is critical for me to deliver now, and what can I delay until the next budget cycle?
    • How do I ensure that Web performance issues don’t affect revenue?
    • Are my partners helping or hindering us?
    • How do I get Marketing to the table to understand the technology boundaries we have?

    VP, Marketing

    • How do I effectively use the Web without alienating customers with slow performance?
    • How do I ensure that our design is delivered appropriately to both fixed-Web and mobile users?
    • What parts of the site are customers unsatisfied with due to performance? Do my promotions scale to handle the surge in customers?
    • How do I get Operations to understand that delivering new experiences with leading-edge technology is critical for us to be successful?

    Director, Operations

    • I spend most of my time on troubleshooting conference calls. How can I reduce this drain on my time and resources?
    • My team spends most of its time trying to correlate data between 5 different systems. Help!
    • The latest design is putting a massive strain on our infrastructure. Didn’t anyone test this on the production servers before it went live?
    • I know that we need to take a load of our servers, but I don’t know how to choose a CDN. What do I need to do?

    Operations Staff, NOC

    • Man, I get a lot of alerts. How do I tell which ones I need to care about?
    • This sure looks like a problem. How do I show the appropriate folks that this issue is their responsibility?
    • Most of the time, the issues I investigate are with one third-party. Who is responsible for fixing this and does it really affect customers?
    • I get bonused on fast MTTR. How can I figure out what the problem is faster?

    In the sections above, notice that none of the questions need to be answered with product descriptions. Companies are desperate to understand not how other companies deployed the latest Kazoo to solve their Waka-waka problem, but how they made life easier and more manageable.

    Coming to the customer with an open mind and a listening ear is the new hallmark of Web performance.

  • The Joy of the Platform

    In the last few months, I have found myself uttering the word platform on an almost daily basis. As I was flying home last night, I began to consider what that actually means.

    In the world I work in, customers bought a product or a tool. The purchase is driven by a desire to solve a problem or prevent a problem from appearing in the first place. It was a point solution, a single point of entry into an organization and added a very limited amount of value to a siloed compartment of an organization for a limited period of time, before the next shiny toy came along that purported to do the same thing, only better.

    Economies of scale be damned, full inefficiencies ahead!

    Companies that sell platforms, or have begun to to consider doing more than just paying lip-service to the word, look at the world with a different filter. The customer is seen as a holistic entity, as complex as any patient who comes to a doctor for treatment. If two people come to a doctor with the flu, they don’t always get the same treatment, as the one patient may be sent home for rest and the other rushed to hospital because their compromised immune system means the flu will kill them without specialist care.

    The best platforms are those that are focused on one to three key aspects of customers business or way of doing business and provide a unified way to perform these 1-3 functions. The customer should not be forced to go to completely different places to use each tool on the platform.

    Platforms have unified flows, and customers can expect that using different parts of the platform will be easy to learn, as they all work the same way. An example of a bad platform is Microsoft Office. When you go to the File Menu in a Microsoft Office product, you know that regardless which product in the suite you are using, the same items will be there. Where Microsoft Office fails as a platform is in the way that the rest of the menus and actions are not unified, with Powerpoint behaving differently from Word, which are both different from Excel. Microsoft Office is a the case study of history is getting in the way of ease of use, of standalone products loosely linked, like cheap knock-offs of Lego™.
    Platforms are truly extensible. If a customer needs an additional component of the platform, it can be enabled (after the appropriate business negotiations) in minutes.

    Platforms need to allow simplicity when needed and complexity where required. While a 10-person company and a 10,000-person company have different needs, the same platform should be able to support these needs. Salesforce.com is a classic example of this – in their world, they don’t care what the size of your company is.

    And, platforms have to guided by product management teams, to have a shared vision. Product management has to enforce a strong adherence to the core values of focus, unity, extensibility, and complex simple complexity. A product management team that lacks the leadership to drive these values will produce a broken platform

    How does your platform compare against this checklist?

  • A Long Six Months

    If you had asked me six months ago what I would be doing today, I am pretty sure that I would have been wrong. Wrong on a galatic scale.

    GPS has stopped working. Maps tell me that even the Dragons don’t come out this far – I saw a few Dragons in cheap suits selling insurance to travellers on the edge of this territory; should have been a clue. Even the people who have better maps than I do are leading expeditions into uncharted sinkholes and rivers that are “in the wrong place”.

    There is no way to fully map what is going to come next. There is a general plan, usually a back-of-the-napkin-map from a dive bar in the last “town” (interestingly enough, drawn by a dragon in a cheap suit). Sometimes there is even a guide (for many expeditions, the guide is a one-eyed, deaf jungle-dweller who communicates via a language of spit and arm movements). In the end, it’s up to your cunning and intelligence to avoid a death so embarrassing, your ancestors will need to changes their names.

    So, in order to bring order to this trackless territory, I have begun to lay down some simple rules, ones that will help me keep the jungle madness at bay.

    A full calendar does not mean an effective day. The last 10 weeks have seen me in meetings up to six hours a day. Most of the meetings were not for extracting information from customers. Some fell into the Kafka-esque “debrief on the project status meeting so we can update the project plan and prepare for the next project status meeting” family.

    And I am not alone.

    Customers I work with are often in so many meetings that must be attended personally that the only time to get work done is in meetings. If you’re wandering around the trackless wilds spending all your time updating the map, tracking the menu, and inventorying the clean socks, when you look up from your work, you have missed the point of the trip. Or worse, realize that in the minutiae of the “important” stuff, you have reached a place that you can’t get out of without losing a good chunk of your expedition team.

    So, why did you bother coming? Showing up is only 10% of being successful. Paying attention is the real 90%.

    Guess what? Other people can help you do it all. Guess what? You’re not the only person who can count socks.

    I have become very good at delegating, for reasons that are good, under circumstances that were questionable. You came on this expedition with a team. Use it. There are people who can do the things you think only you can do. This frees you up to do the things that you are good at, like avoiding the impassible canyon (unless your team is big on a diet of bark and spider eggs).

    And maybe one or two of the sock counters will show promise by projecting the date of clean sock exhaustion and suggesting that a route to clean water be prioritized for everyone’s foot health. This means that they could help take over some of the map-reading so you can collect those bug samples you’re after.

    Delegating work does not mean you are failing. Delegating work means that you are succeeding in maintaining a focus on what’s important.

    Every day is a learning experience. I am so far into uncharted territories for myself right now that the map isn’t even helpful at figuring out where I came from. All I know is that new challenges pop up every day, and learning from them helps me get by and press forward.

  • Black Friday 2011

    Black Friday 2010 is upon us.

    Now, what are you doing to get your Web site ready for Black Friday 2011?

    While this may be a shocking slap in the face, it is a very realistic one. If you take what happened today, and what you think may happen over the next 4 weeks, what will your organization really need to be ready for next year – same time, same place?

    You were thinking about that as you got ready for this year, right?

    Well, it’s never too early to start planning. Here are some items you should be putting on your January 1 2011 wish-list.

    • Better Web monitoring. What did you get caught without any insight into this year? Where do you need to get more information? Inside or outside the firewall? Third-party components? What surprised you this year?
    • Earlier load testing. Is it less stressful to test your capacity and focus your optimization efforts in Q1 2011 or in October 2011? The advanced customers we work with start running their load tests in April, not September. How much change can you make to your systems by the time you discover a problem in September?
    • Real-world inputs and projected growth. When you take your analytics data and project your growth for next year, are you factoring in macro-economic inputs? No, I’m not an economist, but if the economy isn’t projected to grow as fast, aim your projected growth for the middle of the range for testing, not for the top-end.
    • Test capacity to the maximum. No, this is not mutually exclusive of the previous item. When you test your capacity, you want to make sure that you know exactly how much growth it can take. Even if growth is not projected to break it this year (and you can prove this with load testing), how about in 2012?
    • Mobile Commerce Readiness. Mobile is the latest buzzword. But do you have a real plan to handle a rush of people checking your prices from other stores on Black Friday? And if they want to buy it right there, can they? Mobile is not a separate silo; All sales channels make you money, so stop treating them differently. If you are going mobile, do it with a plan that scales with sales.

    Whatever you do, don’t rest on your laurels (or bed of broken glass, depending on how your day went). Have a plan. Write it down. Set some deadlines.

    Give yourself a head start.

    Black Friday 2011 is only 364 days away.

  • Web Performance Concepts: Customer Anywhere

    Companies are beginning to fully grasp the need to measure performance from all perspectives: backbone, last mile, mobile, etc. But this need is often driven by the operational perspective – “We need to know how our application/app is doing from all perspectives”.

    While this is admirable, and better than not measuring at all, turning this perspective around will provide companies with a whole new perspective. Measure from all perspectives not just because you can, but because your customers demand performance from all perspectives.

    The modern company needs to always keep in mind the concept of Customer Anywhere. The desire to visit your site, check a reservation, compare prices, produce coupons can now occur at the customer’s whim. Smartphones and mobile broadband have freed customers from the wires for the first time.
    If I want to shop poolside, I want your site to be as fast over a mobile connection on my Android as it is on my WiFi iPad as it is on my Alienware laptop on ethernet. I don’t care what the excuse is: If it’s not fast, it’s not revenue.

    Knowing how a site performs over the wire, in the browser, around the world made “Web” performance a lot harder. The old ways aren’t enough.

    How does your “Web” performance strategy work with Customer Anywhere?

  • Career Reform: Selling your Way Out of the Paper Bag

    One of the things that all consultants have to accept is that selling is a part of the territory. Doesn’t matter if you are a solopreneur or an associate consultant in 10,000 person firm, selling is an everyday occurrence.

    Sounds like a cliché, but it’s true. Everything a consultant does or says is part of their ongoing selling process. Skills and experience must constantly be sold to customers.

    It’s hard to sell, if you stop and think about it. You have to convince people, strangers, that you or your firm have the skills to solve the problem that the customer has identified, and to demonstrate that you can identify and solve problems that the customer may not know they have.

    How do you do it? There is no easy answer. My experience is that selling customers is often not about the products or services themselves, but about selling the value and the solutions that your experience brings to the equation on top of the products or services. Selling is about believing that what you can do for the customer is beyond what they could achieve themselves, but which will make them far more successful than they would be on their own.

    Selling Consulting services requires self-confidence, and a willingness to leave your ego at the door. What the customer thinks they need, and what they position with you or the sales team that they have been working with, are often only their tactical, short-term needs. Customers often are unwilling to accept the solution they really need. Sometimes, the consultant has to accept starting with the partial solution sale to get the customer to accept the larger problem.

    Leaving your ego at the door makes accepting the initial compromise easier to accept. Good consultants see the short-term, tactical project as the way in. But if that’s all that you are able to sell, then you may need to reflect on how you are positioning yourself.

    Selling consulting is a process that is continuous, even with customers that you are already working with. Being a consultant means that you must always listen, observe, and sell. Reputation, relationships, and experience/skills only get you so far. Selling it, be it yourself or the solution the customer really needs, is what takes you the next step.

    How do you sell consulting services? How do you sell yourself as a consultant?

  • Career Reform – From Analyst to Consultant

    For many years my professional title has included the word “consultant”, and with it the gravitas that comes with being able to use that term. In the cold, hard light of my early-40s, in all honesty I have say that I was not a consultant for most of that time: I was an analyst.

    Analyst versus consultant. What’s the difference?
    In black and white terms, an analyst is a tactical consultant, with a specific set of skills and knowledge that can be used to solve a particular problem. And a consultant is…?

    You keep using that word. I do not think it means what you think it means. Inigo Montoya

    Consultant is over-used and mis-used word. All the people I know who call themselves consultants are actually analysts, contractors, or skilled professionals who call themselves consultants for lack of a better term to describe what they do to pay the bills, and because putting Gun For Hire on a business card tends to attract the wrong clientele.

    On the other end of the spectrum, consultant is more than a term to describe a person who works in a large consultancy or professional services firm (or as, Andrea Mulligan is working through in public, a professional service practice in a software or SaaS firm). A consultant comes to a customer with a set of skills that cannot be had just anywhere, be it in a programming language, GAAP restructuring, or, in my case, Web performance measurement and load testing.

    A true consultant must be more than a skilled analysts who has chosen the freedom of working outside large companies, leaping from challenge to challenge. A consultant brings years of experience and a view of the larger world with them. In fact, many of the best consultants can’t do what their analysts do for them (or maybe the consultant’s skills are just too rusty) on a daily basis.

    Analysts solve a specific problem. Consultants ensure that the problem never happens again.

    Consultants put the problem that analysts solve into context.

    For more than a decade, I have been an analyst, solving whatever thorny riddle is put in front of me using whatever tools and skills I could cobble together. Analysts don’t have a lifetime career ahead of them, as their skill-set falls out of favor or is replaced by younger, more talented analysts.

    Consultants take what they have learned during their analyst/apprentice days and convert that into a strategic view. Not simply How do we solve this problem? but Is this the right problem to solve? or How did we get to the point where we needed to solve this problem?

    And, most importantly, Is the solution we’re developing flexible enough to adapt to solve and prevent problems we can’t even foresee now?

    It’s hard for someone like me who revels in solving the problems no one else can to let go and realize that the problem isn’t everything. To realize that there are people out there who can do what I do as well as or better than I can.

    Letting go of one thing means that you have to have something else to grab onto. I do not relish Wily Coyote moments: looking down to see the fall that’s about to come.

    So, at 42, I am stepping back to embrace a very new and different career question: What does it really mean to be a strong consultant?

    It’s not easy to shift gears, and drop into the career lane that I had avoided for so long, feeling it a trap. I now know that to survive and flourish, I have to understand how the business works, how practice/company goals are set and met, how to effectively sell professional service (something I am awful at a lot of the time), and how to position professional services within the SaaS model.

    It is a somewhat disheartening realization that the 10 years I spent fighting becoming a strong consultant now have to be made up in a very short amount of time, but the games everyone remembers are those that are won from behind in overtime.

  • Smartphones: On Moving Back To Blackberry

    A couple of weeks ago, I moved my mobile life back to a Blackberry Bold 9700 from T-Mobile after being on a Dash 3G  for the last 6 months.

    It’s like breathing air again.

    Admittedly, I am not the typical modern smartphone user. I prefer a full keyboard over a touchscreen, and I still operate in a mainly text-based world. So the Blackberry is exactly what I need to get through my day. I get my work email fast, the GMail app is fantastic, and UMA is really an astounding thing.

    When you compare the Bold 9700 to its predecessor in my life, it is like moving from a broken Windows 3.11 486DX to a new MacBook Pro. WinMo 6.5 is not a modern mobile platform and on the Dash 3G, it only gets worse.

    It’s not T-Mobile’s fault that they have the Dash 3G. But I am glad it’s not with me anymore.

  • The Complexity of Web Performance

    Helping a colleague this week, we uncovered some odd behavior with a site whose performance he was analyzing. Upon first glance, it was clear that this site had a performance issue – they had HTTP persistence disabled. Immediate red flag in the areas of network overhead and geographic latency.

    Further digging exposed something more sinister. It seems that HTTP persistence was only disabled for browsers with MSIE in the user-agent string. Even if the user-agent string was just MSIE, HTTP persistence was off.

    The customer was very forthcoming and sent us their standard httpd.conf file. This showed no sign of the standard (and frustrating) global disabling of persistence for Internet Explorer.

    Finally, it came to us. The customer had provided a simple network diagram, and there, just before packets hit the Internet, was a Layer 7 firewall. How did we know the Layer 7 firewall was the likely cause? Because this device was also the one that provided compression for the content going out to customers.

    A Layer 7 firewall happily rewrites HTTP headers to reflect the nature of the compressed content (content-length or transfer-encoding: chunked) and to add the gzip flag (accept-encoding:gzip). Since this device was already doing this, it was pretty clear to us that it also had a rule that disabled HTTP persistence for anything with MSIE in the user-agent string.

    This was a fine example of the complexity of the modern Web application infrastructure. In effect, there were two groups with different ideas of how Internet Explorer should be handled at the network layer, and neither of them seems to have talked to the other.

    When you have a Web performance problem, indulge in a thought experiment. Create an imaginary incoming Web request and try to see if you can follow it through all the systems it touches on your system. Put it on a whiteboard, a mindmap, whatever works.

    Then invite the system architects and network engineers in and get them to fill in the gaps.

    No doubt that will lead to the “ah ha!” moment. If nothing else, it’s a good excuse to put pizza on the company card. But I have no doubt that you will walk away with a better understanding of your systems, which will make it easier for you to talk to all the people responsible for keeping your systems running.

    TAKEAWAY: Just because the part of the Web application you work on is working fine, it may be affected by other components that are not tuned or configured for performance. Get to know the entire application at a high level.

  • Compression and the Browser – Who Supports What?

    The title is a question I ask because I hear so many different views and perspectives about HTTP compression from the people I work with, colleagues and customers alike.

    There appears to be no absolute statement about the compression capabilities of all current (or in-use) browsers anywhere on the Web.
    My standard line is: If your customers are using modern browsers, compress all text content — HTML (dynamic and static), CSS, XML, and Javascript. If you find that a subset of your customers have challenges with compression (I suggest using a cross-browser testing tool to determine this before your customers do), write very explicit regular expressions into your Web server or compression device configuration to filter the user-agent string in a targeted, not a global, way.

    For example, last week I was on a call with a customer and they disabled compression for all versions of Internet Explorer 6, as the Windows XP pre-SP2 version (which they say you could not easily identify) did not handle it well. My immediate response (in my head, not out loud) was that if you had customers using Window XP pre-SP2, those machines were likely pwned by the Russian Mob. I find it very odd that an organization would disable HTTP compression for all Internet Explorer 6 visitors for the benefit of a very small number of ancient Windows XP installations.

    Feedback from readers, experts, and browser manufacturers that would allow me to compile a list of compatible browsers, and any known issues or restrictions with browsers, would go a long way to resolving this ongoing debate.

    UPDATE: Aaron Peters pointed me in the direction of BrowserScope which has an extensive (exhaustive?) list of browsers and their capabilities. If you are seeking the final word, this is a good place to start, as it tests real browsers being used by real people in the real world.

    UPDATE – 09/24/2012: I found a site today that was still configured incorrectly. Please, please, check your HTTP Compression settings for ALL browsers your customers use. Including you MOBILE clients.