The Rule of Thirds: The Web Performance Analyst

Blurry Man - Brian Auer -, there has been a big push for the Dev/Ops culture, an integrated blending of development and operations who work closely together to ensure that poor performing web and mobile applications don’t make it out the door. They have become the rockstars of the conference circuit and the employment boards.
I fit into neither of these categories. I have never run anything more than a couple of linux servers with Apache and MySQL. I write code because I’m curious, not because I’m good at it – in fact, I write the worst code in the world and I am willing to prove it!
I am a member of a web and mobile performance culture that is language and platform independent, to use some buzzwords.
I am a web and mobile performance consultant and analyst.
I can take apart reams of data to find statistical patterns and anomalies. I believe that averages are evil, and have believed this for more than a decade. I have been using frequency and percentile distributions for almost as long and watched as the industry finally caught up.
I can link the business issue that faces your company with the technical concerns you are facing and help guide you to the middle ground where performance and the balance sheet are in careful equilibrium.
I don’t care what you write your code in. I don’t care what you run it on. Now, don’t get me wrong: I respect and admire the Dev/Ops folks I have met and know. I am just not in their tribe.

HTTP Compression – Have you checked ALL your browsers?

Apache has been my web server of choice for more than a decade. It was one of the first things I learned to compile and manage properly on linux, so I have a great affinity for it. However, there are still a few gotchas that are out there that make me grateful that I still know my way around the httpd.conf file.

HTTP compression is something I have advocated for a long time (just Googled my name and compression – I wrote some of that stuff?) as just basic common sense.
Make Stuff Smaller. Go Faster. Cost Less Bandwidth. Lower CDN Charges. [Ok, I can’t be sure of the last one.]
But, browsers haven’t always played nice. At least up until about 2008. After then, I can be pretty safe in saying that even the most brain-damaged web and mobile browsers could handle pretty much any compressed content we threw at them.
Oh, Apache! But where were you? There is an old rule that is still out there, buried deep in the httpd.conf file that can shoot you. I actually caught it yesterday when looking at a site using IE8 and Firefox 8 measurement agents at work. Firefox was about 570K while IE was nearly 980K. Turns out that server was not compressing CSS and JS files sent to IE due to this little gem:

 BrowserMatch \bMSIE !no-gzip gzip-only-text/html

This was in response to some issues with HTTP Compression in IE 5 and early versions of IE6 – remember them? – and was appropriate then. Guess what? If you still have this buried in your Apache configuration (or any web server or hardware device that does compression for you), break out the chisels: it’s likely your httpd.conf file hasn’t been touched since the stone age.
Take. It. Out.
Your site shouldn’t see traffic from any browsers that don’t support compression (unless they’re robots and then, oh well!) so having rules that might accidentally deny compression might cause troubles. Turn the old security ACL rule around for HTTP compression:

Allow everything, then explicitly disable compression.

That should help prevent any accidents. Or higher bandwidth bills due to IE traffic.

What does success mean for you?

Talking to customers is always teaches me new ways of looking at the industry I’m in. I don’t talk to as many customers as I used to, but when I do, it is interesting how many companies, be they large and established or small and emerging, are focused on the problems of now.
I’ve talked about the different perspectives on the problems of now that I have seen in my industry (here and here), but if you ask any consultant or analyst, similar questions can be traced through any company/organization in any industry/sector.
I always look at the problems of now as a passing fad. As I answer each question, a new one would arise, appearing from the ashes of the previous one. To prevent endless flailing about, dancing from question to question like a tactical pinball, I ask the most important question of any scoping process as early as I can:

What does a successful engagement look like to you?

Innocuous. Simplistic. But powerfully effective. Putting this simple question into the scoping process helps the customer explain to you how you’re going to help them be successful, because they know what success means to them.
What applies in the macro can often trickle-down to the personal. I am notorious for not having a “plan” – I have driven a number of Type A (A+++?) managers to madness with my lack of a plan. But I know what success means to me; I just leave the details of how I achieve it open-ended.
Have you considered what success looks like to you?

Black Friday 2011

Black Friday 2010 is upon us.
Now, what are you doing to get your Web site ready for Black Friday 2011?
While this may be a shocking slap in the face, it is a very realistic one. If you take what happened today, and what you think may happen over the next 4 weeks, what will your organization really need to be ready for next year – same time, same place?
You were thinking about that as you got ready for this year, right?
Well, it’s never too early to start planning. Here are some items you should be putting on your January 1 2011 wish-list.

  • Better Web monitoring. What did you get caught without any insight into this year? Where do you need to get more information? Inside or outside the firewall? Third-party components? What surprised you this year?
  • Earlier load testing. Is it less stressful to test your capacity and focus your optimization efforts in Q1 2011 or in October 2011? The advanced customers we work with start running their load tests in April, not September. How much change can you make to your systems by the time you discover a problem in September?
  • Real-world inputs and projected growth. When you take your analytics data and project your growth for next year, are you factoring in macro-economic inputs? No, I’m not an economist, but if the economy isn’t projected to grow as fast, aim your projected growth for the middle of the range for testing, not for the top-end.
  • Test capacity to the maximum. No, this is not mutually exclusive of the previous item. When you test your capacity, you want to make sure that you know exactly how much growth it can take. Even if growth is not projected to break it this year (and you can prove this with load testing), how about in 2012?
  • Mobile Commerce Readiness. Mobile is the latest buzzword. But do you have a real plan to handle a rush of people checking your prices from other stores on Black Friday? And if they want to buy it right there, can they? Mobile is not a separate silo; All sales channels make you money, so stop treating them differently. If you are going mobile, do it with a plan that scales with sales.

Whatever you do, don’t rest on your laurels (or bed of broken glass, depending on how your day went). Have a plan. Write it down. Set some deadlines.
Give yourself a head start.
Black Friday 2011 is only 364 days away.

Career Reform: Selling your Way Out of the Paper Bag

One of the things that all consultants have to accept is that selling is a part of the territory. Doesn’t matter if you are a solopreneur or an associate consultant in 10,000 person firm, selling is an everyday occurrence.
Sounds like a cliché, but it’s true. Everything a consultant does or says is part of their ongoing selling process. Skills and experience must constantly be sold to customers.
It’s hard to sell, if you stop and think about it. You have to convince people, strangers, that you or your firm have the skills to solve the problem that the customer has identified, and to demonstrate that you can identify and solve problems that the customer may not know they have.
How do you do it? There is no easy answer. My experience is that selling customers is often not about the products or services themselves, but about selling the value and the solutions that your experience brings to the equation on top of the products or services. Selling is about believing that what you can do for the customer is beyond what they could achieve themselves, but which will make them far more successful than they would be on their own.
Selling Consulting services requires self-confidence, and a willingness to leave your ego at the door. What the customer thinks they need, and what they position with you or the sales team that they have been working with, are often only their tactical, short-term needs. Customers often are unwilling to accept the solution they really need. Sometimes, the consultant has to accept starting with the partial solution sale to get the customer to accept the larger problem.
Leaving your ego at the door makes accepting the initial compromise easier to accept. Good consultants see the short-term, tactical project as the way in. But if that’s all that you are able to sell, then you may need to reflect on how you are positioning yourself.
Selling consulting is a process that is continuous, even with customers that you are already working with. Being a consultant means that you must always listen, observe, and sell. Reputation, relationships, and experience/skills only get you so far. Selling it, be it yourself or the solution the customer really needs, is what takes you the next step.
How do you sell consulting services? How do you sell yourself as a consultant?

Career Reform – From Analyst to Consultant

For many years my professional title has included the word “consultant”, and with it the gravitas that comes with being able to use that term. In the cold, hard light of my early-40s, in all honesty I have say that I was not a consultant for most of that time: I was an analyst.
Analyst versus consultant. What’s the difference?
In black and white terms, an analyst is a tactical consultant, with a specific set of skills and knowledge that can be used to solve a particular problem. And a consultant is…?

You keep using that word. I do not think it means what you think it means. Inigo Montoya

Consultant is over-used and mis-used word. All the people I know who call themselves consultants are actually analysts, contractors, or skilled professionals who call themselves consultants for lack of a better term to describe what they do to pay the bills, and because putting Gun For Hire on a business card tends to attract the wrong clientele.
On the other end of the spectrum, consultant is more than a term to describe a person who works in a large consultancy or professional services firm (or as, Andrea Mulligan is working through in public, a professional service practice in a software or SaaS firm). A consultant comes to a customer with a set of skills that cannot be had just anywhere, be it in a programming language, GAAP restructuring, or, in my case, Web performance measurement and load testing.
A true consultant must be more than a skilled analysts who has chosen the freedom of working outside large companies, leaping from challenge to challenge. A consultant brings years of experience and a view of the larger world with them. In fact, many of the best consultants can’t do what their analysts do for them (or maybe the consultant’s skills are just too rusty) on a daily basis.
Analysts solve a specific problem. Consultants ensure that the problem never happens again.
Consultants put the problem that analysts solve into context.
For more than a decade, I have been an analyst, solving whatever thorny riddle is put in front of me using whatever tools and skills I could cobble together. Analysts don’t have a lifetime career ahead of them, as their skill-set falls out of favor or is replaced by younger, more talented analysts.
Consultants take what they have learned during their analyst/apprentice days and convert that into a strategic view. Not simply How do we solve this problem? but Is this the right problem to solve? or How did we get to the point where we needed to solve this problem?
And, most importantly, Is the solution we’re developing flexible enough to adapt to solve and prevent problems we can’t even foresee now?
It’s hard for someone like me who revels in solving the problems no one else can to let go and realize that the problem isn’t everything. To realize that there are people out there who can do what I do as well as or better than I can.
Letting go of one thing means that you have to have something else to grab onto. I do not relish Wily Coyote moments: looking down to see the fall that’s about to come.
So, at 42, I am stepping back to embrace a very new and different career question: What does it really mean to be a strong consultant?
It’s not easy to shift gears, and drop into the career lane that I had avoided for so long, feeling it a trap. I now know that to survive and flourish, I have to understand how the business works, how practice/company goals are set and met, how to effectively sell professional service (something I am awful at a lot of the time), and how to position professional services within the SaaS model.
It is a somewhat disheartening realization that the 10 years I spent fighting becoming a strong consultant now have to be made up in a very short amount of time, but the games everyone remembers are those that are won from behind in overtime.

Smartphones: On Moving Back To Blackberry

A couple of weeks ago, I moved my mobile life back to a Blackberry Bold 9700 from T-Mobile after being on a Dash 3G  for the last 6 months.
It’s like breathing air again.
Admittedly, I am not the typical modern smartphone user. I prefer a full keyboard over a touchscreen, and I still operate in a mainly text-based world. So the Blackberry is exactly what I need to get through my day. I get my work email fast, the GMail app is fantastic, and UMA is really an astounding thing.
When you compare the Bold 9700 to its predecessor in my life, it is like moving from a broken Windows 3.11 486DX to a new MacBook Pro. WinMo 6.5 is not a modern mobile platform and on the Dash 3G, it only gets worse.
It’s not T-Mobile’s fault that they have the Dash 3G. But I am glad it’s not with me anymore.

The Complexity of Web Performance

Helping a colleague this week, we uncovered some odd behavior with a site whose performance he was analyzing. Upon first glance, it was clear that this site had a performance issue – they had HTTP persistence disabled. Immediate red flag in the areas of network overhead and geographic latency.
Further digging exposed something more sinister. It seems that HTTP persistence was only disabled for browsers with MSIE in the user-agent string. Even if the user-agent string was just MSIE, HTTP persistence was off.
The customer was very forthcoming and sent us their standard httpd.conf file. This showed no sign of the standard (and frustrating) global disabling of persistence for Internet Explorer.
Finally, it came to us. The customer had provided a simple network diagram, and there, just before packets hit the Internet, was a Layer 7 firewall. How did we know the Layer 7 firewall was the likely cause? Because this device was also the one that provided compression for the content going out to customers.
A Layer 7 firewall happily rewrites HTTP headers to reflect the nature of the compressed content (content-length or transfer-encoding: chunked) and to add the gzip flag (accept-encoding:gzip). Since this device was already doing this, it was pretty clear to us that it also had a rule that disabled HTTP persistence for anything with MSIE in the user-agent string.
This was a fine example of the complexity of the modern Web application infrastructure. In effect, there were two groups with different ideas of how Internet Explorer should be handled at the network layer, and neither of them seems to have talked to the other.
When you have a Web performance problem, indulge in a thought experiment. Create an imaginary incoming Web request and try to see if you can follow it through all the systems it touches on your system. Put it on a whiteboard, a mindmap, whatever works.
Then invite the system architects and network engineers in and get them to fill in the gaps.
No doubt that will lead to the “ah ha!” moment. If nothing else, it’s a good excuse to put pizza on the company card. But I have no doubt that you will walk away with a better understanding of your systems, which will make it easier for you to talk to all the people responsible for keeping your systems running.
TAKEAWAY: Just because the part of the Web application you work on is working fine, it may be affected by other components that are not tuned or configured for performance. Get to know the entire application at a high level.

The Rocky Mountain Parks: A Privilege Unappreciated

Ken Burns’ tale of the US National Parks reminds me of a heritage that I have, for most of my life, taken for granted. It was in another country, but it is a heritage that I have assumed will always be there.
I grew up amongst the Canadian Rocky Mountain Parks. Dead center amongst them you might say. Within two hours drive, there were five spectacular parks – Yoho, Banff, Jasper, Kootenay, Glacier, and Mt. Revelstoke.
All of these parks played a part in my childhood, adolescence, and young adult life. It has been nearly 20 years since I spent any time in these parks, but the experience I had there have shaped how I see the world around me. But only now can I really appreciate what these parks mean to us all, in all places.
The parks are a powerful reminder of the transitory effect that man has. Each of them contains some amount of ruins as a visible reminder of man’s failed attempts to exploit and tame the parks. The carcasses of hotels, remains of viaducts, the skeletons of towns litter these refuges.
A part of that failed heritage is something I carry with me, as I am descended from one of the last group permanent residents of an industrial town in a Canadian National Park, as my grandfather lived for a time in the now abandoned town of Bankhead Alberta. My family took me to this place as a child and told me that ‘Grandpa lived here’, a concept I could not understand, as I was in a National Park, wasn’t I? I had no idea of the conflict over what it meant to be a Canadian National Park at the time, as I saw them as the refuges and preserves they had become.
Growing up amongst these special places has left with a certain jaded perspective on beauty in the world. Yosemite does not awe the way it does others, as I was raised surrounded by beauty comparable to Yosemite, and perhaps exceeding it. But now I give my unrestrained thanks to those who made the effort to preserve, protect, and conserve these places.
Within the gently protective walls of the Canadian Mountain Parks, I have seen the sublime and the ridiculous. The commercial and the ethereal. Untouched wilderness and unabashed capitalism. And despite protests on both sides, it is clear that they work together, for without the treasure and largesse of one type of visitor, the other would not have a place to go.
Banff is the greatest eyesore amongst those who see the parks as the preserve of untrammeled wilderness. However, if Banff had not existed, the desire and initiative needed to protect the other four parks would not have gained ground. So a commercial pit keeps the wilderness protected, a balance that we can accept in a day of far greater compromises.
So though the idea of a National Park may have been originated in the US, Canada has done well to develop the idea on its own terms. Only now that I am many thousands of miles removed from them, can I appreciate what they have done to to shape me. These memories leave me breathless in the realization of the great privilege I have taken for granted for all of these years.

If you have WordPress, Use WP Super Cache

I just did a quick experiment to validate my hunch, and it’s true – WP Super Cache can cut your HTML load time in half in your WP deployment. Just check out the GrabPERF Measurement that backs this up.