Category: Uncategorized

One additional changes was made to GrabPERF today. The homepage with the Top and Bottom Performers, has been changed from a dynamic page to one that is autogenerated every two minutes. This graph should explain why.

The dynamic page was starting to push 20-25 seconds just for the Top 20 List. When I switched to the static list, times dropped to less than 1 second.
It’s always bad when a Web performance measurement site has poor performance.

Today we undertook two maintenance and upgrade tasks at GrabPERF that have been neglected for too long.

1. The Agent code was streamlined and removed the connection error sub-routine. It seems that the latest versions of cURL no longer support the connection error determination (I can only imagine the madness of trying to support this on multiple OSs), so it has been removed as part of the error detection process. This change has been pushed out to four agents for testing and will be distributed to all other active agents after this is complete.
2. Upgrade of cURL to 7.19.4 on four agents. The same four agents that have the new agent code have also had the underlying HTTP(S) engine (cURL) upgrade to 7.19.4. Although this supports no new features that I am aware of, it is always good to be on top of the latest release with bug-fixes.

We are also trying to determine how to capture the URL that we connect to when we take a measurement. As far as cURL is documented, it still does not appear to support this feature.

The Web analytics firm StatCounter has released a set of metrics that mines their entire dataset to provide worldwide metrics on browsers, searches, and operating systems.
This is the kind of data that everyone should be interested in. And it’s free. Check it out at StatCounter GlobalStats.

The GrabPERF Retail Web Performance Index is an extension of the Holiday Performance Index created for the Black Friday/Cyber Monday craze in 2008.
There is a methodology statement explaining more about the index below the data table.
Please add any comments or questions to this post.

On the weekend, I upgraded the database engine for GrabPERF to Mysql 5.1 and switched the main data table from MyISAM to InnoDB.
The switch to InnoDB was done because of the locking issues that were occurring during long queries, especially when doing ad-hoc analysis. The row-level (versus table-level) locking of InnoDB has removed most of these issues.
I have been seeing some strange behavior with the new engine. As a result of this, I will be re-starting the database engine twice a day. There should be no degradation, as this is simply a daemon re-start, not a machine re-start.

Well, it’s done. After a week of trying this and that, I finally got DSL (DamnSmallLinux) rolling on the Fujitsu B2131 last night.
To remind folks what the challenge was (and is for some of the linux dev teams out their who claim to support older platforms): Install a fully-functioning OS on a laptop machine built in 1999/2000 without a CD drive.
DSL has a great boot floppy image that works with their embedded version of the OS installed on a Flash drive – ok, the flash drive made the challenge less problematic than originally described.
However, I lay the challenge out to all of the Linux distros: Create an across the INTERNET (not that crazy PXE boot stuff) install that can be started with boot floppies.
The B2131 has a large enough hard drive and enough power to run Xubuntu, but installing it easily (i.e. my 10 year-old son could do it) makes modern linux distros completely unreachable for people trying to easily make old machines go.
So, linux geeks, think you can do this?

A co-worker gave me an ancient Fujitsu Lifebook B-2131. I want to turn it into an ultra-portable netbook thingie.
The catch? No cd-rom.
Apparently the world has forgotten the world of the boot floppy and internet installs.
Well, almost everyone.
Turns out that if you have a floppy drive, and 5 spare 1.44MB disks, there is a way to install Ubuntu over the Internet! [SEE HERE]
We ran out of time and floppy disks today, but this looks like a cool father-son project for this week.
Further updates as they are warranted.

Budgets are shrinking. Resources are tight or shrinking. In a recent post, I discussed how ideas that I had been a proponent of for 2-3 years suddenly became extremely valuable to companies during the downturn of 2001-2003.
This downturn is a different beast. This means that you will need more than basic technical smarts to get through. To survive in the world of e-commerce for the next 18 months, here are some strategies you should take to heart.

• Clean up the cruft. Development and infrastructure changes are going to slow during the downturn – accept this. So do a code audit. Make sure that your applications aren’t calling files that no longer exist. Get rid of (backup and archive or Sarbanes-Oxley will bite your butt) old directories or application code that is no longer supported or maintained. In other words, do your Spring cleaning.

• Take care of the annoyances. In the rush to get code out or fix issues, little things that could help your site are often overlooked. Take the time now to optimize your images, minify your JS files, ensure your CSS files only contain styles that you use. Tune your SQL queries. Optimize your app code. All those things that got lost in the rush to get stuff out the door. As Microsoft and Apple have realized, people want performance, not more features.

• Make your site Browser Neutral. I work for a firm that allows me to view various sites with different OS/browser combinations easily and recommend to clients that they do the same. I hang my head when I look at my own blog and realize that it is not Browser Neutral. Take some time to investigate to how to eliminate this frustrating annoyance that makes people cranky and doubt your technical savvy.

• Get to know your visitors/customers. Unlike a ‘real’ store, you can’t step out onto the floor and talk to customers. But you can do this far more easily in today’s world than ever before. In the past, visitor analytics would have been the extent to which a company would have gone to determine information about their visitors/customers. Today, Twitter, Facebook, and all other manner of social sites make listening and talking to customers much easier. Just remember to be real.

• Get to know the people you work with. Just like getting to know your visitors/customers, you need to spend some time getting to know people in your organization. I know this sounds like manager-speak, but if you have contacts in Finance, Product Management, Operations, Admin, you are more likely to be able to more effectively due your job. If you understand the ebb and flow and stresses that are going on outside your little enclosed silo, you can place things in a larger perspective. While social-networking may work here, be careful to back it up with face-to-face contact.

• Get to know people! I made a number of contacts in the last downturn by taking my key interests (Web compression, Web caching) and turning them into sites, published articles, and one failed conference appearance (contact me if you want to hear more). Over the years these contacts have had me comment on their posts, edit their books, and keep them up-to-date on where I think the latest trends are headed. Going into this downturn, I have a whole group of new contacts that I am building on to do the same things. People aren’t bad, they’re just misunderstood.

• Read the fine business plan. I know these things are evil. They are in horrible management-speak. So don’t read it that way: Active read it. Make notes in the columns. Turn it into a MindMap (that’s what I did this year). Extract the meaning of what the goals of the business are and how the Web site fits into it. If you understand what the rest of the business is doing and what their challenges are, your problems will have context!

• Know what the next big thing is. While the objective of the organization in a downturn is to try and be more efficient with fewer resources, remember that you have to take time (whatever you have left) to know what’s coming next that could affect your Web site. Read the tech news. Understand the fundamentals that are driving the latest ideas. Keep on top of security. Sometimes a side-project that uses a technology that is new can become a bigger part of your site when resources free up.

I know these are general, but when this downturn started, I looked back on what worked for me during the last downturn. These are a few of the things that got me through, kept me busy, and helped me make an impact on the company and industry I am in.
These things also don’t hurt your reputation, as you become known as someone who understands the industry, the business, and the customers.

Over the space of two years I have created a series of posts that explore the analytical and statistical concepts that compose the foundation of a solid Web performance strategy.

These mark my attempt to move from a purely technical analysis of providing a solution to the problem at hand. These posts move my approach to one that examines the issues from a higher perspective, one where I can place the singular issues that face people in the areas of Technical Operations and Business Operations into a cross-industry, global Web performance context.
These posts highlight some of the core concepts which need to be re-examined, especially within the reality that faced by all online businesses right now: tightening budgets, static or contracting staffing, and the need to not lose the performance edge.

I hope that these nine posts help you reconsider how you examine Web performance data when you are trying to maintain and grow your business in these troubled times.

1. Web Performance, Part I: Fundamentals
2. Web Performance, Part II: What are you calling ‘average’?
3. Web Performance, Part III: Moving Beyond Average
4. Web Performance, Part IV: Finding The Frequency
5. Web Performance, Part V: Baseline Your Data
6. Web Performance, Part VI: Benchmarking Your Site
7. Web Performance, Part VII: Reliability and Consistency
8. Web Performance, Part VIII: How do you define fast?
9. Web Performance, Part IX: Curse of the Single Metric

Today, our Green Cards arrived in the mail.
The process started on March 23 2005 when our applications were accepted. So, slightly less than four years later, the process is complete. At least for 10 years.
I want to thank the lawyers who handled this for us, and my employer for sponsoring me and going through this, well, special process.