Category: Uncategorized

This morning’s bounty of posts brought in two that will make you think.

First was Port80 Software’s comments on using the Cache-Control mechanism embedded in all browsers. This is interesting to read, as I have been trying to get companies to use this mechanism more intelligently for a number of years. I know that the Port80 team gets it, but it is always nice to have some outside validation of a position you have tried to evangelize for a long time.

The second was Tim O’Reilly’s post on Cal Henderson’s new book on Web scalability. While I am likely to purchase the book for a professional interest, I have one problem with Flickr’s current configuration: static.flickr.com does not use HTTP persistence, something I noted last week. This strikes me as weird.

It’s always good to see Web performance rear its head.

Technorati Tags: Tim O’reilly, Cal Henderson, Cache-Control, http, port80 software, persistent connections, Flickr

In the next 10 minutes, I will be turning off the TailRank Measurements. The performance has been abysmal for the last month.

I’ll check back in a couple of weeks to see if they snap back.

Technorati Tags: TailRank, GrabPERF, Web performance

In the last three days I have gone from WordPress 2.0.2 to 2.0.3, then from 2.0.3 to 2.0.4. All went seamlessly, and the blog working as expected.

Upgrades in both cases took less than 5 minutes.

I love WordPress. I now know why Moveable Type is tanking so hard.

Technorati Tags: WordPress, upgrades, 2.0.4

At our garage sale today, I sold off the original GrabPERF Web server. Sad to see it go, but it was time.

I am still trying to sell the database server. But I might turn it into a file server with a couple of massive drives.

Technorati Tags: GrabPERF

It appears that I will likely have to be fingerprinted every time I come into the US from now on. [here and here]

I love being an “enemy alien”.

Technorati Tags: Immigration, US-VISIT, USA, Canada

Over the last 12 months, GrabPERF has added a lot of measurements, especially from folks who thought that the system was cool, looked at the data a few times, and never returned.
So, I have looked at the logs for the last few weeks, and determined that there are definitely a large number measurements that no one looks at. In order to absorb the load on the system and to free up space in the database, I inactivated approximately 30 measurements in the last hour.
If I inactivated a measurement you were using, please let me know. If there is a measurement you want added, please let me know.
Technorati Tags: GrabPERF

If you want to own a piece of GrabPERF history, let me know.

Technorati Tags: GrabPERF

I came in this morning to read on Netcraft that MySpace had been knocked offline by a power outage at their datacenter. [here]

The question is now one of how did this happen. There should have been contingency planning in the extreme for this type of event, especially for operations in the Western US. Backup generators, UPS, etc. should have been in place.

It would be very interesting to sit in on a post-mortem of this event.

Technorati Tags: MySpace, Web performance

As of right now, five of the seven GrabPERF measurement agents are updated with the new code I created last night.

I am also planning some addition changes to the service, including posts, HTTP Basic authentication, content matching and some basic per-measurement cookie jar technology.

Thank you all for your continued support of GrabPERF.

Technorati Tags: GrabPERF, measurement technology

Dear Flickr:

I have been wondering for sometime why downloads from your site seemed a little sluggish at times.

At first I blamed your unprecedented growth and success. For a little Vancouver startup (I am a BC boy myself), your entrance onto the stage of social networking applications has been phenomenal. The move from zero to infinity may have played a part in the performance I was seeing.

Nope. There was something else going on; I could see it every time I loaded a Flickr page in my browser. There was something else going on.

So today, I checked something out, and found the problem.

You need to enable persistent TCP connection on the static.flickr.com servers.

Now, that is the simple answer. I know that with large, web-based applications, enabling something as monumental as persistent connections could cause serious issues. If the architecture of the system was not designed to handle persistent connections, turning them on could cause the entire system to collapse.

There are legitimate, if mis-guided, reasons for disabled persistent connections. Some administrators believe that it is actually more efficient to have a client open a connection for every object. Easier to manage state, etc. The only problem is that in order to do that, you have to tune the systems serving data to shorten the amount of time a closed connection spends in a TIME_WAIT state.

When a TCP connection is closed, the socket is not immediately closed by the system in a default configuration. The TIME_WAIT state is the holding pen that these connections are pushed into. While in this state, the socket is locked and this may count against the incoming TCP connection queue, forcing the network stack to delay or reset new incoming connections.

Still, as Flickr is a worldwide company, the delay that the lack of persistent connections injects is astounding for locations in Asia. If you want to grow your business, and support more services, this will likely become a bottleneck very quickly.

Have a great weekend!

smp

Technorati Tags: Flickr, persistent connections, TCP, http, web performance, scalability