Month: July 2006

GrabPERF: Measurement Cleanup

Over the last 12 months, GrabPERF has added a lot of measurements, especially from folks who thought that the system was cool, looked at the data a few times, and never returned.
So, I have looked at the logs for the last few weeks, and determined that there are definitely a large number measurements that no one looks at. In order to absorb the load on the system and to free up space in the database, I inactivated approximately 30 measurements in the last hour.
If I inactivated a measurement you were using, please let me know. If there is a measurement you want added, please let me know.
Technorati Tags:

Web Performance: MySpace Offline

I came in this morning to read on Netcraft that MySpace had been knocked offline by a power outage at their datacenter. [here]

The question is now one of how did this happen. There should have been contingency planning in the extreme for this type of event, especially for operations in the Western US. Backup generators, UPS, etc. should have been in place.

It would be very interesting to sit in on a post-mortem of this event.

Technorati Tags: ,

Web Performance — Flickr: Do you want to get faster?

Dear Flickr:

I have been wondering for sometime why downloads from your site seemed a little sluggish at times.

At first I blamed your unprecedented growth and success. For a little Vancouver startup (I am a BC boy myself), your entrance onto the stage of social networking applications has been phenomenal. The move from zero to infinity may have played a part in the performance I was seeing.

Nope. There was something else going on; I could see it every time I loaded a Flickr page in my browser. There was something else going on.

So today, I checked something out, and found the problem.

You need to enable persistent TCP connection on the static.flickr.com servers.

Now, that is the simple answer. I know that with large, web-based applications, enabling something as monumental as persistent connections could cause serious issues. If the architecture of the system was not designed to handle persistent connections, turning them on could cause the entire system to collapse.

There are legitimate, if mis-guided, reasons for disabled persistent connections. Some administrators believe that it is actually more efficient to have a client open a connection for every object. Easier to manage state, etc. The only problem is that in order to do that, you have to tune the systems serving data to shorten the amount of time a closed connection spends in a TIME_WAIT state.

When a TCP connection is closed, the socket is not immediately closed by the system in a default configuration. The TIME_WAIT state is the holding pen that these connections are pushed into. While in this state, the socket is locked and this may count against the incoming TCP connection queue, forcing the network stack to delay or reset new incoming connections.

Still, as Flickr is a worldwide company, the delay that the lack of persistent connections injects is astounding for locations in Asia. If you want to grow your business, and support more services, this will likely become a bottleneck very quickly.

Have a great weekend!

smp

Technorati Tags: , , , , ,

GrabPERF: New Agent Code in Testing

After a few month hiatus, I am starting to code for GrabPERF again. I need to exercise my brain; as I am a hobbyist code mangler, I have to take on a project every now and then to keep my not-so-l33t skillz honed.

The change to the agent is one of efficiency. The current production agent opens two database connections to run tests: one to retrieve the test configuration data; the other to insert the results of the tests. This means I loop through one set of database query results while doing inserts inside the loop on a second database connection.

This is stupid.

The new code opens a single connection to the database, retrieves the test configuration, dumps the results to an array of arrays, then inserts the data on the same connection. This is more efficient, as I use persistent connections and compression to MySQL to improve performance.

I have this running as TEST AGENT 1 from the Technorati #2 site.

Let me know if you see any madness…outside of Washington DC, and specifically with GrabPERF.

Technorati Tags: , ,

Copyright © 2024 Performance Zen

Theme by Anders NorenUp ↑