Category: Uncategorized

Dear Flickr:

I have been wondering for sometime why downloads from your site seemed a little sluggish at times.

At first I blamed your unprecedented growth and success. For a little Vancouver startup (I am a BC boy myself), your entrance onto the stage of social networking applications has been phenomenal. The move from zero to infinity may have played a part in the performance I was seeing.

Nope. There was something else going on; I could see it every time I loaded a Flickr page in my browser. There was something else going on.

So today, I checked something out, and found the problem.

You need to enable persistent TCP connection on the static.flickr.com servers.

Now, that is the simple answer. I know that with large, web-based applications, enabling something as monumental as persistent connections could cause serious issues. If the architecture of the system was not designed to handle persistent connections, turning them on could cause the entire system to collapse.

There are legitimate, if mis-guided, reasons for disabled persistent connections. Some administrators believe that it is actually more efficient to have a client open a connection for every object. Easier to manage state, etc. The only problem is that in order to do that, you have to tune the systems serving data to shorten the amount of time a closed connection spends in a TIME_WAIT state.

When a TCP connection is closed, the socket is not immediately closed by the system in a default configuration. The TIME_WAIT state is the holding pen that these connections are pushed into. While in this state, the socket is locked and this may count against the incoming TCP connection queue, forcing the network stack to delay or reset new incoming connections.

Still, as Flickr is a worldwide company, the delay that the lack of persistent connections injects is astounding for locations in Asia. If you want to grow your business, and support more services, this will likely become a bottleneck very quickly.

Have a great weekend!

smp

Technorati Tags: Flickr, persistent connections, TCP, http, web performance, scalability

After a few month hiatus, I am starting to code for GrabPERF again. I need to exercise my brain; as I am a hobbyist code mangler, I have to take on a project every now and then to keep my not-so-l33t skillz honed.

The change to the agent is one of efficiency. The current production agent opens two database connections to run tests: one to retrieve the test configuration data; the other to insert the results of the tests. This means I loop through one set of database query results while doing inserts inside the loop on a second database connection.

This is stupid.

The new code opens a single connection to the database, retrieves the test configuration, dumps the results to an array of arrays, then inserts the data on the same connection. This is more efficient, as I use persistent connections and compression to MySQL to improve performance.

I have this running as TEST AGENT 1 from the Technorati #2 site.

Let me know if you see any madness…outside of Washington DC, and specifically with GrabPERF.

Technorati Tags: GrabPERF, PERL, MySQL

My musical tastes run the gamut. The sublime includes Gregorian Chants, Baroque music, and ethereal soaring symphonies.

But I was raised in a small town in the BC interior. Metallica is in my blood.

Oh, you may try and deny your Metallica-ness, but you know it’s there. It’s the one that makes you want to drive fast on the 280 between San Mateo and Sand Hill Road. The one that sees you doing the rocker salute and hair-wave when the kids are out. The one that makes do those 10 extra reps at the gym.

I thought Motorhead and AC/DC could soothe the beast.

No hope.

Out comes Kill ‘Em All.

Now, all I need is a very fast car.

Technorati Tags: Metallica, Kill ’em all, Motorhead, AC/DC

Wandered into the neighbourhood Staples today for an office-supply fix — didn’t buy anything, just like the smell of paper and ink.
Down the pen aisle, I found a Fisher Space Pen.
For $15.99.
This is about $15 cheaper than anywhere else I have seen.
Now, when they start selling Moleskines or Rite in the Rain, I will be impressed.

It’s just about noon and the temperature is racing towards 95F.

The humidity is high enough that the dog went for a swim, not a walk this morning.

The atmosphere is currently: 80% Nitrogen, 19% Pollution, and 1% Oxygen.

I am trying to imagine why so many people want to live here.

Technorati Tags: massachusetts, heat, humidity, east coast, pollution

In Canada, the norm is to have a community recreation centre, usually with a pool, an arena and a curling rink.

In the parts of the US where we have lived, community pools are almost unheard of. Private fitness clubs are the only places with pools, other than private homes.

This explains why my seven year-old still can’t swim.

While in Victoria, we have been able to enjoy

• Juan de Fuca Recreation Centre
• Saanich Commonwealth Pool
• Recreation Oak Bay
• Esquimalt Recreation Centre

All of these locations have VERY reasonably priced swim sessions, as well as public lessons and swim teams.

And in Massachusetts and California? Well, I think that if you are privileged enough to live in Sudbury, MA or Palo Alto, CA, you might have access to a public pool.

Oh yeah, how’s that Protestant Work Ethic thing working out for you?

Technorati Tags: Canada, US, Recreation Centres, pools, protestant work ethic

This trip home has been an eye-opener. It has reminded me why I want to call myself Canadian, and live in this country.

There is a flow here. A bounce. A knowledge that we are NOT as doomed as the United States is. Canada is not crippled by a devastating and useless war, a moralistic and divisive leadership, or a fatalistic sense of empire.

Even in the middle of nowhere, I knew I was in another place. A place where things get done. Slowly, but they do get done. A place where people all understand that there is a lot more to the world than navel-gazing introspection and misplaced imperialism. The world is a place to celebrate, not conquer.

Canada is home. The United States is the place I reside.

The next five years are going to be very interesting. And I know what side of the 49th Parallel I want to be on.

Technorati Tags: Canada, United States

Gee, I guess you don’t want my business anymore…

Technorati Tags: American Express, Web performance, outage

We made it back to Victoria yesterday, after a 14-hour marathon drive across Southern BC. I can’t think of a place I would rather be right now.

Yesterday was a six-summit day.

• Salmo-Creston | Kootenay Pass (1774m)
• Nancy Greene (1575m)
• Paulson (1535m)
• Anarchist (1233m)
• Sunday Creek (1284m)
• Allison Pass (1342m)

By the time we rolled into Hope, Samantha was ranting that she never wanted to see another twisting mountain road again.

Two days to rest, then off to Seattle and back “home”.

Oh, and did I mention that the plunge into Osoyoos still scares the bejeezus out of me?

Technorati: Crowsnest+Highway Allison+Pass Paulson+Summit Kootenay+Pass Anarchist+Summit Osoyoos

We stayed at the Travelodge Merritt last night. A word of advice: don’t.

Pillows that were non-existent…errr, wafer-thin.

Ice machines that didn’t work.

Unheated pool (try about 75F — the boys were freezing in 10 minutes).

Breakfast? Sure, as long as you eat it in their cramped and soulless breakfast room, not your room. And selection was Superstore muffins and still-frozen bread.

Staff needs some hospitality

Avoid this place if at all possible. Please.

Technorati Tags: Travelodge Merritt, Merriitt BC, BC