Category: Uncategorized

Greg Gershman from BlogDigger found a bug in the Index charts just now. Seems that the chart legend was out of whack with the lines on the chart.
Easy fix. Turns out I was submitting the test ids in a random order. The graph generation component was fine with this, but when I went to build the legend, MySQL ordered the dynamic legend build result in a proper ascending fashion.
Simple fix: when I convert the test ids to an array, I sort the array before proceeding.
Thanks Greg!

Over the last two days I have been trying to optimize the performance of the Index Charts/Graphs. I think I have found the fix, and you should see results for these charts in the sub 1-second range.
To explain how this works, I have to open up the GrabPERF data model and let you folks peer under the hood.
The Daily Index graphs are created using two tables: hourly_site and data. hourly_site is a rollup produced asynchronously at the end of every hour that aggregates the Arithmetic and Geometric means, as well as a count of successful measurements. data contains over 30 days of raw measurement data which is purged at the end of each day.
All of the days in the daily graph, and all of the hours in the hourly graph are pulled from hourly_site. That is with the exception of the most recent day (daily) and the most recent hour (hourly). These are pulled dynamically from data to ensure that all current results appear on the right-hand side of the graph.
With this design, performance was awful. I have tweaked the code as much as I feel comfortable doing (I am an analyst, not a developer!), but it was slow. I increased the size of the query cache, but the boost only lasted for a while. Then I realized that the data I most cared about was being pushed out of the cache by all of the other cached data.
I have now set MySQL to cache queries only on demand. This reduces the load and means that only the results I care about, and which are accessed most frequently, are cached.
When I restart the database, the Daily Blog Search chart takes about 5.5 seconds to generate, and the hourly chart takes about 30 seconds. Yikes!
After this initial pain, both charts take less than one second to be generated.
I am still working on it, so let me know if you see any weirdness.

UPDATE: Seems that the first query of every hour pays the performance penalty, and all other queries will not.

Got a quick note from Ed Lin at MaxMind that their free, country-level GEOIP database is updated monthly. You can get the updated files here.
This is in response to some out-of-date info in an old article (here and here) I wrote.
Most of this article is now deprecated, and the GrabIP service is the only remaining element on my site.

People laugh when I say that I am from a part of the world where, in my lifetime, snow has fallen in every month of the year.
Crowsnest Pass still recovering from freak storm
Power still out for 100 after Crowsnest Pass snowfall
Heavy snow cuts power
I don’t make this stuff up.

Well, I am starting to try and improve the performance of the slowest graphs on the site, the Index/Multi-Test graphs.
I have managed to completely eliminate the need for the temp tables I had been using, migrating to a pure associative array method. This is in the code that ensures that any tests that return no results for a given time period don’t cause PHPlot to throw a hissy fit.
There has to be a way to do it faster. I have a couple of methods I am considering, but arrays within arrays scare me just a little!
Let me know if you see any performance degradations.

Heh.

Count on SkyNews to bring you an unintended laugh.
Via: Fred Wilson
Then do the following:

Turns out that if you type “failure” into Google and hit “I feel lucky” you’ll get the official biography of, well, I don’t want to ruin the suprise.

Heh!
Via: Scoble

Here you go. Read this.
New Orleans has exposed the lie that is the Melting Pot.
The United States is one white, christian nation, indivisible, except when profit is to be made.
Then slave labour is allowed.

Ouch.
Technorati accelerator.
This one has to hurt. They are getting better…
Via: The Blog Herald

Oracle Agrees to Buy Siebel for $10.66 a Share
Vaults Oracle to Number 1 in Customer Relationship Management
Nice to be number one…unfortunately, the well is deep and it will be a while before this shotgun wedding finds any love.

The weekly GrabPERF Search Index Results are in.
Week of September 4-11, 2005

TEST                     RESULT  SUCCESS  ATTEMPTS
--------------------  ---------  -------  --------
PubSub - Search       0.5421396    99.72      5680
Google - Search       0.5741554   100.00      5681
Yahoo - Search        0.8028792    99.95      5678
eBay - Search         0.9255761   100.00      5681
Feedster - Search     1.2524916    99.86      5679
MSN - Search          1.2737601    98.40      5681
BestBuy.com - Search  1.3964269    99.91      5680
Technorati - Search   1.4128937    99.98      5681
BlogLines - Search    1.6020669    99.16      5681
Amazon - Search       1.9909649    99.74      3108
BENCHMARK RESULTS     2.0313721    99.50     76849
Blogdigger - Search   2.3336642    98.53      5577
Newsgator - Search    2.5181818    99.68      5681
IceRocket - Search    4.3043685    99.77      5680
Blogpulse - Search    7.4959768    98.40      5681

These results are based on data gathered from three remote measurement locations in North America. Each location takes a measurement approximately once every five minutes.
The measurements are for the base HTML document only. No images or referenced files are included.

Technorati: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO
IceRocket: GrabPERF, Web Performance, Web Performance Measurement, Web Performance Monitoring, search index, SEO