Had an on-site meeting with Dorion Carroll and the Technorati team. We discussed some ways to extend and expand the GrabPERF infrastructure, and improve the measurement gathering.
Look for changes in the next 6 months that could radically improve the system you know and love!
It has been at least a year since I last updated everyone on the state of GrabPERF. That’s because for most of the last year, the system has been rolling along without a hitch or a major systemic change. The last major change to the agent code was alomost exactly a year ago, when I added the ability to capture text matches.
It wasn’t until last week, however, that I allowed folks to be able to see the results of these text match failures. Let’s just say that motivation has been low and my real job has been keeping me busy.
I did want to share the growth, and mellowing of the system as it progresses into year three.
Total Measurements Per Day
Unique Tests Measured Per Day
Back in July 2005 when I started this grand experiment, I was gathering 10,000 measurements a day from less than forty tests. The system spiked at 390,000 measurements per day (April 2006) and 147 tests (June 2006).
Starting just after that, I started reducing the number of tests to improve system efficiency, and began developing the text match capability.
There have been some changes to the number of measurements, but on the whole, the system has been completely stable for the last 12 months.
As some of you may have noted, I have add some new features in the last 10 days, and re-organized the structure of the system to allow for better tracking of usage. Over the next few months, I will be attacking the code to make it process things more efficiently, but not substantially change the appearance or functionality of GrabPERF.
It has been noted by some commentators that the design doesn’t pop and sizzle. No AJAX, DHTML, or other flashy gizmos. Guess what? The system is designed to deliver data efficiently and effectively. And as someone who has seen the performance fall-out from badly delivered Web 2.0 implementations, I will stick with clunky and effective, as, in the end, you gotta put all those bytes on the wire.
For those that have stuck with the system over the last year, thanks. I enjoy delivering the best measurement data money can’t buy, and hope you stick around for the ride.
Last night, I got motivated.
Ok, I got manic. Goes with my life.
As a part of that mania, I had a breakthrough on how to present GrabPERF data that I’ve actually been collecting for nearly a year: text match failures.
GrabPERF has the ability to match text on page results using a standard PERL regex. By putting a regex into the measurement configuration, I can confirm that the data downloaded matches what should be there.
If there is not a match, the headers and page text are captured and inserted into a table. Up until yesterday, I was the only one who could view the data. Now, if you go to the graph configuration page (http://grabperf.org/measure_page.php?test=[insert test id here]), and see the following type of result, then click through the links.
If your graph configuration page says no text match configured, and you want one, let me know!
As a part of the reworking of the GrabPERF code, I removed the Google ads from all pages. They were an annoyance, and displayed items were incredibly irrelevant for the site.
If you use GrabPERF on a regular basis, the somewhat flaky navigation method has become second nature to you. In fact, to circumvent some of the idiosyncrasies, you have probably bookmarked your favourite pages.
Yesterday, I broke your links.
When I redesigned GrabPERF in February 2006, I had just discover the require function in PHP, and decided to build the entire the structure using a single container page as the framework, and individual functions called using URL parameters.
As time went on, my own “brilliance” started to get in the way of maintaining and updating the code. It took me 10-15 minutes to figure out how I constructed pages, and then find the right code to fix or update.
Yesterday, I got completely fed up with this structure.
Now, all functions have their own unique pages, making maintenance a snap. And as an added benefit, I can now effectively track the usage of individual pages, so I know where to through development efforts.
Some of the changes.
http://grabperf.org/homepage.php?page=compare&test=2&tests%5B%5D=276&tests%5B%5D=277&tests%5B%5D=279&tests%5B%5D=280
becomes
http://grabperf.org/homepage.php?page=scatter&test=277&hours=2
becomes
http://grabperf.org/scatter.php?test=277&hours=2
I apologize for the confusion that this may cause, but in the long run, this will help me make the code better, and more robust.
So, a colleague of mine told me about the YSlow plugin for Firebug yesterday. I had a look at it and it does do an amazing job of analyzing Web pages according to Steve Souders performance rules.
The only hitch in the get-along: a number of pages I visited when the plugin was activated were effectively unusable for up to a minute, presumably as YSlow analyzed them.
Great idea, all Web developers whould use it to put me out of a job. But the overall performance of the plugin will need to be improved before I re-install it.
A colleague of mine suggested that the Tour de France give up all pretense of being drug free, and embrace the performance-enhancing image it has developed so carefully over the last quarter century.
His idea was to have the racing teams sponsored by the major pharmaceutical firms, pitting one performance-enhancing approach against another, in a competition to demonstrate not the strength of the human spirit, but the power to manipulate the human body.
Samantha further suggested that they then abandon all of the rules that make the race civilized, and turn it into a free-for-all, a Tour de France, EXTREME EDITION!
That would get the le tour on Spike.
Well, now that I’m certified, the slope gets slippery very quickly.
We have until early August to get all of our docs to the lawyers before the August 17 deadline.
Many calls, pictures, doctors appointments, and huge expense.
Going to be a fun month.
Well folks, I got the good news last night: I am Department of Labor Certified.
This means that I can now actually apply for a Green Card.
Oh yeah, and based on some of the other happenings in the world of US Immigration (here and here), I am part of the group that is either going to be royally screwed over, or be part of some sort of general amnesty due to political manipulation and greed entering into the process.
Could be a fun month.
It seems that more than the usual immigration backlog reduction process has been at work in the USCIS. There are two likely scenarios that appear to be running around immigration circles these days, regarding the Green Card slot tease that has turned into such a furore.
The first is that the Department of State, which issues the Visas, was pressuring the USCIS to fill the Fiscal 2007 Green Card quota, something that has happened rarely in the last few years. What most people in the US don’t know is that most years, thousands of eligible Green Card slots simply disappear because the applications can’t be processed fast enough by the USCIS.
Recent events have highlighted this, and the Department of State may have applied pressure to USCIS to completely exhaust the 2007 pool, to avoid the embarrassment of having to explain to Congress why they can’t process applications faster.
The second reason is greed: as of August 1 2007, the government fees for Green card applications increases massively. For a family of four, the cost will increase by $2,500. So, by not allowing the flood of applications from all of those expectant people, they have guaranteed themselves a higher revenue stream for next year.
All things considered, the whole event smells.
Now, for the long-term affect on skilled immigrants, Microsoft has set the trend by announcing that it will be moving development over the border to Canada [here]. As a country with a skills-based immigration policy, highly-trained technical professionals feel welcomed and wanted in Canada, something that is not the case with the archaic and glacial immigration policy of the United States.
In the next 5-10 years, US companies will face a serious inability to recruit employees from anywhere other than the United States. Skilled professionals will simply not come to a country that actively discourages them from staying permanently and making a contribution.
The US policy policy will be a boon to Canada, Ireland, and other countries who actively seek and encourage skilled professional immigrants.
Tags: USCIS, Department of State, immigration, Green Card, skilled professionals, Canada, Ireland, brain drain
Copyright © 2025 Performance Zen
Theme by Anders Noren — Up ↑
