Helping a colleague this week, we uncovered some odd behavior with a site whose performance he was analyzing. Upon first glance, it was clear that this site had a performance issue – they had HTTP persistence disabled. Immediate red flag in the areas of network overhead and geographic latency.
Further digging exposed something more sinister. It seems that HTTP persistence was only disabled for browsers with MSIE in the user-agent string. Even if the user-agent string was just MSIE, HTTP persistence was off.
The customer was very forthcoming and sent us their standard httpd.conf file. This showed no sign of the standard (and frustrating) global disabling of persistence for Internet Explorer.
Finally, it came to us. The customer had provided a simple network diagram, and there, just before packets hit the Internet, was a Layer 7 firewall. How did we know the Layer 7 firewall was the likely cause? Because this device was also the one that provided compression for the content going out to customers.
A Layer 7 firewall happily rewrites HTTP headers to reflect the nature of the compressed content (content-length or transfer-encoding: chunked) and to add the gzip flag (accept-encoding:gzip). Since this device was already doing this, it was pretty clear to us that it also had a rule that disabled HTTP persistence for anything with MSIE in the user-agent string.
This was a fine example of the complexity of the modern Web application infrastructure. In effect, there were two groups with different ideas of how Internet Explorer should be handled at the network layer, and neither of them seems to have talked to the other.
When you have a Web performance problem, indulge in a thought experiment. Create an imaginary incoming Web request and try to see if you can follow it through all the systems it touches on your system. Put it on a whiteboard, a mindmap, whatever works.
Then invite the system architects and network engineers in and get them to fill in the gaps.
No doubt that will lead to the “ah ha!” moment. If nothing else, it’s a good excuse to put pizza on the company card. But I have no doubt that you will walk away with a better understanding of your systems, which will make it easier for you to talk to all the people responsible for keeping your systems running.
TAKEAWAY: Just because the part of the Web application you work on is working fine, it may be affected by other components that are not tuned or configured for performance. Get to know the entire application at a high level.
