Scalable Internet Architectures

Yesterday was the second and last tutorial day at OSCON.  My morning session was on scalable internet architectures which was presented by Theo Schlossnagle.  This session was excellent.  It's hard to put together a meaningful summary as there was so much covered in the three or so hours.  Some of the topics that were covered include:

• proper configuration of static resources such as javascript files and images in order to maximize your bandwidth and make the most out of browser cache
• traditional and less traditional databases and other persistence solutions
• sharding, and why you want to avoid it when possible
• network routing with dos and don'ts
• concurrency and multi threading models
• staffing
• programming practices
• ..and much more that I didn't have time to jot down.

Here are some of the take-aways that come to mind:

Cache or Accuracy?

Proper caching can usually drastically improve performance but the drawback is information accuracy.  If information is cached for 5 minutes, when that information changes, it can be reported inaccurately for 5 minutes.  The proper balance will depend on the application.  You'll often see accuracy as a requirement, such as System Xyz must be accurate.  Period.  As if you would ever assume the opposite, but still, it needs to be explicit.  Your peers in fancy suits will tell you that you can't cache during a particularly expensive task because it needs to be accurate.  But if that certain operation takes 100 hits per second and you can show that implementing a 30 second cache can avoid the need to purchase more hardware, mr. fancy suit might decide that it's a reasonable trade off.  Better yet, 30 seconds may have been reasonable by his definition the entire time.

Premature Optimization

"We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.  Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified."
   -- Sir Tony Hoare (popularized by Donald Knuth)

This famous quote is stating that a good developer will know when time is worth spending on micro-optimizations.  Some developers will use optimization methods such as caching as a cop out for not spending the time it takes to design more efficient routines and algorithms.  You should design and write code to be as efficient as possible but really only worry about optimization techniques once you can prove that a part of the code is inefficient.  You will then be in a better position to properly analyse the problem and find a suitable optimization technique.  Of course this doesn't mean you should always ignore possible bottlenecks and performance issues, but a good developer will know when that premature optimization is appropriate.  

Mind your URL

This is usually not a proper URL:

http://www.domain.com/super_promo 

this is:

http://www.domain.com/super_promo/

and that trailing slash that your marketing team might forget to add to the URL that will be printed in flyers and typed into emails that will fill up people's spam folder could be a killer for your system.  This is extremely subtle, but most web servers will return a 302 "Object moved" response to redirect the user to the same address but with a trailing slash. That extra noise on the network could be expensive on a busy site.  It's not exactly practical to try to avoid it, the point is to plan for it.  If you load test your system using the trailing slash and think your system can handle a reasonable load with room to spare, you might be in a world of hate when your site gets Digg'ed with a URL that doesn't have a trailing slash.

A few other drive-bys:

• Your solution will never be perfect.  You can spend an extra two weeks or month on it and it will still not be perfect.  You are often better off to implement a solution and instrument the heck out of it so that you can analyse, learn and improve.
• Architectures are without specification of a vendor.  People tend to stick to what they know, it's in our nature.  It's important to put those personal biases aside to propose the best tool for the task.
• Decouple your services.  This doesn't just make your code easier to maintain, it makes it easier to deploy and scale.  But most importantly, if implemented properly, it will isolate failures.

There was so much more that was discussed during this presentation, this is just what comes to mind (and what I can make out from my digital scribbles).  The presentation was recorded, but that may only have been for the live feed.  If the video is made public, I strongly suggest you grab some popcorn and sit through the entire 3 hours.  In the least, the slides for this presentation (and others) should appear on this page shortly.