Books For Systems Geeks

I've been involved in lots of system releases over the years. Not only major versions and upgrades for the various companies I've worked for but the monthly or weekly minor (or, "minor") releases that are the norm in startups everywhere.

And like everyone else who's gone through those circuits, I've picked up a bunch of ad hoc, empirically proven knowledge about how to do these things. I know what's worked, and I know what hasn't.

But as the person helming the online systems for my studio's next major game, I thought it was a good idea to see what other people have to say about the complex art of launching a big system. Not just making it scale but the processes and practices that make it a smooth, worry-free (or as close as one can hope) launch. No sense in avoiding the wisdom of others, after all.

Here are my thoughts on a few of the books I've read recently.

The Art of Scalability - This is one of my favorites, and it's a book I'll come back to again and again. While it's light on actual technical details, it does a great job of explaining how scalability decisions are actually business decisions and how you need a culture of scalability, not just a decision to shove it all in at the end.

That sounds obvious, right? That's the other thing the book does really well: Encapsulating ideas you, like me, have probably learned on your own into nice, articulate concepts. I know about horizontal scaling, sharding, and splitting up servers based on functionality. But when they talk about their X-, Y-, and Z-axis scaling cube, it summarizes it quickly. Little phrases like "technology agnostic design" and "swim lanes" become keywords that you can quickly call up when you're thinking, "Something about this doesn't sound right."

Release It! - I was scribbling down notes constantly while reading this book. This is the distilled advice of someone who's seen lots of systems work well and poorly. It's one of the few tech books that is super relevant even five years or so after it was published. (Though it merely hints at the cloud technology that has sprung up since then.)

It documents a slew of patterns and antipatterns that will have you nodding your head constantly. A lot of the topics are things that you probably kind of know if you've done a bunch of launches. But this book takes them out of the realm of intuition and into concrete knowledge, real experience, and practical advice.

Its notion that the time before launch is a comparably short time in your product's life versus the time it spends live won't necessarily apply if, like me, you work in the games industry, where years of work go into a product whose user base drops off, under normal circumstances, rather than grows. But that does not diminish the value of the text. (And I won't always be in games.)

Scalability Rules - This book is the sequel to The Art of Scalability, though perhaps companion is a better word. Whereas the authors were light on tech and heavy on business in the first book, this book is all about the technical concepts you need to launch scalable systems. Again, lots of great advice that I scribbled into my notes. (This book has more to say on cloud systems than the original, earlier book did.)

Best of all, they provide a handy list of the 50 rules in order of "amount of risk management you get" and also provide a rough guess at the amount of work. Sort by most bang for the least buck, and you'll be knocking out a few in no time.

Scalable Internet Architectures - On the other hand, if you really want technical advice, this book gets way down to the nitty-gritty. There's a lot of good info here, but it's going to be most useful if you're on the IT/operations side of the fence versus the development side. That said, his argument that you should focus on scaling down (i.e., cutting costs) as well as scaling up has become a fixture in my architecture.

Continuous Delivery - The notion of continuous delivery, which means that your software is always ready for launch, is a compelling one. The authors are basically trying to convert the seemingly inevitable tension around a production release into a ho-hum experience that most of the team probably doesn't even need to be aware of.

It sounds great, but I'm not sure how much I'll be able to apply to my day-to-day work. The authors set a high bar when they cavalierly suggest that you should have 90 percent coverage just from unit tests. I used to feel fairly good about the fact that I had 40 percent coverage from unit tests and integration tests.

Still, I definitely came out with some ideas that will make our own launches more relaxed once I implement them.