Tuesday, November 23, 2010

Dynamic Feature Disabling

Often, when I'm rolling out a new, substantial feature, I add a config file property that marks it as enabled or disabled. This lets the team shut it down quickly if it's causing problems. It's a good practice, overall.

Keeping the status in a config file is easy to maintain, but it's problematic. Being able to "switch off" a feature in a config file really means, for most Java deployments, rebuilding and redeploying the war file, which can take a while if you're including a bunch of big libraries.

Recently, inspired by two pieces I read about how disabling features helps scale a site and mitigate risk, I came up with an enhancement to my older idea, and it's one of those brain-dead-obvious changes that I wish I had done before: A feature's enabled or disabled status now lives in the database.

I have a simple table (with Hibernate object and service) that stores a feature name, its current enabled status, and the date on which that status changed. Any other service or controller in the system can query that service to see if a given feature is enabled. (Since I run in the Spring framework, access to the service is a simple matter.)

What that means practically is that I can disable an entire subsystem across all instances of my application simply by updating a database row. (In the live system, these will probably be cached and refreshed every few minutes so that incoming requests aren't slowed down.) It also means I can get the status of the systems across an entire bank of servers with a simple query.

I put it in as groundwork for one feature — capturing real incoming traffic to a site to then play back as loadtesting scripts — but quickly back-ported a few other systems that have single points of entry. Our profiling system, for instance, can now be turned off application-wide simply by setting the enabled status of "Profiling" to false.

Of course, once you have subsystems that have a dynamic mechanism for checking their enabled/disabled status, it's a small step to enabling/disabling on a per-user basis. Which means we can gradually roll out new features, checking the load on the system at each pass and fixing bugs.

It's not an earth-shattering idea, but I was immediately entranced by the power it gives to my system.