I recently realized a minor ambition.
As I've said before, EA's loadtesting group doesn't meet all my needs: They come in near the end of a project and test the nearly-finished system, which means that it's a big headache all around. Fixing performance issues means rummaging through months — or even years — of code to try and suss out issues they're seeing. Also, the mere act of getting them up and running is usually a big disruption for one or more members of your team.
I had loadtesting scripts running already, but they suffered in a few ways. JMeter is good at what it does, but I want easier programmatic control of the scripts (especially for measuring things it's not so good at, such as the lag time on asynchronous processes under load). Also, I didn't have a loadtesting environment. I was running the scripts on a dev server against that same server. I could have put together an environment of multiple servers, but that would incur expense to the studio for machines that were largely idle.
Then I read an article somewhere about a company that uses EC2 as a dynamic loadtest environment. Yes, I thought, that's what I want.
Off and on over the last few weeks, I put together a Ruby script that can, with a single command, set up a full environment in Amazon's cloud-based servers. That one command instantiates an RDS database, an EC2 server that will act as our application server, and another EC2 server that acts as an instance for running The Grinder, a loadtesting tool that another team in the studio is using and that I liked for its extensibility. That one command also builds a custom version of our war file (to handle all the dynamic addresses you get from EC2/RDS), sets up access permissions between all the servers, and runs checks to make sure the whole environment is ready to go.
A second command fires off the loadtest itself, running the test script through The Grinder and collecting the results back to the local machine.
Finally, a third command shuts the whole system down. I do have one command that runs all three steps, but at the moment I tend to run each one manually.
Each run takes less than an hour for now, so the cost to the studio is on the order of $.28 per loadtesting run. Not something you'd want to run continually, but certainly something you could run once a day. As we move along in development, we'll probably need to spend more to set up bigger, more realistic environments, but that will also be when the studio has more budget for my project. The environments we need will scale up alongside the money we have.
One of the key pieces that I set up was the concept of a profile. A config file lets you specify which script to run and how many of each item to set up. So, for my first profile, I just set up one each of the different servers. But you could imagine some profile later down the road that sets up 2 of each machine with some sort of autoscaling system. And you could imagine one much further down the road that sets up something approximating our production environment and runs tests against it. All that is mostly supported.
The advantages to this system are huge. First, it makes real loadtesting feasible even early on in our project. But it also empowers developers to do rapid iteration on performance fixes without having to push them live and see if they make a difference. See a problem in the results, figure out a fix, run the script again to test. Repeat and then release when you can prove that your fix makes a difference. Because each script fires up its own environment, I'll be able to distribute performance fix work to my team.
This ad hoc loadtesting tool is already proving its worth, and I've only just started employing it. I can't wait to see how effective it is going forward.
Showing posts with label Philosophy. Show all posts
Showing posts with label Philosophy. Show all posts
Sunday, January 16, 2011
Saturday, October 2, 2010
Integration Tests
I've been a fan of unit tests for a couple of years now. Once I buckled down and wrote some, I realized how powerful they could be. They give you more stability out of the gate and they provide good regression tests. They allow me, the developer, to keep moving forward in tasks and minimize the time I spend going backward to debug or test older functions.
But I've always held the stance that unit tests shouldn't test things such as interactions with Hibernate or Spring. Those are well-tested frameworks with strong community support. Writing unit tests that touch those layers always struck me as a waste of time.
That said, most server-side Java code ends up moving into those layers. Virtually everything I write ends up talking to a database. And the server code itself is a loosely knit tapestry of services that chat with each other. That code can certainly have bugs — incorrect queries, edge conditions, bad assumptions, whatever. So how do I get tests against it?
Integration tests. Like unit tests, integration tests demonstrate that function A returns result X given inputs Y and Z. The difference is that in an integration test, you are testing the interaction between systems versus the simple in and out of a self-contained function.
I finally decided that I wasn't getting the test coverage I wanted with just unit tests. I was finding subtle bugs tucked away in database calls and service-to-service communication. Mock objects — code that presents the expected interface to a layer without providing the full functionality — only get you part of the way. If your mock object is keeping information in a local map instead of hitting the database, you're not testing that the query to the database returns the right thing.
Once I decided to incorporate integration tests, however, I went down a bit of rabbit hole. Your services need to talk to each other, so you need to wire them up to each other. But in the real, running code, I use Spring to manage all that. Fortunately, I can use Spring in the test environment, too. And Spring allows me to have a supplemental bean configuration file that overrides the production-code config file. So, for instance, I can have a bean named "telemetryService" that overrides the bean of the same name in the main config file. The test version doesn't actually send telemetry information. It effectively becomes a mock object. (Though in that particular case, it's done with a simple boolean.) I have an S3 service layer in the test config file that points to a test S3 bucket instead of our development one. Any beans that aren't overridden are pulled from the main file.
My integration tests do have to call a method to set up that configuration, however. Since the Spring config doesn't change at runtime, I have a static utility method that checks to see if the configuration is set up and, if it isn't, sets it up. This violates the principle that a test should not rely on existing state from another test, but that configuration load is time consuming.
But services don't just talk to each other. They also talk to the database. You don't want them to point to a database you actually use, so you need a test database. And that test database needs to a) be kept up to date, b) have some test data, and c) not let data linger (to ensure that no test ends up passing because of the coincidence of some other test running before it).
For the last two requirements, I incorporated DBUnit. DBUnit can do unit testing against the database, but I don't really need its form of that. Because I'm using Spring to set up my app in test mode, all my services can continue to use Hibernate for database work. (They can also use my JRuby-based query files.) But DBUnit offers two key services: It can take a data file of test data and load the database with it, and it can wipe any desired tables clean between test runs. When a test suite starts, it calls a method that ensures the Spring configuration is read (if it hasn't already been) and wipes/reloads the test database. When a test suite starts, I know that there is predictable data in the database and that that's the only data in the database.
What about between tests, though? I could do a wipe and reload around every test, but that's time-consuming. Instead, I have a setup method that configures a transaction and a teardown method that rolls back that transaction. All of this means that developers have to remember to do all this when writing an integration test, which I dislike, but I hope to integrate some tools that will automate that.
My test database also needs to be current with the latest schema. For that, I incorporated Autopatch, which will automatically get your database up to date when a code base runs against it.
So far, so good. But integration tests take a while to run, and I worry that developers might, faced with progressively longer test runs, disable tests on local builds. To keep our tests spry, I separate integration tests so that they're only run if you tell Buildr you're in the "longtests" environment. (That environment also specifies the test database properties.) The build system, which runs within minutes of any check-in, always runs them, however, so even a developer who gets hasty has an automatic monitoring.
All of this took three solid days to get working properly, but now that's it running, I got exactly what I wanted: the ability to have much deeper test coverage. The other day, I wrote a test that called service methods that altered the database and moved files around on S3. At each step in the chain, I could check the state of each component and verify that it had worked succesfully. And I knew that it was executing real queries and running real code. No mock objects.
The ability to have deeper test coverage means I've set myself a goal of writing those tests. Every feature I work on now has tests against it, even if they require database work or multiple services. And when I find a bug in my existing code, I force myself to write a test to reproduce it and then work on the code until the test passes. That way I get free regression testing whenever a check-in happens.
I have noticed one negative aspect, however. Unit tests, by their very nature, force you to write smaller and smaller methods that are easier to test. With no limit on the amount of activity you can do in an integration test, however, I find that my methods aren't channeled into smaller pieces. I try and do that because I know that ultimately produces more maintainable code, but my integration test system doesn't give an almost automatic push the way unit tests do.
Big methods or not, I still get to run a complete suite of extensive, realistic tests with a single command. That's pretty powerful.
But I've always held the stance that unit tests shouldn't test things such as interactions with Hibernate or Spring. Those are well-tested frameworks with strong community support. Writing unit tests that touch those layers always struck me as a waste of time.
That said, most server-side Java code ends up moving into those layers. Virtually everything I write ends up talking to a database. And the server code itself is a loosely knit tapestry of services that chat with each other. That code can certainly have bugs — incorrect queries, edge conditions, bad assumptions, whatever. So how do I get tests against it?
Integration tests. Like unit tests, integration tests demonstrate that function A returns result X given inputs Y and Z. The difference is that in an integration test, you are testing the interaction between systems versus the simple in and out of a self-contained function.
I finally decided that I wasn't getting the test coverage I wanted with just unit tests. I was finding subtle bugs tucked away in database calls and service-to-service communication. Mock objects — code that presents the expected interface to a layer without providing the full functionality — only get you part of the way. If your mock object is keeping information in a local map instead of hitting the database, you're not testing that the query to the database returns the right thing.
Once I decided to incorporate integration tests, however, I went down a bit of rabbit hole. Your services need to talk to each other, so you need to wire them up to each other. But in the real, running code, I use Spring to manage all that. Fortunately, I can use Spring in the test environment, too. And Spring allows me to have a supplemental bean configuration file that overrides the production-code config file. So, for instance, I can have a bean named "telemetryService" that overrides the bean of the same name in the main config file. The test version doesn't actually send telemetry information. It effectively becomes a mock object. (Though in that particular case, it's done with a simple boolean.) I have an S3 service layer in the test config file that points to a test S3 bucket instead of our development one. Any beans that aren't overridden are pulled from the main file.
My integration tests do have to call a method to set up that configuration, however. Since the Spring config doesn't change at runtime, I have a static utility method that checks to see if the configuration is set up and, if it isn't, sets it up. This violates the principle that a test should not rely on existing state from another test, but that configuration load is time consuming.
But services don't just talk to each other. They also talk to the database. You don't want them to point to a database you actually use, so you need a test database. And that test database needs to a) be kept up to date, b) have some test data, and c) not let data linger (to ensure that no test ends up passing because of the coincidence of some other test running before it).
For the last two requirements, I incorporated DBUnit. DBUnit can do unit testing against the database, but I don't really need its form of that. Because I'm using Spring to set up my app in test mode, all my services can continue to use Hibernate for database work. (They can also use my JRuby-based query files.) But DBUnit offers two key services: It can take a data file of test data and load the database with it, and it can wipe any desired tables clean between test runs. When a test suite starts, it calls a method that ensures the Spring configuration is read (if it hasn't already been) and wipes/reloads the test database. When a test suite starts, I know that there is predictable data in the database and that that's the only data in the database.
What about between tests, though? I could do a wipe and reload around every test, but that's time-consuming. Instead, I have a setup method that configures a transaction and a teardown method that rolls back that transaction. All of this means that developers have to remember to do all this when writing an integration test, which I dislike, but I hope to integrate some tools that will automate that.
My test database also needs to be current with the latest schema. For that, I incorporated Autopatch, which will automatically get your database up to date when a code base runs against it.
So far, so good. But integration tests take a while to run, and I worry that developers might, faced with progressively longer test runs, disable tests on local builds. To keep our tests spry, I separate integration tests so that they're only run if you tell Buildr you're in the "longtests" environment. (That environment also specifies the test database properties.) The build system, which runs within minutes of any check-in, always runs them, however, so even a developer who gets hasty has an automatic monitoring.
All of this took three solid days to get working properly, but now that's it running, I got exactly what I wanted: the ability to have much deeper test coverage. The other day, I wrote a test that called service methods that altered the database and moved files around on S3. At each step in the chain, I could check the state of each component and verify that it had worked succesfully. And I knew that it was executing real queries and running real code. No mock objects.
The ability to have deeper test coverage means I've set myself a goal of writing those tests. Every feature I work on now has tests against it, even if they require database work or multiple services. And when I find a bug in my existing code, I force myself to write a test to reproduce it and then work on the code until the test passes. That way I get free regression testing whenever a check-in happens.
I have noticed one negative aspect, however. Unit tests, by their very nature, force you to write smaller and smaller methods that are easier to test. With no limit on the amount of activity you can do in an integration test, however, I find that my methods aren't channeled into smaller pieces. I try and do that because I know that ultimately produces more maintainable code, but my integration test system doesn't give an almost automatic push the way unit tests do.
Big methods or not, I still get to run a complete suite of extensive, realistic tests with a single command. That's pretty powerful.
Tuesday, September 28, 2010
Config Files As Programs
While working on a few recent tasks, I've hit upon a small but powerful technique: My configuration files are actually programs in their own right.
Expression languages of some form are de rigueur in modern frameworks. Especially in server code, where you need to deploy the same code base to different servers and have them point to different databases, mail servers, or whatever. In Spring, for instance, you can write something like
But my technique is vastly more powerful. I'm sure others have thought of it, too, but it sort of snuck up on me.
The first time I used this trick was when I put my work's SQL into Ruby scripts. I did that for readability's sake, but as I added more queries, I began to do the things one does does with programs. I refactored two similar queries into a method that took a differentiating argument and returned the query. I declared constants to hold common values. I started to ponder a Ruby module that I could import for various utility functions. The end result from the Java code's view was a set of query definitions. But within the query definition itself, I had normal programming tools at my disposal.
On a personal project (which I hope to write about soon), I decided to use YAML for the config files. I've gotten used to the format for config files, thanks to my work with AppEngine, and Ruby can easily work with it. My original idea for the config file was just fixed dates and times when I wanted my script to wake up and record an Internet radio stream. But then I realized that I wanted to record some streams on every weekday or on the first Sunday of the month.
So I thought, what if, before I feed the file to the YAML parser, I first run it through ERB, Ruby's powerful templating system? As long as the YAML parser sees something that looks like YAML, it doesn't matter how it gets there, right?
Right. I defined a method in my config file — in my config file! — that would return the next weekday after the given date. Then, when defining the value for a field, I called that method to calculate the value. My "YAML" file looks something like:
Expression languages of some form are de rigueur in modern frameworks. Especially in server code, where you need to deploy the same code base to different servers and have them point to different databases, mail servers, or whatever. In Spring, for instance, you can write something like
${project.database.url} and it will assume you mean to use not the literal string but the value of the project.database.url property, which is probably different in your dev environment and your production environment.But my technique is vastly more powerful. I'm sure others have thought of it, too, but it sort of snuck up on me.
The first time I used this trick was when I put my work's SQL into Ruby scripts. I did that for readability's sake, but as I added more queries, I began to do the things one does does with programs. I refactored two similar queries into a method that took a differentiating argument and returned the query. I declared constants to hold common values. I started to ponder a Ruby module that I could import for various utility functions. The end result from the Java code's view was a set of query definitions. But within the query definition itself, I had normal programming tools at my disposal.
On a personal project (which I hope to write about soon), I decided to use YAML for the config files. I've gotten used to the format for config files, thanks to my work with AppEngine, and Ruby can easily work with it. My original idea for the config file was just fixed dates and times when I wanted my script to wake up and record an Internet radio stream. But then I realized that I wanted to record some streams on every weekday or on the first Sunday of the month.
So I thought, what if, before I feed the file to the YAML parser, I first run it through ERB, Ruby's powerful templating system? As long as the YAML parser sees something that looks like YAML, it doesn't matter how it gets there, right?
Right. I defined a method in my config file — in my config file! — that would return the next weekday after the given date. Then, when defining the value for a field, I called that method to calculate the value. My "YAML" file looks something like:
start_date: <%next_weekday.strftime('%Y-%m-%d')%> 18:00. Which won't give you anything useful from the YAML parser. But run it through the ERB parser, and the YAML parser sees start_date: 2010-09-28 18:00, which is perfectly valid YAML syntax and is what you intend for the value to be.
Sunday, August 1, 2010
Watch Your Users Use
Time and time again, I re-learn this lesson: Nothing makes your system more usable than watching your users use it.
As I mentioned in my last post, I'm building a telemetry system on top of Google AppEngine so that my studio can understand how our games will be played. As I've demoed pieces of it, I've put together web pages and interactive charts and other nice little visualizations. I've made JavaScript libraries to make it easy to build those sorts of things, and I've demoed some of the prettier ones to the stable.
But I also put in a way to export data as tab-delimited fields. I figured this would be a good last resort if the JavaScript version wasn't quite there yet, or if we hadn't written a specialized view on the data, but I figured we'd favor those.
Naturally, then, my principal users have all just used the tab-delimited export and not pushed beyond that. In particular, they grab a bunch of tab-delimited data, import it into Excel, and then do manipulations on it there. Though there are macros and various other things, it still seems to take a senior engineer a couple of hours to compile all the data for his weekly reports.
I know this because I sit with him a fair amount as he explains his process.
The other day, I was thinking of his process and realized that Python can create Excel spreadsheets. So I could remove one minor barrier by just letting him get the data in Excel format instead of tab-delimited text. Easy.
But that doesn't buy him much. For his weekly reports, he trims some of the unneccesary columns (my exports grab all the fields and dump them out, including some that are system-level fields), runs some macros to insert formulas to give him percentages, sorts the data, and then copies and pastes that into an email. He does that for about 10-15 reports, in one form or another.
What if, I thought, I could let him define the format of the spreadsheet the system generates? Then he could say, "give me this data, but put it in this layout." That would shave a big chunk of time from his flow.
One thing about working with him is that he's not an online engineer. This turns out to be very good, because it makes me think in terms of letting him modify config files instead of modifying server code. (A medium-term goal is building a real interface on top of my system so that anyone in the studio can interact with it, but for now, config files are how we do things.) One of our other online engineers used the system by writing a custom request handler that interacted with the AppEngine datastore directly to generate a customized HTML page. Very neat, but if he were my only user, my system wouldn't be very evolved. I'd just say to someone new, "write a request handler."
I mentally sketched out what the config file would need to contain, what abilities it would need to give him, and within 45 minutes had a system in place that will let him generate something much closer to the final spreadsheets he needs for his reports.
A key component of the new feature is the Django templating language that comes with AppEngine. I actually dislike the language for HTML generation, but for letting users construct simple templates, it's pretty good. One important feature: It can render a template contained in a string, not just one contained in a file. And you can give the renderer a context, filling in some variables that are accessible within the template. In my case, I create a context that contains the current piece of aggregation data (which is a summary object of a large number of events) and the current row number.
From his perspective, to generate a report about how much damage a creature is doing in our testing, he creates a YAML config file that looks like this:
Then, when he requests data in Excel format, he can tell it to go through the layout he specifies. Rather than getting a dump of every piece of data in each object, he gets a spreadsheet with just the information he needs. The {{}} is part of the templating system and translates as "spit out the result of this expression."
Let's say he also wanted to capture the number of times that creature did damage. That's actually available to him in another field in the aggregation, but let's say it wasn't. My system also supports formulas. So he could do this:
That formula will become, in the output for the first row,
This is still a long way from user-friendly, but I think it's going to give him back an hour and a half of time each week. And building the system this way means that a user-friendly version just needs to write the format into this config file, and away it will go.
As I mentioned in my last post, I'm building a telemetry system on top of Google AppEngine so that my studio can understand how our games will be played. As I've demoed pieces of it, I've put together web pages and interactive charts and other nice little visualizations. I've made JavaScript libraries to make it easy to build those sorts of things, and I've demoed some of the prettier ones to the stable.
But I also put in a way to export data as tab-delimited fields. I figured this would be a good last resort if the JavaScript version wasn't quite there yet, or if we hadn't written a specialized view on the data, but I figured we'd favor those.
Naturally, then, my principal users have all just used the tab-delimited export and not pushed beyond that. In particular, they grab a bunch of tab-delimited data, import it into Excel, and then do manipulations on it there. Though there are macros and various other things, it still seems to take a senior engineer a couple of hours to compile all the data for his weekly reports.
I know this because I sit with him a fair amount as he explains his process.
The other day, I was thinking of his process and realized that Python can create Excel spreadsheets. So I could remove one minor barrier by just letting him get the data in Excel format instead of tab-delimited text. Easy.
But that doesn't buy him much. For his weekly reports, he trims some of the unneccesary columns (my exports grab all the fields and dump them out, including some that are system-level fields), runs some macros to insert formulas to give him percentages, sorts the data, and then copies and pastes that into an email. He does that for about 10-15 reports, in one form or another.
What if, I thought, I could let him define the format of the spreadsheet the system generates? Then he could say, "give me this data, but put it in this layout." That would shave a big chunk of time from his flow.
One thing about working with him is that he's not an online engineer. This turns out to be very good, because it makes me think in terms of letting him modify config files instead of modifying server code. (A medium-term goal is building a real interface on top of my system so that anyone in the studio can interact with it, but for now, config files are how we do things.) One of our other online engineers used the system by writing a custom request handler that interacted with the AppEngine datastore directly to generate a customized HTML page. Very neat, but if he were my only user, my system wouldn't be very evolved. I'd just say to someone new, "write a request handler."
I mentally sketched out what the config file would need to contain, what abilities it would need to give him, and within 45 minutes had a system in place that will let him generate something much closer to the final spreadsheets he needs for his reports.
A key component of the new feature is the Django templating language that comes with AppEngine. I actually dislike the language for HTML generation, but for letting users construct simple templates, it's pretty good. One important feature: It can render a template contained in a string, not just one contained in a file. And you can give the renderer a context, filling in some variables that are accessible within the template. In my case, I create a context that contains the current piece of aggregation data (which is a summary object of a large number of events) and the current row number.
From his perspective, to generate a report about how much damage a creature is doing in our testing, he creates a YAML config file that looks like this:
header:
- value: Name
- value: Avg Damage
- value: Total Damage
row:
- value: "{{aggregation.groupByValue}}"
- value: "{{aggregation.average}}"
- value: "{{aggregation.sum}}"
Then, when he requests data in Excel format, he can tell it to go through the layout he specifies. Rather than getting a dump of every piece of data in each object, he gets a spreadsheet with just the information he needs. The {{}} is part of the templating system and translates as "spit out the result of this expression."
Let's say he also wanted to capture the number of times that creature did damage. That's actually available to him in another field in the aggregation, but let's say it wasn't. My system also supports formulas. So he could do this:
header:
- value: Name
- value: Count of Attacks
- value: Avg Damage
- value: Total Damage
row:
- value: "{{aggregation.groupByValue}}"
- formula:"ROUND(D{{row_num}} / C{{row_num}},0)"
- value: "{{aggregation.average}}"
- value: "{{aggregation.sum}}"
That formula will become, in the output for the first row,
ROUND(D1/C1,0). This is still a long way from user-friendly, but I think it's going to give him back an hour and a half of time each week. And building the system this way means that a user-friendly version just needs to write the format into this config file, and away it will go.
Saturday, January 23, 2010
Polyglot Programming
I recently read through The ThoughtWorks Anthology, a collection of essays by Big Thinkers in the realm of systems design. The essays were largely interesting, but one in particular resonated with me: Polyglot Programming. The author made a compelling case for using the Java Virtual Machine — a robust, mature, well-tested infrastructure — as a platform in which any number of languages can co-exist.
Java's a good language, of course, but it's not good at everything. Why not mix in other languages that can run in the virtual machine but offer strengths in the face of Java's weaknesses, asked the author. I've toyed with this idea before, especially with adding Scala's potential for highly concurrent code, but the essay lit a new fire in me.
I came up with a way to try out the concept. We have a bunch of queries in our service layers, and Java blows when it comes to formatting long strings. Without the ability to have one string span multiple lines, you end up with something like this:
Easy reading, right? Not only is it annoying to read, it's error prone. I almost always forget a space in one of these long strings, causing SQL exceptions that don't show up until runtime.
Ruby, like many other scripting languages, allows for a "here document" which basically says, "Treat this text following double less-than signs as a double quote and just pull in everything after it until you see the same text again." In Ruby, you might write the query above as follows:
Which is more readable. I admit this is not a monumental problem, but it did offer an opportunity.
Enter JRuby. JRuby is a Ruby interpreter written in Java. Curiously, it now outperforms the C-based interpreter in lots of benchmarks, which is not only a testament to the maturity of the JVM but to the dedicated open-source team that have devoted themselves to improving JRuby. JRuby's main benefit is that you can access the sweeping Java API from within your Ruby scripts, but you can also invoke Ruby scripts from your Java code.
I made a new class called
What's that
But it gets more complicated. If you have a SQL query that returns everything you need to construct a Hibernate object, you need to tell Hibernate what kind of object it is. (You don't need to do this for HSQL.) I added an
Here's the final flow. Some method in the service layer wants to run a query. It calls a method in the base class called
So how does it work? Well, on the one hand, it accomplishes what I wanted. My queries have been factored out into new files, and they've been re-produced in a format that's easier to read and less error-prone. The entire rest of the system is ignorant of their source. It makes the case for adding languages that have strengths (in this case, the relatively minor advantage of string literals that can span multiple lines) to a deploy.
But on the other hand, JRuby seems to have added a sizable chunk of memory to our app. Shortly after I put in this system, our dev server started running into
I've started looking at Groovy as an alternate. At least if I go that route, only
Java's a good language, of course, but it's not good at everything. Why not mix in other languages that can run in the virtual machine but offer strengths in the face of Java's weaknesses, asked the author. I've toyed with this idea before, especially with adding Scala's potential for highly concurrent code, but the essay lit a new fire in me.
I came up with a way to try out the concept. We have a bunch of queries in our service layers, and Java blows when it comes to formatting long strings. Without the ability to have one string span multiple lines, you end up with something like this:
String query = "SELECT table_a.*,table_b.* FROM table_a,table_b,table_c " +
"WHERE table_a.some_column = table_b.some_column " +
"AND table_b.some_column = table_c.some_column " +
"AND table_c.id = :idValue";
Easy reading, right? Not only is it annoying to read, it's error prone. I almost always forget a space in one of these long strings, causing SQL exceptions that don't show up until runtime.
Ruby, like many other scripting languages, allows for a "here document" which basically says, "Treat this text following double less-than signs as a double quote and just pull in everything after it until you see the same text again." In Ruby, you might write the query above as follows:
query = <<QUERY
SELECT
table_a.*,
table_b.*
FROM
table_a,
table_b,
table_c
WHERE
table_a.some_column = table_b.some_column
AND table_b.some_column = table_c.some_column
AND table_c.id = :idValue
QUERY
Which is more readable. I admit this is not a monumental problem, but it did offer an opportunity.
Enter JRuby. JRuby is a Ruby interpreter written in Java. Curiously, it now outperforms the C-based interpreter in lots of benchmarks, which is not only a testament to the maturity of the JVM but to the dedicated open-source team that have devoted themselves to improving JRuby. JRuby's main benefit is that you can access the sweeping Java API from within your Ruby scripts, but you can also invoke Ruby scripts from your Java code.
I made a new class called
QueryContainer that would serve as a facade for managing the Ruby invocations and giving Hibernate Query objects back to the service layer. No other layer in the code would need to know about invoking Ruby: QueryContainer would translate the scripts into objects useful elsewhere in the system. Inside each Ruby script, I made a class to act as a namespace (because I opted for a singleton of the Ruby interpreter instead of multiple copies), and then inside each class defined hash literals that looked something like this:
QUERY_1 = {
:type=>AppConstants::SQL,
:query => <<QUERY
SELECT
table_a.*,
table_b.*
FROM
table_a,
table_b,
table_c
WHERE
table_a.some_column = table_b.some_column
AND table_b.some_column = table_c.some_column
AND table_c.id = :idValue
QUERY
}
What's that
AppConstants::SQL thing? AppConstants is a Java class in our system that has some globally useful constants. Because it's JRuby, I can use constants from my Java classes. We have two query languages in our system: normal SQL and Hibernate's abridged SQL. QueryContainer needs to know which query language it is because Hibernate defines a createQuery method for HSQL and a createSQLQuery method for SQL.But it gets more complicated. If you have a SQL query that returns everything you need to construct a Hibernate object, you need to tell Hibernate what kind of object it is. (You don't need to do this for HSQL.) I added an
entityClass key to the SQL hash literals, and had it reference a Java class object (.java_class when you're in JRuby, since .class has meaning in the Ruby world. In other words:
QUERY_1 = {
:type => AppConstants::SQL,
:entityClass => BusinessObject.java_class,
…
}
Here's the final flow. Some method in the service layer wants to run a query. It calls a method in the base class called
getQueryForKey, passing in the query key it wants. That base class method calls a similar method on a QueryContainer instance variable held by the base class. QueryContainer was initialized with the Ruby script that will act as a resource, and it reaches into it to find the keys in the hash literal with the same name as the key that's been moving through the chain. e.g.,: QuerySet1::QUERY_1[:query]. If it's an HSQL query (QuerySet1::QUERY_1[:type]), QueryContainer just constructs a regular Query object. If it's a SQL query, QueryContainer constructs a SQLQuery object and calls addEntity on it, passing in the Java class from the :entityClass key of the hash literal.So how does it work? Well, on the one hand, it accomplishes what I wanted. My queries have been factored out into new files, and they've been re-produced in a format that's easier to read and less error-prone. The entire rest of the system is ignorant of their source. It makes the case for adding languages that have strengths (in this case, the relatively minor advantage of string literals that can span multiple lines) to a deploy.
But on the other hand, JRuby seems to have added a sizable chunk of memory to our app. Shortly after I put in this system, our dev server started running into
OutOfMemory errors on a regular basis, a process that I've contained somewhat by disabling some other systems. And this is with a singleton of the interpreter. I've found little information about this, and so I'm wondering if JRuby is the way to go. I haven't hooked up a profiler yet to determine the real source, but it's the only thing that's changed.I've started looking at Groovy as an alternate. At least if I go that route, only
QueryContainer needs to change.
Sunday, November 29, 2009
Knowledge Creates Possibilities
When I first left college 16 years ago — a bio major with a head for computers — I worked for Bank of America. I helped design software for a lucrative trading desk. One of the strongest memories I have from that time is showing a piece of software to the head of the trading desk, a tall, nervous, white-haired man who stuttered when his thoughts outpaced his ability to voice them. My boss and I had taken his requirements and created software that met them. He looked at the software and said, "That's nice. I wonder: Could it do this, too?"
I don't remember what the "this" was, but it wasn't a feature he had originally mentioned. This happened again. And again. And again. Couldn't he just tell us what he wanted the first time around? It took me a year or so to realize what was happening: He couldn't define the features he wanted because he didn't know what was possible. Once he saw one feature, it expanded his horizon of what was doable, and he could then make the mental leap to bigger, and ultimately better, feature ideas.
It may be the most important thing I learned on that job.
It's certainly one reason that Agile programming made sense to me when I first learned about it: The mindset urges you to get functional software in front of customers as early as possible so that they can give feedback. The stated reason for this idea is that customers can tell you what's important and get what they want. I have always added a mental note: It also lets them know what's possible.
But there's no line between users and developers here. I think everyone works this way. Well, everyone except those few people who are truly visionaries.
A co-worker and I are both prototyping features that require APIs. Being modern web programmers, we're building them as RESTful APIs. REST simply means that you present data organized hierarchically by "nouns," or resources (compare flickr.com/photos and flickr.com/photos/melissanicole, for instance). You typically present it as XML or JSON via HTTP so that even inexperienced programmers can retrieve the data and parse it.
He showed me a page he had set up at the top level of his API. It didn't reference any particular resource. Instead, it presented a list of all the calls in the API. I thought it was a good idea: Rather than have the documentation in some separate place, he had put it right in front of the user. At the time, he was updating this manually. I've maintained API documents before, and it's a pain in the ass.
But seeing his page, I realized that I could generate a similar page by having the system itself analyze its own internals. Look at all the request mappings you've set up, see which ones are under the path for your API calls, and add them to the page. Voila! Documentation that keeps itself up to date. (See below if you want some of the details.)
It took some effort. I dug around in Spring code, read through tips on forums, and yelled at my computer. But by the end of the day, I had a page of API documentation, in the normal format of the API, that I would never have to maintain or remember to update. New API calls would get added without me thinking about it. A simple annotation on the method that handles the request gives a human-readable definition of what the call does. (I also added an exclude capability for calls that we don't want to expose.) You could argue that the time I spent on this feature was greater than the time I would spend maintaining the documentation. That may be true.
But a week later, I had an epiphany.
As I've said, I'm experimenting with the idea of having load testing scripts that run early and often for new features. To that end, I had a JMeter script where I manually added the API calls as I added them to the system.
See the problem?
What if I could make a load testing script that would hit that top-level page and use it to figure out all the other calls that were possible? I'd have a script that would, with no maintenance on my part, load test every API call as I added it.
Or when someone else added one. They wouldn't even have to know about the load testing script, and their call would get load tested.
It took a bit of digging in JMeter's sparse documentation, but I got it working after a few hours. I add an API call, it shows up in the generated documentation page, and then the loadtesting script picks it up. Beyond my initial excitement of seeing the flow work, I stopped thinking about it. (Again, see below for some tips.)
I felt like the head of the trading desk from all those years ago. I didn't start with a vision of a load testing script that would dynamically adjust itself. But when I implemented one feature, that script came into view. And now I will always see it as a possibility, and that knowledge will create new possibilities I can't see today.
A Self-Documenting API
I'm using Spring 3.0 as a framework, and that's what made this work. If you're in the same boat, there are a few things that will frustrate you along the way.
You'll want a reference to the DefaultAnnotationHandlerMapping object that Spring maintains. However, you can't autowire a reference into your class, because Spring doesn't, by default, register the DefaultAnnotationHandlerMapping as a bean. But you can create a bean of your own for that class, and then it will work. (Note that you'll probably want to Autowire for an array of AbstractUrlHandlerMapping objects, in case you add more down the road.)
DefaultAnnotationHandlerMapping will give you a map of all the request mappings in your system, but the values in that map are controllers, not methods. You'll have to use reflection to go through all the methods in the class and figure out which ones actually have the @RequestMapping annotation for the given request mapping.
On that note, keep in mind that you may actually be getting a CGLIB proxy for the controller, so you'll need to get the actual class name in order to use reflection.
A Self-Discovering Load Testing Script
Because my API documentation page returns its results in XML, I was able to use JMeter's XPath Extractor to derive all the URLs to hit. Couple that with a ForEach controller, and you've got a loop that will iterate over each URL in the document. Put an Http Request Sampler in that loop, set the path field to the iteration variable created by the controller, and you're hitting whatever the current URL is. That's the basic format.
Spring's request mapping syntax looks something like /foo/bar/{id}, where {id} gets replaced at request time by whatever's in that space. (This is, incidentally, an awesome feature.) But /foo/bar/{id} is not a useful URL. I wrote a preprocessor for the Http Request Sampler that replaces {id} with some other value. You may be clever enough to do it with a Regex preprocessor; I did it with a BeanShell script that takes the current path — /foo/bar/{id} — and transforms it to /foo/bar/1234. Then it puts that value into a new variable called correctedPath.
JMeter allows you to use variables in the name field for a component. This is very useful. If you leave the sampler's name as Http Request Sampler, every single URL will get put in the "Http Request Sampler" test name, which will make your load testing reports useless. If you set the sampler's name to — using the above example — ${correctedPath}, the test name will get set to /foo/bar/1234, and each discovered URL will get its own line item in your report.
I don't remember what the "this" was, but it wasn't a feature he had originally mentioned. This happened again. And again. And again. Couldn't he just tell us what he wanted the first time around? It took me a year or so to realize what was happening: He couldn't define the features he wanted because he didn't know what was possible. Once he saw one feature, it expanded his horizon of what was doable, and he could then make the mental leap to bigger, and ultimately better, feature ideas.
It may be the most important thing I learned on that job.
It's certainly one reason that Agile programming made sense to me when I first learned about it: The mindset urges you to get functional software in front of customers as early as possible so that they can give feedback. The stated reason for this idea is that customers can tell you what's important and get what they want. I have always added a mental note: It also lets them know what's possible.
But there's no line between users and developers here. I think everyone works this way. Well, everyone except those few people who are truly visionaries.
A co-worker and I are both prototyping features that require APIs. Being modern web programmers, we're building them as RESTful APIs. REST simply means that you present data organized hierarchically by "nouns," or resources (compare flickr.com/photos and flickr.com/photos/melissanicole, for instance). You typically present it as XML or JSON via HTTP so that even inexperienced programmers can retrieve the data and parse it.
He showed me a page he had set up at the top level of his API. It didn't reference any particular resource. Instead, it presented a list of all the calls in the API. I thought it was a good idea: Rather than have the documentation in some separate place, he had put it right in front of the user. At the time, he was updating this manually. I've maintained API documents before, and it's a pain in the ass.
But seeing his page, I realized that I could generate a similar page by having the system itself analyze its own internals. Look at all the request mappings you've set up, see which ones are under the path for your API calls, and add them to the page. Voila! Documentation that keeps itself up to date. (See below if you want some of the details.)
It took some effort. I dug around in Spring code, read through tips on forums, and yelled at my computer. But by the end of the day, I had a page of API documentation, in the normal format of the API, that I would never have to maintain or remember to update. New API calls would get added without me thinking about it. A simple annotation on the method that handles the request gives a human-readable definition of what the call does. (I also added an exclude capability for calls that we don't want to expose.) You could argue that the time I spent on this feature was greater than the time I would spend maintaining the documentation. That may be true.
But a week later, I had an epiphany.
As I've said, I'm experimenting with the idea of having load testing scripts that run early and often for new features. To that end, I had a JMeter script where I manually added the API calls as I added them to the system.
See the problem?
What if I could make a load testing script that would hit that top-level page and use it to figure out all the other calls that were possible? I'd have a script that would, with no maintenance on my part, load test every API call as I added it.
Or when someone else added one. They wouldn't even have to know about the load testing script, and their call would get load tested.
It took a bit of digging in JMeter's sparse documentation, but I got it working after a few hours. I add an API call, it shows up in the generated documentation page, and then the loadtesting script picks it up. Beyond my initial excitement of seeing the flow work, I stopped thinking about it. (Again, see below for some tips.)
I felt like the head of the trading desk from all those years ago. I didn't start with a vision of a load testing script that would dynamically adjust itself. But when I implemented one feature, that script came into view. And now I will always see it as a possibility, and that knowledge will create new possibilities I can't see today.
A Self-Documenting API
I'm using Spring 3.0 as a framework, and that's what made this work. If you're in the same boat, there are a few things that will frustrate you along the way.
You'll want a reference to the DefaultAnnotationHandlerMapping object that Spring maintains. However, you can't autowire a reference into your class, because Spring doesn't, by default, register the DefaultAnnotationHandlerMapping as a bean. But you can create a bean of your own for that class, and then it will work. (Note that you'll probably want to Autowire for an array of AbstractUrlHandlerMapping objects, in case you add more down the road.)
DefaultAnnotationHandlerMapping will give you a map of all the request mappings in your system, but the values in that map are controllers, not methods. You'll have to use reflection to go through all the methods in the class and figure out which ones actually have the @RequestMapping annotation for the given request mapping.
On that note, keep in mind that you may actually be getting a CGLIB proxy for the controller, so you'll need to get the actual class name in order to use reflection.
A Self-Discovering Load Testing Script
Because my API documentation page returns its results in XML, I was able to use JMeter's XPath Extractor to derive all the URLs to hit. Couple that with a ForEach controller, and you've got a loop that will iterate over each URL in the document. Put an Http Request Sampler in that loop, set the path field to the iteration variable created by the controller, and you're hitting whatever the current URL is. That's the basic format.
Spring's request mapping syntax looks something like /foo/bar/{id}, where {id} gets replaced at request time by whatever's in that space. (This is, incidentally, an awesome feature.) But /foo/bar/{id} is not a useful URL. I wrote a preprocessor for the Http Request Sampler that replaces {id} with some other value. You may be clever enough to do it with a Regex preprocessor; I did it with a BeanShell script that takes the current path — /foo/bar/{id} — and transforms it to /foo/bar/1234. Then it puts that value into a new variable called correctedPath.
JMeter allows you to use variables in the name field for a component. This is very useful. If you leave the sampler's name as Http Request Sampler, every single URL will get put in the "Http Request Sampler" test name, which will make your load testing reports useless. If you set the sampler's name to — using the above example — ${correctedPath}, the test name will get set to /foo/bar/1234, and each discovered URL will get its own line item in your report.
Monday, September 14, 2009
The Simplest Thing That Could Possibly Work
One of the maxims of Agile development — and especially Extreme Programming — is to write the simplest thing that can possibly work.
There are a few driving forces behind this motto. First, a core tenet of Agile development is to get testable, functional code up and running as soon as possible. If you don't spend a lot of time writing complex code, you can get it running sooner. And complex code is harder to maintain and thus more likely to have subtle bugs. Second, Agile assumes that requirements will change. This in part because requirements always change, but also because the mere act of showing a user a current piece of software will give them new ideas for the features that will really solve their problems. Users aren't good at articulating what they need until they see something "tangible" that isn't what they need. If you get a piece of code up sooner, you give your users the chance to steer the application in the direction they really want, not the direction they thought they wanted. And when they present this abrupt change in direction — as they inevitably will — you don't have to dismantle a complex system.
A close cousin of the "simplest thing" motto is "You're Not Gonna Need It." Experienced programmers tend to think in terms of frameworks they can build, good object-oriented principles, and so forth. But if you spend a lot of time building a nice, modular system with layers and abstraction, you may have wasted your time and effort if there will only ever be one module, one implementation of a given interface, or one class in a layer.
This is not to say that you should be sloppy, or churn out quick and dirty, copy-paste code. You are a professional and should act like one. Your "simplest thing" should be tested and it should be clean code. As the requirements change beyond what your "simplest thing" can do, you can then start to build layers on top of it, refactor to make it more modular, and so on.
All these arguments came back to me when I had to implement templatable text at work. I wanted text that a non-programmer could modify, but it needed to allow variable substitution.
Java's localization system allows this to some degree, except that its variables are numeric, not symbolic. So item zero is always the same, item one is always the same, etc. regardless of where they show up in the string. This makes sense for localization, but is fragile for more general templates. What if we wanted the text to use a different set of variables from one week to the next?
There are a number of general-purpose templating systems for Java: Probably the best-known is Velocity. As I started researching what it would take to implement it, however, the "simplest thing" motto came back to mind. Not that Velocity is difficult to implement, but we only needed to support five variables, and we didn't need any of the flow-of-control logic that Velocity offers.
So I wrote a very simple set of methods that allows me to do string replaces on the five symbolic variables I allowed. The (internal) user can write any text, using some or all of those variables once, twice, or whatever they want. And just in case we ever need to move to Velocity, I used the same variable naming syntax to define our variables.
The code is simple. It has noticeable restrictions (can't use undefined variables, can't define new ones), but in this case, it works exactly the way I need it to. It satisfied the business need, and truthfully, will probably never need to support more. When I needed to add an additional piece of templated text with the same variables, it took about 15 minutes including 10 minutes of testing.
There are a few driving forces behind this motto. First, a core tenet of Agile development is to get testable, functional code up and running as soon as possible. If you don't spend a lot of time writing complex code, you can get it running sooner. And complex code is harder to maintain and thus more likely to have subtle bugs. Second, Agile assumes that requirements will change. This in part because requirements always change, but also because the mere act of showing a user a current piece of software will give them new ideas for the features that will really solve their problems. Users aren't good at articulating what they need until they see something "tangible" that isn't what they need. If you get a piece of code up sooner, you give your users the chance to steer the application in the direction they really want, not the direction they thought they wanted. And when they present this abrupt change in direction — as they inevitably will — you don't have to dismantle a complex system.
A close cousin of the "simplest thing" motto is "You're Not Gonna Need It." Experienced programmers tend to think in terms of frameworks they can build, good object-oriented principles, and so forth. But if you spend a lot of time building a nice, modular system with layers and abstraction, you may have wasted your time and effort if there will only ever be one module, one implementation of a given interface, or one class in a layer.
This is not to say that you should be sloppy, or churn out quick and dirty, copy-paste code. You are a professional and should act like one. Your "simplest thing" should be tested and it should be clean code. As the requirements change beyond what your "simplest thing" can do, you can then start to build layers on top of it, refactor to make it more modular, and so on.
All these arguments came back to me when I had to implement templatable text at work. I wanted text that a non-programmer could modify, but it needed to allow variable substitution.
Java's localization system allows this to some degree, except that its variables are numeric, not symbolic. So item zero is always the same, item one is always the same, etc. regardless of where they show up in the string. This makes sense for localization, but is fragile for more general templates. What if we wanted the text to use a different set of variables from one week to the next?
There are a number of general-purpose templating systems for Java: Probably the best-known is Velocity. As I started researching what it would take to implement it, however, the "simplest thing" motto came back to mind. Not that Velocity is difficult to implement, but we only needed to support five variables, and we didn't need any of the flow-of-control logic that Velocity offers.
So I wrote a very simple set of methods that allows me to do string replaces on the five symbolic variables I allowed. The (internal) user can write any text, using some or all of those variables once, twice, or whatever they want. And just in case we ever need to move to Velocity, I used the same variable naming syntax to define our variables.
The code is simple. It has noticeable restrictions (can't use undefined variables, can't define new ones), but in this case, it works exactly the way I need it to. It satisfied the business need, and truthfully, will probably never need to support more. When I needed to add an additional piece of templated text with the same variables, it took about 15 minutes including 10 minutes of testing.
Sunday, May 24, 2009
I Am Not a Graphics Programmer
I consider myself a good programmer. I can comfortably chat about anything from high-level topics such as methodologies and architecture to the minutiae of multithreaded code and Java virtual machine idiosyncracies. I understand how to solve a wide range of scalability issues. I have a good eye for the subtle details that cause bugs. I know at least the basics of when one algorithm is better suited for a task than another.
Put me in front of a graphics library, however, and I freeze up. I've been doing some simple graphics work in my iPhone app — filling an area with a gradient, for instance — and it's been slow going. I don't have the terminology in my head for this work to come naturally. Transforms, alpha channels, drawing paths, blending modes. I might as well be reading Greek. And it doesn't help that Quartz, the graphics drawing system on the iPhone, is all based on C, a procedural language, instead of Objective C, an object-oriented language. It's been a very long time since I've worked in a low-level procedural language on a regular basis. (I will say, though, that Quartz tries its best to encapsulate the C code in macros that provide a straightforward coding experience. The graphics folks around me at Maxis think that it's pretty sweet, which makes me wonder what sort of libraries Microsoft gives to Windows programmers.)
Some of my frustration comes from simple lack of experience. I didn't used to know a lot about server-side programming, after all. As I do more graphics programming, I suppose more of it will come naturally, and I will learn more of the subtleties. But I've also never been drawn to that side of the coding fence. I like to say that I have no design sense. This isn't strictly true: I did the design work for Obsession With Food, and I've got some stunning quilt tops that I designed. But I'm always surprised when something I design comes out well.
Yet here I am, literally a one-man development team for this application. I can manage the architecture just fine — and it's pretty good — but there's no one I can hand off the graphics work to. It's just me. I speed through the object-oriented development side and slow to a crawl when I'm dealing with Quartz. In the long run, it's good for me to expand my horizons. But in the short term, the part of me that's used to quickly getting code up and running is growling at the long periods of time it takes me to implement the simplest little things in the graphics world.
Back to figuring out why my transparent blue is sometimes the only thing in the rectangle and is sometimes on top of the gradient that's supposed to be gone at that point. And why doesn't my grey rectangle show up at all? All this for a tiny-but-necessary bit of visual feedback on an already functional feature.
Put me in front of a graphics library, however, and I freeze up. I've been doing some simple graphics work in my iPhone app — filling an area with a gradient, for instance — and it's been slow going. I don't have the terminology in my head for this work to come naturally. Transforms, alpha channels, drawing paths, blending modes. I might as well be reading Greek. And it doesn't help that Quartz, the graphics drawing system on the iPhone, is all based on C, a procedural language, instead of Objective C, an object-oriented language. It's been a very long time since I've worked in a low-level procedural language on a regular basis. (I will say, though, that Quartz tries its best to encapsulate the C code in macros that provide a straightforward coding experience. The graphics folks around me at Maxis think that it's pretty sweet, which makes me wonder what sort of libraries Microsoft gives to Windows programmers.)
Some of my frustration comes from simple lack of experience. I didn't used to know a lot about server-side programming, after all. As I do more graphics programming, I suppose more of it will come naturally, and I will learn more of the subtleties. But I've also never been drawn to that side of the coding fence. I like to say that I have no design sense. This isn't strictly true: I did the design work for Obsession With Food, and I've got some stunning quilt tops that I designed. But I'm always surprised when something I design comes out well.
Yet here I am, literally a one-man development team for this application. I can manage the architecture just fine — and it's pretty good — but there's no one I can hand off the graphics work to. It's just me. I speed through the object-oriented development side and slow to a crawl when I'm dealing with Quartz. In the long run, it's good for me to expand my horizons. But in the short term, the part of me that's used to quickly getting code up and running is growling at the long periods of time it takes me to implement the simplest little things in the graphics world.
Back to figuring out why my transparent blue is sometimes the only thing in the rectangle and is sometimes on top of the gradient that's supposed to be gone at that point. And why doesn't my grey rectangle show up at all? All this for a tiny-but-necessary bit of visual feedback on an already functional feature.
Saturday, May 9, 2009
Refactoring Pays Down Technical Debt
One of my first posts on this site mentioned technical debt, a brilliant metaphor for the way that hurrying imperfect code out the door provides short-term gains but long-term costs for new features. Enough technical debt, and your feature development slows to a crawl. One way to pay down technical debt is to refactor code: Refactoring's main role is to adapt a code base to requirements that weren't on the table when the programmers started typing.
I've talked about continual refactoring before, which I do as a general rule in any code I find myself in, but that is usually directionless cleanup. A deliberate refactoring phase is even better. (As long as you don't fall into the "rewrite everything" trap that Netscape found itself in, which caused such a long delay in version 6 that the company essentially handed the entire browser market to Internet Explorer.)
In my iPhone app, I have three "detail" views that, when I first wrote them, had only surface-level similarities.* They had different DAL requirements, used different objects, and so forth. But as I added new functionality to each, such as my ScheduleButton widget and my date picker widget, they began to look more alike than not. I entered a bug for myself to refactor all the common code into one base class.
A week later, I did the work. There was no design requirement driving this: I just knew the code would end up in a better place at the end. I abstracted common functionality into superclass utility methods and used the template method design pattern to put a flow into the base class that used abstract methods ("abstract," because Objective C doesn't support this) to fill in view-specific details.
A week after that, I realized I needed to change the method signature for my date picker delegate methods to support a new feature. That meant going into all the delegates and updating the code. A simple search/replace wouldn't do what I needed.
Before my refactor, that fix would have affected eight methods in four classes. After my refactor, it affected four methods in two classes. I halved the development effort it took to support this new feature. But I also halved the testing effort. Since all my detail views descend from a base class that handles all the date chooser work, I only had to test one of them — plus one unrelated view — to ensure that all three were working. If I need to add a feature across all the detail views, it will take me one-third the time it would have before.
* As always, I should point out that my iPhone app just isn't that big. I'm unlikely to ever have a vast amount of technical debt in it. Still, it illustrates the point in a microcosm. Larger systems have proportionally more complexity and thus more opportunity for technical debt.
I've talked about continual refactoring before, which I do as a general rule in any code I find myself in, but that is usually directionless cleanup. A deliberate refactoring phase is even better. (As long as you don't fall into the "rewrite everything" trap that Netscape found itself in, which caused such a long delay in version 6 that the company essentially handed the entire browser market to Internet Explorer.)
In my iPhone app, I have three "detail" views that, when I first wrote them, had only surface-level similarities.* They had different DAL requirements, used different objects, and so forth. But as I added new functionality to each, such as my ScheduleButton widget and my date picker widget, they began to look more alike than not. I entered a bug for myself to refactor all the common code into one base class.
A week later, I did the work. There was no design requirement driving this: I just knew the code would end up in a better place at the end. I abstracted common functionality into superclass utility methods and used the template method design pattern to put a flow into the base class that used abstract methods ("abstract," because Objective C doesn't support this) to fill in view-specific details.
A week after that, I realized I needed to change the method signature for my date picker delegate methods to support a new feature. That meant going into all the delegates and updating the code. A simple search/replace wouldn't do what I needed.
Before my refactor, that fix would have affected eight methods in four classes. After my refactor, it affected four methods in two classes. I halved the development effort it took to support this new feature. But I also halved the testing effort. Since all my detail views descend from a base class that handles all the date chooser work, I only had to test one of them — plus one unrelated view — to ensure that all three were working. If I need to add a feature across all the detail views, it will take me one-third the time it would have before.
* As always, I should point out that my iPhone app just isn't that big. I'm unlikely to ever have a vast amount of technical debt in it. Still, it illustrates the point in a microcosm. Larger systems have proportionally more complexity and thus more opportunity for technical debt.
Tuesday, April 28, 2009
Meaty Learning Projects
My job description implicitly includes understanding current technologies, philosophies, and methodologies. Usually, this is just a surface-level knowledge about what's going on in the industry, but occasionally I decide to dig deeper.
For instance, right now I'm learning Scala because I think it will solve some thorny performance issues at work. But it's a Catch-22: I don't want to introduce a whole new language without knowing if it will really benefit us, but I can't know if it will benefit us until I try it with a meaty project. And it's even worse than normal in this case: The feature I really want to explore — actors — requires a decent chunk of knowledge before I can take advantage of it.
Over the years, I've developed a solution to this problem. I have a substantial, but not overly complicated, program that I write whenever I come to a new platform. I write a program that assists me in finding solutions to the kind of thorny word puzzles you get from the National Puzzlers League. It requires me to open a file, read in words, and see if they match the pattern the user is searching for. No network connection. No database work. No scalability issues.
It's a problem space I understand, so I can dive in without much thought. I get a deeper knowledge of the language than I'd get from working through a tutorial's abstract problems. It forces me to probe and dig through the libraries to accomplish what I need. I don't get a thorough understanding of the language, but I get pretty far.
Overall, it's a plan that's worked well for me.
For instance, right now I'm learning Scala because I think it will solve some thorny performance issues at work. But it's a Catch-22: I don't want to introduce a whole new language without knowing if it will really benefit us, but I can't know if it will benefit us until I try it with a meaty project. And it's even worse than normal in this case: The feature I really want to explore — actors — requires a decent chunk of knowledge before I can take advantage of it.
Over the years, I've developed a solution to this problem. I have a substantial, but not overly complicated, program that I write whenever I come to a new platform. I write a program that assists me in finding solutions to the kind of thorny word puzzles you get from the National Puzzlers League. It requires me to open a file, read in words, and see if they match the pattern the user is searching for. No network connection. No database work. No scalability issues.
It's a problem space I understand, so I can dive in without much thought. I get a deeper knowledge of the language than I'd get from working through a tutorial's abstract problems. It forces me to probe and dig through the libraries to accomplish what I need. I don't get a thorough understanding of the language, but I get pretty far.
Overall, it's a plan that's worked well for me.
Friday, April 24, 2009
The Root Of All Evil
In my most recent round of interviews, I chatted with the founder of U1 Technologies. He and I had a lot in common philosophically and, with a slight shift in timing, I might very well have been working for him instead of Maxis.
One of the things that caught my attention was his phone screen questions. I've been on both ends of these things, and they usually consist of simple questions that ensure that the candidate isn't obviously inflating his or her resume. Things like: "Explain how an Iterator works." or "When would you use a List versus a Map?" (Yes, I've actually been asked these.)
Nathaniel's first question reflected his company's decision to only hire senior engineers: "When should you optimize your code?" Fifteen years of software engineering provided a quick answer: "When you know there's a performance problem with it."
"Premature optimization is the root of all evil," is a quote that's been attributed to both Donald Knuth and C.A.R. Hoare. (Most people cite Knuth, who cites Hoare, who denies saying it, according to Wikipedia.)
This may sound at odds with one of my other favorite programming axioms: "The earlier you find the bug, the cheaper it is to fix." But optimizing code usually goes hand in hand with making it less maintainable and reusable. Specializing algorithms, caching values, adding different control structures, and more all add complexity. If you don't have performance numbers to back up that work, you're adding the complexity and (probably) not getting any benefit from it.
You may have inefficient code that rarely gets called. You may think something's inefficient when in fact the compiler or optimizer or processor does just fine with it. Or your code could actually be inefficient and it might just not matter enough to affect the user.
I've seen this in practice for years. And yet the temptation to optimize is always there, coaxing even the best programmers into a fen of tangles and mazes and confusion.
While I was working on the "real time time update" feature of my reusable date picker pop-up, I set it up so that the delegate would be called for every UIControlEventValueChanged message from the UIDatePicker.
I envisioned the date and time changing as the wheels in the date picker spun to their new locations, so I decided to be smart about calling the delegate method.
I could have written this in my event-handling method:
But I assumed that
I changed my event-handling logic accordingly:
Then I fired up my app.
And discovered something I didn't know. That UIControlEventValueChanged message only gets sent once the date picker wheels have finished spinning. Rather than sending a message for every number that cruises by, it only sends, essentially, one message per gesture. My optimization didn't buy me much, since the code can't possibly interfere with the spinning wheel animation.
So what did my "clever" optimization get me?
I've added more complexity (the extra layer of logic) and more lines of code (the
Premature optimization is the root of all evil.
* Yes, of course, this isn't much extra complexity or code. But add up enough items like this, and you get a significant amount of technical debt
One of the things that caught my attention was his phone screen questions. I've been on both ends of these things, and they usually consist of simple questions that ensure that the candidate isn't obviously inflating his or her resume. Things like: "Explain how an Iterator works." or "When would you use a List versus a Map?" (Yes, I've actually been asked these.)
Nathaniel's first question reflected his company's decision to only hire senior engineers: "When should you optimize your code?" Fifteen years of software engineering provided a quick answer: "When you know there's a performance problem with it."
"Premature optimization is the root of all evil," is a quote that's been attributed to both Donald Knuth and C.A.R. Hoare. (Most people cite Knuth, who cites Hoare, who denies saying it, according to Wikipedia.)
This may sound at odds with one of my other favorite programming axioms: "The earlier you find the bug, the cheaper it is to fix." But optimizing code usually goes hand in hand with making it less maintainable and reusable. Specializing algorithms, caching values, adding different control structures, and more all add complexity. If you don't have performance numbers to back up that work, you're adding the complexity and (probably) not getting any benefit from it.
You may have inefficient code that rarely gets called. You may think something's inefficient when in fact the compiler or optimizer or processor does just fine with it. Or your code could actually be inefficient and it might just not matter enough to affect the user.
I've seen this in practice for years. And yet the temptation to optimize is always there, coaxing even the best programmers into a fen of tangles and mazes and confusion.
While I was working on the "real time time update" feature of my reusable date picker pop-up, I set it up so that the delegate would be called for every UIControlEventValueChanged message from the UIDatePicker.
I envisioned the date and time changing as the wheels in the date picker spun to their new locations, so I decided to be smart about calling the delegate method.
I could have written this in my event-handling method:
if ([delegate respondsToSelector:@selector(dateChangedWithoutSaving:)]) {
[delegate dateChangedWithoutSaving: date];
}
But I assumed that
respondsToSelector would be a relatively expensive call. I also assumed that the runtime binding to dateChangedWithoutSaving: method would be costly. And I didn't want that cost slowing down the user's interaction with the date picker wheels. Instead, I made a boolean in the controller that tracked whether or not the delegate actually supported the method and set it at the same time I stored the reference to the delegate:
- (void) setDelegate: (id) delegate {
...
delegateRespondsToContinuousDateChanged = [delegate respondsToSelector:@selector(dateChangedWithoutSaving:)];
}
I changed my event-handling logic accordingly:
if (delegateRespondsToContinuousDateChanged) {
[delegate dateChangedWithoutSaving:[myDatePicker date]];
}
Then I fired up my app.
And discovered something I didn't know. That UIControlEventValueChanged message only gets sent once the date picker wheels have finished spinning. Rather than sending a message for every number that cruises by, it only sends, essentially, one message per gesture. My optimization didn't buy me much, since the code can't possibly interfere with the spinning wheel animation.
So what did my "clever" optimization get me?
- My own implementation of
setDelegate(rather than the one created by Objective C'ssynthesizekeyword) - An extra layer of logic in my event-handling code.
I've added more complexity (the extra layer of logic) and more lines of code (the
setDelegate method) with the concurrent possibility of bugs.* Performance gain: virtually none.Premature optimization is the root of all evil.
* Yes, of course, this isn't much extra complexity or code. But add up enough items like this, and you get a significant amount of technical debt
Sunday, April 19, 2009
Constant Refactoring: A Practical Benefit
As I program, I continually refactor the code I'm in. It's habit at this point. I rarely get the opportunity to do a massive change, but even the basic work I do — removing duplicated code, generalizing logic, dividing up large functions, and so on — reaps unexpected benefits.
Refactoring partly makes code more maintainable, but its main purpose is to make the code flexible enough to respond to new design requirements. I try to code on the assumption that change will come, but if I could predict the exact direction of every future change and thus craft a perfect design, I'd be writing this from my French Chateau.
Here's a simple example I recently found in my iPhone app.
One of the tabs lets you move between dates, one at a time, to see your schedule for that particular date. In my initial design, I didn't let you change dates because that view exists to show you what's going on today. But I quickly realized that I wanted to at least see what tomorrow would bring.
When I made that change, I had two buttons that would switch to either the next date or the previous date. They each called separate methods — nextDay and previousDay — that were fairly similar to each other. I was working through the basic functionality, so I didn't register the commonality. I think this is common enough: You're so focused on one thing that you don't notice something else.
On a later pass through the code, fixing something else, I realized how similar the methods were, and I wrote a new method that would just adjust the date based on some time interval. I gutted nextDay and previousDay and replaced them with calls to the new method that passed either 86400 or -86400. (The number of seconds in a day.)
After that, one of my testers suggested that I let the user swipe across the date to move to the next or previous day, depending on the direction. Once I figured out the gesture system for the iPhone, I just made each swipe call either of the two appropriate methods, which in turn called into that common method.
But then I realized that I needed to add animation to show the user what was happening. I put the animation logic into the generalized method and made it smart enough to know that a negative number meant animate one way, while a positive number meant animate the other way (animate from the left in the first case, because you're moving to the previous day, and animate from the right in the second). That handled the animation for the two cases.
A couple of my testers asked for the ability to jump to an arbitrary day. I decided to look into implementing that this morning, and once I opened the code, I realized it would be a snap. I bring up a UIDatePicker, the "choose from a list" UI in the iPhone, and then just pass the time interval between the user-selected date and the current date to that one common method. The logic is the same, and the code animates appropriately: If you jump to some arbitrary, far-future date, the screen wipes from right to left, as you'd expect. Aside from bringing up the date picker, I wrote exactly one line of code to enable the feature.
For this simple example, I probably would have found my way to the same end result: an all-purpose method for jumping to any given date. But the fact that I had refactored earlier — just because — made the solution obvious and quick.
When I started this feature, you couldn't switch dates at all. Piece by piece, I built the infrastructure so that now you can jump to any date with gestures and animations. That wasn't the original design, but constant refactoring allowed me to eventually support this radically new direction with a minimum of "from scratch" code.
Refactoring partly makes code more maintainable, but its main purpose is to make the code flexible enough to respond to new design requirements. I try to code on the assumption that change will come, but if I could predict the exact direction of every future change and thus craft a perfect design, I'd be writing this from my French Chateau.
Here's a simple example I recently found in my iPhone app.
One of the tabs lets you move between dates, one at a time, to see your schedule for that particular date. In my initial design, I didn't let you change dates because that view exists to show you what's going on today. But I quickly realized that I wanted to at least see what tomorrow would bring.
When I made that change, I had two buttons that would switch to either the next date or the previous date. They each called separate methods — nextDay and previousDay — that were fairly similar to each other. I was working through the basic functionality, so I didn't register the commonality. I think this is common enough: You're so focused on one thing that you don't notice something else.
On a later pass through the code, fixing something else, I realized how similar the methods were, and I wrote a new method that would just adjust the date based on some time interval. I gutted nextDay and previousDay and replaced them with calls to the new method that passed either 86400 or -86400. (The number of seconds in a day.)
After that, one of my testers suggested that I let the user swipe across the date to move to the next or previous day, depending on the direction. Once I figured out the gesture system for the iPhone, I just made each swipe call either of the two appropriate methods, which in turn called into that common method.
But then I realized that I needed to add animation to show the user what was happening. I put the animation logic into the generalized method and made it smart enough to know that a negative number meant animate one way, while a positive number meant animate the other way (animate from the left in the first case, because you're moving to the previous day, and animate from the right in the second). That handled the animation for the two cases.
A couple of my testers asked for the ability to jump to an arbitrary day. I decided to look into implementing that this morning, and once I opened the code, I realized it would be a snap. I bring up a UIDatePicker, the "choose from a list" UI in the iPhone, and then just pass the time interval between the user-selected date and the current date to that one common method. The logic is the same, and the code animates appropriately: If you jump to some arbitrary, far-future date, the screen wipes from right to left, as you'd expect. Aside from bringing up the date picker, I wrote exactly one line of code to enable the feature.
For this simple example, I probably would have found my way to the same end result: an all-purpose method for jumping to any given date. But the fact that I had refactored earlier — just because — made the solution obvious and quick.
When I started this feature, you couldn't switch dates at all. Piece by piece, I built the infrastructure so that now you can jump to any date with gestures and animations. That wasn't the original design, but constant refactoring allowed me to eventually support this radically new direction with a minimum of "from scratch" code.
Technical Debt
Some time in the last year, I learned the term Technical Debt. It's a brilliant metaphor for the more hurried, less-thought-out code that crops up in any project with a ship date.
The idea is simple: Just as you actively take on financial debt to give yourself new opportunities, you take on technical debt to ship a product out the door and make money from it. Financial debt is simple money, but technical debt is more complex. Any decision you make to ship the product at the expense of a clean code base qualifies. Every test you don't write is technical debt. Every piece of duplicated code that hasn't been refactored into common functionality is technical debt. Every todo comment is technical debt. Every bug you ship with is technical debt.
But too much technical debt, and you never pay down the principal, in this case the ability to quickly add new features or fix old bugs. That duplicated code? If you never refactor it, every bug fix in that code has a cost proportional to the number of places it's used. And that's if you actually remember where all those pieces are. That todo comment you never go back to? Some day you'll need to enhance the code around it, and that todo will add to the cost of the work. Add enough of these little debts, and the cost for any given feature multiplies.
Unfortunately, while it's easy to measure financial debt, it's tough to measure technical debt, which makes it difficult to use when arguing for time to fix things up in the code base. If you never need to fix a bug in that duplicated code, for instance, you'll never experience the extra cost of fixing it in every location. Of course, the idea that a chunk of code will sit forever untouched once it's written is ludicrous, but you can't sell the idea that something might be a problem someday outside of development teams.
There's also no good way to flag technical debt. Perhaps as programmers we need a more formal way — beyond todo comments — to note down that we made a deliberate choice to take on technical debt. But then we have to advertise to the larger world that we deliberately did something not in the best possible way. I know that won't sell.
The idea is simple: Just as you actively take on financial debt to give yourself new opportunities, you take on technical debt to ship a product out the door and make money from it. Financial debt is simple money, but technical debt is more complex. Any decision you make to ship the product at the expense of a clean code base qualifies. Every test you don't write is technical debt. Every piece of duplicated code that hasn't been refactored into common functionality is technical debt. Every todo comment is technical debt. Every bug you ship with is technical debt.
But too much technical debt, and you never pay down the principal, in this case the ability to quickly add new features or fix old bugs. That duplicated code? If you never refactor it, every bug fix in that code has a cost proportional to the number of places it's used. And that's if you actually remember where all those pieces are. That todo comment you never go back to? Some day you'll need to enhance the code around it, and that todo will add to the cost of the work. Add enough of these little debts, and the cost for any given feature multiplies.
Unfortunately, while it's easy to measure financial debt, it's tough to measure technical debt, which makes it difficult to use when arguing for time to fix things up in the code base. If you never need to fix a bug in that duplicated code, for instance, you'll never experience the extra cost of fixing it in every location. Of course, the idea that a chunk of code will sit forever untouched once it's written is ludicrous, but you can't sell the idea that something might be a problem someday outside of development teams.
There's also no good way to flag technical debt. Perhaps as programmers we need a more formal way — beyond todo comments — to note down that we made a deliberate choice to take on technical debt. But then we have to advertise to the larger world that we deliberately did something not in the best possible way. I know that won't sell.
Subscribe to:
Posts (Atom)