Often, when I'm rolling out a new, substantial feature, I add a config file property that marks it as enabled or disabled. This lets the team shut it down quickly if it's causing problems. It's a good practice, overall.
Keeping the status in a config file is easy to maintain, but it's problematic. Being able to "switch off" a feature in a config file really means, for most Java deployments, rebuilding and redeploying the war file, which can take a while if you're including a bunch of big libraries.
Recently, inspired by two pieces I read about how disabling features helps scale a site and mitigate risk, I came up with an enhancement to my older idea, and it's one of those brain-dead-obvious changes that I wish I had done before: A feature's enabled or disabled status now lives in the database.
I have a simple table (with Hibernate object and service) that stores a feature name, its current enabled status, and the date on which that status changed. Any other service or controller in the system can query that service to see if a given feature is enabled. (Since I run in the Spring framework, access to the service is a simple matter.)
What that means practically is that I can disable an entire subsystem across all instances of my application simply by updating a database row. (In the live system, these will probably be cached and refreshed every few minutes so that incoming requests aren't slowed down.) It also means I can get the status of the systems across an entire bank of servers with a simple query.
I put it in as groundwork for one feature — capturing real incoming traffic to a site to then play back as loadtesting scripts — but quickly back-ported a few other systems that have single points of entry. Our profiling system, for instance, can now be turned off application-wide simply by setting the enabled status of "Profiling" to false.
Of course, once you have subsystems that have a dynamic mechanism for checking their enabled/disabled status, it's a small step to enabling/disabling on a per-user basis. Which means we can gradually roll out new features, checking the load on the system at each pass and fixing bugs.
It's not an earth-shattering idea, but I was immediately entranced by the power it gives to my system.
Tuesday, November 23, 2010
Saturday, October 2, 2010
Integration Tests
I've been a fan of unit tests for a couple of years now. Once I buckled down and wrote some, I realized how powerful they could be. They give you more stability out of the gate and they provide good regression tests. They allow me, the developer, to keep moving forward in tasks and minimize the time I spend going backward to debug or test older functions.
But I've always held the stance that unit tests shouldn't test things such as interactions with Hibernate or Spring. Those are well-tested frameworks with strong community support. Writing unit tests that touch those layers always struck me as a waste of time.
That said, most server-side Java code ends up moving into those layers. Virtually everything I write ends up talking to a database. And the server code itself is a loosely knit tapestry of services that chat with each other. That code can certainly have bugs — incorrect queries, edge conditions, bad assumptions, whatever. So how do I get tests against it?
Integration tests. Like unit tests, integration tests demonstrate that function A returns result X given inputs Y and Z. The difference is that in an integration test, you are testing the interaction between systems versus the simple in and out of a self-contained function.
I finally decided that I wasn't getting the test coverage I wanted with just unit tests. I was finding subtle bugs tucked away in database calls and service-to-service communication. Mock objects — code that presents the expected interface to a layer without providing the full functionality — only get you part of the way. If your mock object is keeping information in a local map instead of hitting the database, you're not testing that the query to the database returns the right thing.
Once I decided to incorporate integration tests, however, I went down a bit of rabbit hole. Your services need to talk to each other, so you need to wire them up to each other. But in the real, running code, I use Spring to manage all that. Fortunately, I can use Spring in the test environment, too. And Spring allows me to have a supplemental bean configuration file that overrides the production-code config file. So, for instance, I can have a bean named "telemetryService" that overrides the bean of the same name in the main config file. The test version doesn't actually send telemetry information. It effectively becomes a mock object. (Though in that particular case, it's done with a simple boolean.) I have an S3 service layer in the test config file that points to a test S3 bucket instead of our development one. Any beans that aren't overridden are pulled from the main file.
My integration tests do have to call a method to set up that configuration, however. Since the Spring config doesn't change at runtime, I have a static utility method that checks to see if the configuration is set up and, if it isn't, sets it up. This violates the principle that a test should not rely on existing state from another test, but that configuration load is time consuming.
But services don't just talk to each other. They also talk to the database. You don't want them to point to a database you actually use, so you need a test database. And that test database needs to a) be kept up to date, b) have some test data, and c) not let data linger (to ensure that no test ends up passing because of the coincidence of some other test running before it).
For the last two requirements, I incorporated DBUnit. DBUnit can do unit testing against the database, but I don't really need its form of that. Because I'm using Spring to set up my app in test mode, all my services can continue to use Hibernate for database work. (They can also use my JRuby-based query files.) But DBUnit offers two key services: It can take a data file of test data and load the database with it, and it can wipe any desired tables clean between test runs. When a test suite starts, it calls a method that ensures the Spring configuration is read (if it hasn't already been) and wipes/reloads the test database. When a test suite starts, I know that there is predictable data in the database and that that's the only data in the database.
What about between tests, though? I could do a wipe and reload around every test, but that's time-consuming. Instead, I have a setup method that configures a transaction and a teardown method that rolls back that transaction. All of this means that developers have to remember to do all this when writing an integration test, which I dislike, but I hope to integrate some tools that will automate that.
My test database also needs to be current with the latest schema. For that, I incorporated Autopatch, which will automatically get your database up to date when a code base runs against it.
So far, so good. But integration tests take a while to run, and I worry that developers might, faced with progressively longer test runs, disable tests on local builds. To keep our tests spry, I separate integration tests so that they're only run if you tell Buildr you're in the "longtests" environment. (That environment also specifies the test database properties.) The build system, which runs within minutes of any check-in, always runs them, however, so even a developer who gets hasty has an automatic monitoring.
All of this took three solid days to get working properly, but now that's it running, I got exactly what I wanted: the ability to have much deeper test coverage. The other day, I wrote a test that called service methods that altered the database and moved files around on S3. At each step in the chain, I could check the state of each component and verify that it had worked succesfully. And I knew that it was executing real queries and running real code. No mock objects.
The ability to have deeper test coverage means I've set myself a goal of writing those tests. Every feature I work on now has tests against it, even if they require database work or multiple services. And when I find a bug in my existing code, I force myself to write a test to reproduce it and then work on the code until the test passes. That way I get free regression testing whenever a check-in happens.
I have noticed one negative aspect, however. Unit tests, by their very nature, force you to write smaller and smaller methods that are easier to test. With no limit on the amount of activity you can do in an integration test, however, I find that my methods aren't channeled into smaller pieces. I try and do that because I know that ultimately produces more maintainable code, but my integration test system doesn't give an almost automatic push the way unit tests do.
Big methods or not, I still get to run a complete suite of extensive, realistic tests with a single command. That's pretty powerful.
But I've always held the stance that unit tests shouldn't test things such as interactions with Hibernate or Spring. Those are well-tested frameworks with strong community support. Writing unit tests that touch those layers always struck me as a waste of time.
That said, most server-side Java code ends up moving into those layers. Virtually everything I write ends up talking to a database. And the server code itself is a loosely knit tapestry of services that chat with each other. That code can certainly have bugs — incorrect queries, edge conditions, bad assumptions, whatever. So how do I get tests against it?
Integration tests. Like unit tests, integration tests demonstrate that function A returns result X given inputs Y and Z. The difference is that in an integration test, you are testing the interaction between systems versus the simple in and out of a self-contained function.
I finally decided that I wasn't getting the test coverage I wanted with just unit tests. I was finding subtle bugs tucked away in database calls and service-to-service communication. Mock objects — code that presents the expected interface to a layer without providing the full functionality — only get you part of the way. If your mock object is keeping information in a local map instead of hitting the database, you're not testing that the query to the database returns the right thing.
Once I decided to incorporate integration tests, however, I went down a bit of rabbit hole. Your services need to talk to each other, so you need to wire them up to each other. But in the real, running code, I use Spring to manage all that. Fortunately, I can use Spring in the test environment, too. And Spring allows me to have a supplemental bean configuration file that overrides the production-code config file. So, for instance, I can have a bean named "telemetryService" that overrides the bean of the same name in the main config file. The test version doesn't actually send telemetry information. It effectively becomes a mock object. (Though in that particular case, it's done with a simple boolean.) I have an S3 service layer in the test config file that points to a test S3 bucket instead of our development one. Any beans that aren't overridden are pulled from the main file.
My integration tests do have to call a method to set up that configuration, however. Since the Spring config doesn't change at runtime, I have a static utility method that checks to see if the configuration is set up and, if it isn't, sets it up. This violates the principle that a test should not rely on existing state from another test, but that configuration load is time consuming.
But services don't just talk to each other. They also talk to the database. You don't want them to point to a database you actually use, so you need a test database. And that test database needs to a) be kept up to date, b) have some test data, and c) not let data linger (to ensure that no test ends up passing because of the coincidence of some other test running before it).
For the last two requirements, I incorporated DBUnit. DBUnit can do unit testing against the database, but I don't really need its form of that. Because I'm using Spring to set up my app in test mode, all my services can continue to use Hibernate for database work. (They can also use my JRuby-based query files.) But DBUnit offers two key services: It can take a data file of test data and load the database with it, and it can wipe any desired tables clean between test runs. When a test suite starts, it calls a method that ensures the Spring configuration is read (if it hasn't already been) and wipes/reloads the test database. When a test suite starts, I know that there is predictable data in the database and that that's the only data in the database.
What about between tests, though? I could do a wipe and reload around every test, but that's time-consuming. Instead, I have a setup method that configures a transaction and a teardown method that rolls back that transaction. All of this means that developers have to remember to do all this when writing an integration test, which I dislike, but I hope to integrate some tools that will automate that.
My test database also needs to be current with the latest schema. For that, I incorporated Autopatch, which will automatically get your database up to date when a code base runs against it.
So far, so good. But integration tests take a while to run, and I worry that developers might, faced with progressively longer test runs, disable tests on local builds. To keep our tests spry, I separate integration tests so that they're only run if you tell Buildr you're in the "longtests" environment. (That environment also specifies the test database properties.) The build system, which runs within minutes of any check-in, always runs them, however, so even a developer who gets hasty has an automatic monitoring.
All of this took three solid days to get working properly, but now that's it running, I got exactly what I wanted: the ability to have much deeper test coverage. The other day, I wrote a test that called service methods that altered the database and moved files around on S3. At each step in the chain, I could check the state of each component and verify that it had worked succesfully. And I knew that it was executing real queries and running real code. No mock objects.
The ability to have deeper test coverage means I've set myself a goal of writing those tests. Every feature I work on now has tests against it, even if they require database work or multiple services. And when I find a bug in my existing code, I force myself to write a test to reproduce it and then work on the code until the test passes. That way I get free regression testing whenever a check-in happens.
I have noticed one negative aspect, however. Unit tests, by their very nature, force you to write smaller and smaller methods that are easier to test. With no limit on the amount of activity you can do in an integration test, however, I find that my methods aren't channeled into smaller pieces. I try and do that because I know that ultimately produces more maintainable code, but my integration test system doesn't give an almost automatic push the way unit tests do.
Big methods or not, I still get to run a complete suite of extensive, realistic tests with a single command. That's pretty powerful.
Tuesday, September 28, 2010
Config Files As Programs
While working on a few recent tasks, I've hit upon a small but powerful technique: My configuration files are actually programs in their own right.
Expression languages of some form are de rigueur in modern frameworks. Especially in server code, where you need to deploy the same code base to different servers and have them point to different databases, mail servers, or whatever. In Spring, for instance, you can write something like
But my technique is vastly more powerful. I'm sure others have thought of it, too, but it sort of snuck up on me.
The first time I used this trick was when I put my work's SQL into Ruby scripts. I did that for readability's sake, but as I added more queries, I began to do the things one does does with programs. I refactored two similar queries into a method that took a differentiating argument and returned the query. I declared constants to hold common values. I started to ponder a Ruby module that I could import for various utility functions. The end result from the Java code's view was a set of query definitions. But within the query definition itself, I had normal programming tools at my disposal.
On a personal project (which I hope to write about soon), I decided to use YAML for the config files. I've gotten used to the format for config files, thanks to my work with AppEngine, and Ruby can easily work with it. My original idea for the config file was just fixed dates and times when I wanted my script to wake up and record an Internet radio stream. But then I realized that I wanted to record some streams on every weekday or on the first Sunday of the month.
So I thought, what if, before I feed the file to the YAML parser, I first run it through ERB, Ruby's powerful templating system? As long as the YAML parser sees something that looks like YAML, it doesn't matter how it gets there, right?
Right. I defined a method in my config file — in my config file! — that would return the next weekday after the given date. Then, when defining the value for a field, I called that method to calculate the value. My "YAML" file looks something like:
Expression languages of some form are de rigueur in modern frameworks. Especially in server code, where you need to deploy the same code base to different servers and have them point to different databases, mail servers, or whatever. In Spring, for instance, you can write something like
${project.database.url} and it will assume you mean to use not the literal string but the value of the project.database.url property, which is probably different in your dev environment and your production environment.But my technique is vastly more powerful. I'm sure others have thought of it, too, but it sort of snuck up on me.
The first time I used this trick was when I put my work's SQL into Ruby scripts. I did that for readability's sake, but as I added more queries, I began to do the things one does does with programs. I refactored two similar queries into a method that took a differentiating argument and returned the query. I declared constants to hold common values. I started to ponder a Ruby module that I could import for various utility functions. The end result from the Java code's view was a set of query definitions. But within the query definition itself, I had normal programming tools at my disposal.
On a personal project (which I hope to write about soon), I decided to use YAML for the config files. I've gotten used to the format for config files, thanks to my work with AppEngine, and Ruby can easily work with it. My original idea for the config file was just fixed dates and times when I wanted my script to wake up and record an Internet radio stream. But then I realized that I wanted to record some streams on every weekday or on the first Sunday of the month.
So I thought, what if, before I feed the file to the YAML parser, I first run it through ERB, Ruby's powerful templating system? As long as the YAML parser sees something that looks like YAML, it doesn't matter how it gets there, right?
Right. I defined a method in my config file — in my config file! — that would return the next weekday after the given date. Then, when defining the value for a field, I called that method to calculate the value. My "YAML" file looks something like:
start_date: <%next_weekday.strftime('%Y-%m-%d')%> 18:00. Which won't give you anything useful from the YAML parser. But run it through the ERB parser, and the YAML parser sees start_date: 2010-09-28 18:00, which is perfectly valid YAML syntax and is what you intend for the value to be.
Friday, September 3, 2010
Words With Friends Bingos, Redux
In my last post, I described writing a program to find the five-letter combinations most likely to yield a bingo in Words With Friends.
Then I realized there was a subtle bug.
Consider the word abreast. When you look at how Ruby's
Here is the new word list with the corrected code. TEARS and TIERS are still your best bet, but the list has shifted a bit and the numbers are different.
And here's the corrected script.
Then I realized there was a subtle bug.
Consider the word abreast. When you look at how Ruby's
combination method splits it up, you'll see two entries equivalent to STARE, because there are two a's. This is correct from a programming view, but from a player's perspective, it doesn't matter. Having STARE is sufficient to get to abreast. You don't view the a's as different.Here is the new word list with the corrected code. TEARS and TIERS are still your best bet, but the list has shifted a bit and the numbers are different.
aerst 1295
eirst 1237
aeirs 988
aeist 984
einst 983
einrs 972
eorst 949
aelrs 940
eerst 920
aeins 854
aeirt 816
aenrs 805
aenst 799
eeirs 792
aelst 788
eilst 782
aeint 765
aeils 765
ainst 751
eilrs 748
And here's the corrected script.
# find the 5-letter combos most represented in 7- or 8-letter words
wordlist = ARGV[0]
combos_to_count = Hash.new(1)
File.open(wordlist,"r") do |file|
line = file.gets
while(line)
line = file.gets
if line then
line.chomp!
next if line.length != 7 && line.length != 8
chars = line.split(//)
unique_combos = []
chars.combination(5) do |combo|
combo.sort! {|a,b| a <=> b}
unique_combos << combo.join
end
unique_combos.uniq!
unique_combos.each {|combo| combos_to_count[combo] = combos_to_count[combo] + 1}
end
end
end
sorted_combos = combos_to_count.sort {|a,b| b[1] <=> a[1]}
(0...20).each {|num| puts sorted_combos[num][0] + " " + sorted_combos[num][1].to_s}
Optimize For Bingos In Words With Friends
See the updated version here.
Somewhere in my library of books is one called Everything Scrabble, a guide to improving your Scrabble game. I read through it a few years back and have probably forgotten half its contents, but I do remember the author's comment that highly ranked Scrabble players often swap tiles in an effort to increase their chances of hitting a bingo (a word that uses all seven letters on your tray).
I've been playing Words With Friends (screen name: linechef) and have been thinking about that Scrabble strategy. Bingoing is less of an advantage in WWF. They're only worth 35 points instead of 50, and they often open up large swaths of bonus tiles for your opponent to gobble up. But they are usually worth it if you can hit a bonus tile yourself.
Everything Scrabble provides the scoop on which five tiles you should keep in your tray to maximize bingoing, but my mind let that knowledge go into the abyss. So I decided to write a program to rediscover it.
You can imagine how the program works: Go through every seven- and eight-letter word in the Words With Friends dictionary, the ENABLE wordlist, and find every unique combination of five letters in it. Keep track of how many copies of that combination you've seen throughout the dictionary, and then sort.
Let's start with the answers first. Here are the top 20 combinations, along with the number of times they occurred (the ENABLE list has 172,823 words):
And here's the Ruby code. Ruby 1.8.7 added a
Once you have the code, you can start asking other questions. Is the list different for words with exactly seven letters? A little bit:
Note that there are only 676 permutations of the other two letters you could fit in your tray, so with STARE you can bingo with almost any combination. I leave learning those 609 words as an exercise for the reader.
What about a different dictionary? Here's the same run I first did but with the Scrabble Official Dictionary, Third Edition.
Of course, some of this is common sense: EST gives you the superlative of many words. Likewise, ER gives you the comparative along with the prefix RE and the "person who does" suffix. Those four letters plus vowels other than U and Y form top contenders in many of the runs.
If you're playing WWF (or Scrabble), work to keep STARE in your tray, playing or swapping other tiles, until you get a bingo. Placing it on the board, of course, is a different matter.
Somewhere in my library of books is one called Everything Scrabble, a guide to improving your Scrabble game. I read through it a few years back and have probably forgotten half its contents, but I do remember the author's comment that highly ranked Scrabble players often swap tiles in an effort to increase their chances of hitting a bingo (a word that uses all seven letters on your tray).
I've been playing Words With Friends (screen name: linechef) and have been thinking about that Scrabble strategy. Bingoing is less of an advantage in WWF. They're only worth 35 points instead of 50, and they often open up large swaths of bonus tiles for your opponent to gobble up. But they are usually worth it if you can hit a bonus tile yourself.
Everything Scrabble provides the scoop on which five tiles you should keep in your tray to maximize bingoing, but my mind let that knowledge go into the abyss. So I decided to write a program to rediscover it.
You can imagine how the program works: Go through every seven- and eight-letter word in the Words With Friends dictionary, the ENABLE wordlist, and find every unique combination of five letters in it. Keep track of how many copies of that combination you've seen throughout the dictionary, and then sort.
Let's start with the answers first. Here are the top 20 combinations, along with the number of times they occurred (the ENABLE list has 172,823 words):
aerst 2413
eirst 2199
einst 1780
eorst 1698
eerst 1646
einrs 1596
aeist 1587
aeirs 1484
aelrs 1468
aelst 1350
aenst 1346
eilst 1338
eiprs 1286
aeirt 1273
aeprs 1266
aenrs 1258
aeins 1245
einrt 1238
deirs 1233
ainst 1232
And here's the Ruby code. Ruby 1.8.7 added a
combination method to the Array class, which makes this program tiny. You tell it how big you want each subset to be, and you pass it a block which takes each subarray as an argument.
wordlist = ARGV[0]
combos_to_count = Hash.new(1)
File.open(wordlist,"r") do |file|
line = file.gets
while(line)
line = file.gets
if line then
line.chomp!
next if line.length != 7 && line.length != 8
chars = line.split(//)
chars.combination(5) do |combo|
combo.sort! {|a,b| a <=> b}
combo_str = combo.join
combos_to_count[combo_str] = combos_to_count[combo_str] + 1
end
end
end
end
sorted_combos = combos_to_count.sort {|a,b| b[1] <=> a[1]}
(0...20).each {|num| puts sorted_combos[num][0] + " " + sorted_combos[num][1].to_s}
Once you have the code, you can start asking other questions. Is the list different for words with exactly seven letters? A little bit:
aerst 609
eirst 493
eorst 450
aelrs 413
eerst 404
einst 385
aeprs 379
einrs 348
aelst 346
eilst 335
acers 331
aeirs 330
aenst 327
aeist 322
deirs 320
aders 318
eiprs 317
eilrs 297
aerss 296
deers 295
Note that there are only 676 permutations of the other two letters you could fit in your tray, so with STARE you can bingo with almost any combination. I leave learning those 609 words as an exercise for the reader.
What about a different dictionary? Here's the same run I first did but with the Scrabble Official Dictionary, Third Edition.
aerst 2395
eirst 2182
einst 1759
eorst 1670
eerst 1628
einrs 1581
aeist 1562
aeirs 1473
aelrs 1458
aelst 1335
aenst 1327
eilst 1317
eiprs 1278
aeprs 1258
aeirt 1257
aenrs 1239
aeins 1232
ainst 1224
einrt 1219
deirs 1218
Of course, some of this is common sense: EST gives you the superlative of many words. Likewise, ER gives you the comparative along with the prefix RE and the "person who does" suffix. Those four letters plus vowels other than U and Y form top contenders in many of the runs.
If you're playing WWF (or Scrabble), work to keep STARE in your tray, playing or swapping other tiles, until you get a bingo. Placing it on the board, of course, is a different matter.
Wednesday, August 25, 2010
Rails Scaffolding In Java
I recently needed to add a new business object to a system I'm developing at work, and it was a bit of a drag.
If you follow any of the normal Java patterns, you probably know what I'm talking about. You need the file for your business object. You need a service class responsible for returning instances or sets of instances from the DAO layer. You need a SQL file that defines the database table the business objects will be stored in. In our case, we also have queries in resource files, so we need a file for that as well. You need to edit your Spring config files to point to the new service. And so on. Your environment probably has some overlap with mine, and probably requires pieces mine doesn't.
You can simplify a lot of this, of course. You can make a service base class with Java Generics that will give you a lot of the type-safe methods you'd want. IDEs will let you set up templates, but you'll still have to click through a few menu options to get you the files you need. And the more files, the more clicks. And I like running an IDE-neutral team, so I wouldn't want to do something specific to one workflow.
I wanted a better way. Specifically, I wanted what Rails provides. You type a command at the command line, and you get all the files you need for working with the object. (XCode offers similar functionality.)
And I thought, "Well, why not?" My build system is written in Ruby, and Ruby's ERB templating system is built in to the language.
It took about an hour to get the main system up and running. I now type
Here's the heart of the code.
To give you an idea of what the templates look like, here's some code from the java file templates:
I don't yet go the full Rails route and specify all the properties on the command line, but this takes care of getting the boring parts of business object development out of the way, enforcing consistent naming schemes, and ensuring that the developer doesn't forget to check in some file. I also don't yet modify my config files, but that won't be too tough to add.
If you follow any of the normal Java patterns, you probably know what I'm talking about. You need the file for your business object. You need a service class responsible for returning instances or sets of instances from the DAO layer. You need a SQL file that defines the database table the business objects will be stored in. In our case, we also have queries in resource files, so we need a file for that as well. You need to edit your Spring config files to point to the new service. And so on. Your environment probably has some overlap with mine, and probably requires pieces mine doesn't.
You can simplify a lot of this, of course. You can make a service base class with Java Generics that will give you a lot of the type-safe methods you'd want. IDEs will let you set up templates, but you'll still have to click through a few menu options to get you the files you need. And the more files, the more clicks. And I like running an IDE-neutral team, so I wouldn't want to do something specific to one workflow.
I wanted a better way. Specifically, I wanted what Rails provides. You type a command at the command line, and you get all the files you need for working with the object. (XCode offers similar functionality.)
And I thought, "Well, why not?" My build system is written in Ruby, and Ruby's ERB templating system is built in to the language.
It took about an hour to get the main system up and running. I now type
buildr :biz_object object=com.ea.foo.TestTest and I get a src/main/java/com/ea/foo directory, a TestTest.java and TestTestService.java file in that same directory, a sql file with the table definition (named test_test), and an empty query file. I also get a perforce changelist (via p4Ruby) with the description filled in and all the new files added.Here's the heart of the code.
template_to_final is a hash of template file name locations to the end file destination. The local variables exposed by the call to binding include the package name, the Java object name, and the SQL-friendly name:
template_to_final.keys.each do |key|
File.open(key) do |file|
b = binding
erb = ERB.new(file.read)
outfile = File.new(template_to_final[key],"w")
puts "Creating #{outfile.path}"
outfile.write(erb.result(b))
outfile.close
end
end
To give you an idea of what the templates look like, here's some code from the java file templates:
package <%=java_pkg%>;
@Entity
@Table(name="<%=java_sql_name%>")
@SequenceGenerator(name="objIdGenerator",sequenceName="object_id_seq",allocationSize=1)
public class <%=java_obj%> {
I don't yet go the full Rails route and specify all the properties on the command line, but this takes care of getting the boring parts of business object development out of the way, enforcing consistent naming schemes, and ensuring that the developer doesn't forget to check in some file. I also don't yet modify my config files, but that won't be too tough to add.
Thursday, August 5, 2010
Scripting Campfire
My team uses Campfire, a web-based chat tool from 37signals, to communicate throughout the day. We've divided it into various rooms, one of which is a Status room where we do a virtual version of the daily stand-up that many teams do: a quick meeting where everyone says what they're working on. (One of our team members is in England, and our hours vary a bit, so a classic stand-up isn't practical.)
About a month ago, one team member started posting the date in the status room before anyone gave their update. For some reason, Campfire itself doesn't do this reliably, and when you have a room with nothing but status updates, it's not always clear where one day's messages end and another's begin.
Usually one or two people on the team did this, but I followed suit on a couple of occasions, and of course thought about automating it. We already had a bot account for some very early automation, and Campfire has a good web service. Insert Tab A into Slot B.
Here's the relevant Ruby source code, which I have hooked up to a cron job (Note that if you want to script Campfire, I suggest creating a "Bot Test Room" that you can use for experiments without spamming your real rooms):
Filling in the date each day isn't a huge time savings: It will take us a lot of days to recoup the time I spent automating a five-second typing task. But once I figured out the gist of posting to Campfire, I started adding new functionality. Our "Today is ..." status message now includes a few choice statistics — gleaned from our telemetry system — about gameplay from the previous day. I also wrote a "canary" script that does a health check on our dev server and posts to our general chat room if it seems to be slow.
Naturally, some of my co-workers have suggested writing an adventure game on top of the API. That may be a bit silly, but it does emphasize a point I often make: You can't really imagine all the possibilities for a technology until you get your hands dirty a bit and play with it.
This whole experience underlines again why web applications should have APIs. The lack of a date is probably a Campfire bug, but we don't have to wait for them to fix it. And we've added functionality that is only relevant for our team, resulting in something that more closely ties in with our real needs.
About a month ago, one team member started posting the date in the status room before anyone gave their update. For some reason, Campfire itself doesn't do this reliably, and when you have a room with nothing but status updates, it's not always clear where one day's messages end and another's begin.
Usually one or two people on the team did this, but I followed suit on a couple of occasions, and of course thought about automating it. We already had a bot account for some very early automation, and Campfire has a good web service. Insert Tab A into Slot B.
Here's the relevant Ruby source code, which I have hooked up to a cron job (Note that if you want to script Campfire, I suggest creating a "Bot Test Room" that you can use for experiments without spamming your real rooms):
require 'date'
require 'json'
require 'net/http'
require 'net/https'
require 'uri'
def send_text_to_campfire(text)
message = {
:message => {
:type => "TextMessage",
:body => text
}
}
send_to_campfire(message)
end
def send_to_campfire(message)
url = URI.parse("https://<your base URL>/room/<your room number>/speak.json")
request = Net::HTTP::Post.new(url.path)
request.basic_auth(<your auth token>,<any password string>)
request.content_type = 'application/json'
request.body = message.to_json
request.content_length = request.body.length
http = Net::HTTP.new(url.host,url.port)
http.use_ssl = true
response = http.start do |http|
http.request(request)
end
puts response.body.to_s
end
# construct message
today = Date::today
formattedDateString = sprintf("%02d/%02d/%4d",today.mon,today.mday,today.year)
dateStatusString = "=== Today is #{Date::DAYNAMES[today.wday]}, #{formattedDateString} ==="
#send status message to campfire
send_text_to_campfire(dateStatusString)
Filling in the date each day isn't a huge time savings: It will take us a lot of days to recoup the time I spent automating a five-second typing task. But once I figured out the gist of posting to Campfire, I started adding new functionality. Our "Today is ..." status message now includes a few choice statistics — gleaned from our telemetry system — about gameplay from the previous day. I also wrote a "canary" script that does a health check on our dev server and posts to our general chat room if it seems to be slow.
Naturally, some of my co-workers have suggested writing an adventure game on top of the API. That may be a bit silly, but it does emphasize a point I often make: You can't really imagine all the possibilities for a technology until you get your hands dirty a bit and play with it.
This whole experience underlines again why web applications should have APIs. The lack of a date is probably a Campfire bug, but we don't have to wait for them to fix it. And we've added functionality that is only relevant for our team, resulting in something that more closely ties in with our real needs.
Subscribe to:
Posts (Atom)