Database Updates, Redux

Before I came up with my solution for updating the embedded database in my iPhone app, I had a different strategy that I couldn't figure out how to implement. Of course, after I had the new solution in place, I figured out how to implement my first idea.

I wanted to figure out how to construct method names on the fly and invoke them. That way, I could have dbChangeScript0,dbChangeScript1, and so forth, and then invoke them in a while loop that ran until the application didn't have a method with the relevant name. I wanted JavaScript's eval function in Objective C. I didn't see how to do it, though.

Today, I discovered NSSelectorFromString, a function that transforms a string into a selector (essentially, a function pointer). I used it to implement Plan A as follows:

- (void) installChangeScripts {
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];

    int versionNum = [self dbVersionNumber];
    SEL curSelector = NSSelectorFromString([NSString stringWithFormat:@"dbChangeScript%d",versionNum + 1]);
    while ([self respondsToSelector:curSelector]) {
        [self performSelector:curSelector];
        [self updateDbVersion:versionNum +1];
        
        versionNum = [self dbVersionNumber];
        curSelector = NSSelectorFromString([NSString stringWithFormat:@"dbChangeScript%d",versionNum+1]);
    }
    [pool release];
    
}

Is this a better version? I've lost the self-documenting method names such as makeVersionTable, but I've removed an in-memory array and a #define that I had to remember to update when I updated the array. In other words, I've increased the code's maintainability. (The code looks shorter, but only because I also factored out the version update into a new method.) Overall, I consider this a win.

The Ruby on Rails community has a mantra: convention over configuration. In Rails, you can get a full website up and running just by following a few naming conventions — ones you probably would have used anyway. A class of name X will translate to a database table named Y and so on. Compare that to Spring, a powerful Java framework that requires constant work in the XML-based configuration files.

In essence, I voted for the convention side of the fence in this code. By following a consistent naming scheme, I can write a small method that will do all the work. All I have to do is add a new method with a predictable name. Compare that to an array that I had to remember to configure properly.

On-The-Fly Database Updates

I love the fact that the iPhone comes with an installation of SQLite*. As someone who's worked with relational databases and the SQL language for his entire professional career, I happily use it to store the information I need for my iPhone app.

But with an embedded database comes the need to alter its structure on the fly as my data requirements change. I'm still in beta testing, so I could tell all my testers to nuke their data the next time they install my application, but I'd prefer to avoid that, and I certainly won't tell my future customers to do that.

Instead, I came up with a simple system for making the app run its own database updates. I have a class, DBAccess, that puts an object-oriented wrapper around the very procedural SQLite calls. I created a new method, installChangeScripts, that looks like this:

- (void) installChangeScripts {
    SEL changeScripts[CHANGE_SCRIPTS];
    
    changeScripts[0] = @selector(makeVersionTable);
    changeScripts[1] = @selector(addTaskDuration);
    
    int versionNum = [self dbVersionNumber];
    
    for (int changeScriptIndex = versionNum + 1; 
                changeScriptIndex < CHANGE_SCRIPTS; 
                changeScriptIndex++) {
        [self performSelector:changeScripts[changeScriptIndex]];
        NSString *versionUpdateQuery = 
              [[NSString alloc] 
                      initWithFormat:@"update app_version set version_num = %d",
                                                     changeScriptIndex];
        [self executeUpdate:versionUpdateQuery];
        [versionUpdateQuery release];
    }
}

The gist is straightforward: Make an array of method selectors (Objective C's way to find a method at runtime), iterate through them starting at the database's current version number + 1, and update the version number after the script has run. The only tricky part is that the version table doesn't exist in the default version of the database, so I have logic to handle the case where it doesn't exist:

- (int) dbVersionNumber {
    int versionNum = -1;
    // get the current version. handle this case differently since it drives the rest
    // of the logic
    NSArray *queryResult = [self executeQuery:@"select version_num from app_version"];
    if ([queryResult count] > 0) {
        // handle the result 
        NSDictionary *versionResult = [queryResult objectAtIndex:0];
        versionNum = [[versionResult objectForKey:@"version_num"] intValue];
    }
    [queryResult release];
    return versionNum;    
}

Originally, this logic was in the installChangeScripts method, but that was an easy refactor: self-contained logic that only produces a single value used by the rest of the method. See the Extract Method refactoring sequence.


Not surprisingly, the first change script creates the version table.

- (void) makeVersionTable {
    [self executeUpdate:@"create table app_version (version_num integer)"];
    [self executeUpdate:@"insert into app_version values (0)"];
}

There's at least one other good refactoring target here: The setup of the list of install scripts should be in its own method because it's a separate concept from actually running the scripts. I just need to remind myself how to deal with calloc, free, and other denizens of the pointer/array abyss.

* I also love the group's motto, "Fast. Small. Reliable. Choose any three." A nice spin on the various "choose two" sayings such as "Fast. Scalable. Cheap. Choose two."

Old-School Coding Techniques

Computerworld takes a look at old-school programming techniques we should all be happy to forget. But what about old-school techniques that are still relevant or should be revived?

Meaty Learning Projects

My job description implicitly includes understanding current technologies, philosophies, and methodologies. Usually, this is just a surface-level knowledge about what's going on in the industry, but occasionally I decide to dig deeper.

For instance, right now I'm learning Scala because I think it will solve some thorny performance issues at work. But it's a Catch-22: I don't want to introduce a whole new language without knowing if it will really benefit us, but I can't know if it will benefit us until I try it with a meaty project. And it's even worse than normal in this case: The feature I really want to explore — actors — requires a decent chunk of knowledge before I can take advantage of it.

Over the years, I've developed a solution to this problem. I have a substantial, but not overly complicated, program that I write whenever I come to a new platform. I write a program that assists me in finding solutions to the kind of thorny word puzzles you get from the National Puzzlers League. It requires me to open a file, read in words, and see if they match the pattern the user is searching for. No network connection. No database work. No scalability issues.

It's a problem space I understand, so I can dive in without much thought. I get a deeper knowledge of the language than I'd get from working through a tutorial's abstract problems. It forces me to probe and dig through the libraries to accomplish what I need. I don't get a thorough understanding of the language, but I get pretty far.

Overall, it's a plan that's worked well for me.

Article On EasyMock

IBM's DeveloperWorks features an article about EasyMock, a framework for building mock objects for unit testing.

The Root Of All Evil

In my most recent round of interviews, I chatted with the founder of U1 Technologies. He and I had a lot in common philosophically and, with a slight shift in timing, I might very well have been working for him instead of Maxis.

One of the things that caught my attention was his phone screen questions. I've been on both ends of these things, and they usually consist of simple questions that ensure that the candidate isn't obviously inflating his or her resume. Things like: "Explain how an Iterator works." or "When would you use a List versus a Map?" (Yes, I've actually been asked these.)

Nathaniel's first question reflected his company's decision to only hire senior engineers: "When should you optimize your code?" Fifteen years of software engineering provided a quick answer: "When you know there's a performance problem with it."

"Premature optimization is the root of all evil," is a quote that's been attributed to both Donald Knuth and C.A.R. Hoare. (Most people cite Knuth, who cites Hoare, who denies saying it, according to Wikipedia.)

This may sound at odds with one of my other favorite programming axioms: "The earlier you find the bug, the cheaper it is to fix." But optimizing code usually goes hand in hand with making it less maintainable and reusable. Specializing algorithms, caching values, adding different control structures, and more all add complexity. If you don't have performance numbers to back up that work, you're adding the complexity and (probably) not getting any benefit from it.

You may have inefficient code that rarely gets called. You may think something's inefficient when in fact the compiler or optimizer or processor does just fine with it. Or your code could actually be inefficient and it might just not matter enough to affect the user.

I've seen this in practice for years. And yet the temptation to optimize is always there, coaxing even the best programmers into a fen of tangles and mazes and confusion.

While I was working on the "real time time update" feature of my reusable date picker pop-up, I set it up so that the delegate would be called for every UIControlEventValueChanged message from the UIDatePicker.

I envisioned the date and time changing as the wheels in the date picker spun to their new locations, so I decided to be smart about calling the delegate method.

I could have written this in my event-handling method:

if ([delegate respondsToSelector:@selector(dateChangedWithoutSaving:)]) {
[delegate dateChangedWithoutSaving: date];
}


But I assumed that respondsToSelector would be a relatively expensive call. I also assumed that the runtime binding to dateChangedWithoutSaving: method would be costly. And I didn't want that cost slowing down the user's interaction with the date picker wheels. Instead, I made a boolean in the controller that tracked whether or not the delegate actually supported the method and set it at the same time I stored the reference to the delegate:

- (void) setDelegate: (id) delegate {
...

delegateRespondsToContinuousDateChanged = [delegate respondsToSelector:@selector(dateChangedWithoutSaving:)];
}


I changed my event-handling logic accordingly:

if (delegateRespondsToContinuousDateChanged) {
[delegate dateChangedWithoutSaving:[myDatePicker date]];
}


Then I fired up my app.

And discovered something I didn't know. That UIControlEventValueChanged message only gets sent once the date picker wheels have finished spinning. Rather than sending a message for every number that cruises by, it only sends, essentially, one message per gesture. My optimization didn't buy me much, since the code can't possibly interfere with the spinning wheel animation.

So what did my "clever" optimization get me?

• My own implementation of setDelegate (rather than the one created by Objective C's synthesize keyword)


• An extra layer of logic in my event-handling code.

I've added more complexity (the extra layer of logic) and more lines of code (the setDelegate method) with the concurrent possibility of bugs.* Performance gain: virtually none.

Premature optimization is the root of all evil.

* Yes, of course, this isn't much extra complexity or code. But add up enough items like this, and you get a significant amount of technical debt

Delegation And Redesign

One side effect of programming for the iPhone (or Mac OS X) is a heightened awareness of the delegation pattern. It seems like every class in the framework has delegate information.

When I started my UI redesign work in my iPhone app, I saw a perfect opportunity for implementing my own delegate pattern.

The idea behind delegation is straightforward. Object A holds a reference to Object B. Object A goes about its merry way performing its duties, except that every now and then it says to Object B, "Hey, I need you to handle this, okay?" To everyone else, it looks like Object A did the work. A good example is the UITableview in the iPhone framework. It supports two delegates, one for supplying the table's data, one for handling the table's behavior. The data delegate has methods such as numberOfSectionsInTableView; the behavior delegate has methods such as tableView:didSelectRowAtPath:. The table view doesn't care about anything else those objects might do: It just expects them to do what it needs.

The pattern encourages code reuse (I have a single instance of a delegate that handles common behavior for all of my text fields) and separates business logic from other layers (I have specialized objects that act as my tableview data sources, leveraging a model object in different ways, all without modifying UITableView in any way). It lets you make complex object compositions from subsystems. It makes breakfast for you in the morning. It's great. (As with all general-purpose design patterns, you may need to consider performance and threading issues when putting it into practice.)

In Objective C, it's easy to use a delegate:

id delegate; // set by an outside client
[delegate invokeDelegateMethod]


Even though the compiler doesn't know if delegate implements invokeDelegateMethod, it compiles this code with only a warning. (id is shorthand for "an otherwise untyped object.") Of course, if you try and run this, and delegate does not implement invokeDelegateMethod, you'll get a runtime exception. This may be what you want, but you can also use:

id delegate;
if ([delegate respondsToSelector:@selector(invokeDelegateMethod)]) {
[delegate invokeDelegateMethod];
}


This makes the method optional. You can be formal and specify a protocol that the object must declare to get yourself some type safety, but it's not strictly necessary. Java has similar "rummage around in an object's belly and see what it contains" tools in its reflection API, so you could easily hook up delegate patterns the same way in that language.

For my redesign, I had several places where I needed to pop up a view that had a UIDatePicker, a Done button, and a Cancel button. The idea being that the user needs to pick a date and tell me when s/he's done. The delegate pattern seemed to be the perfect approach, since I had one class that I needed to reuse throughout the application and have it fit seamlessly into the flow of each area.

In my first pass, I implemented two delegate methods, dateSaved: (NSDate *)date and datePickerViewDismissed.

I made the first one optional: Why should the date choosing view care whether the client actually wants to know about the date that was chosen? I made the second method required (i.e., I don't check to see if the delegate supports the method before calling) because that's the method that triggers the client to release any memory it used for the view. I wanted to make sure that I'd know right away if I forgot to implement that method.

I wired the view into the first use case: A screen that shows one date at a time would let the user jump to any arbitrary day, rather than nexting and previousing a bunch. Everything worked fine, so I started to wire it into the next part of the app, where you set the time for a single task. That's when I had the eye candy idea for my ScheduleButton. With a bit more work, I could make a delegate method for the date picker view that would fire as the date changed, which meant that I could update the button's text as the user spun the wheels.

My new version added a new delegate method, dateChangedWithoutSaving: (NSDate *)date. But what if the user cancels? My architecture didn't support that, so I changed the datePickerViewDismissed method into datePickerViewDismissedWithButton: (DateChooserButton)btn (DateChooserButton being an enum representing the two choices). Now a client could grab a copy of the date, bring up the view, update the ScheduleButton helper object as the date changed, and then reset it to the original date if the user pressed cancel.

Enhancing UIButton. Or Not.

As part of a UI redesign in my iPhone app, I realized I had a few concepts that would be repeated in various places, so of course I wanted to bundle them into reusable components.

One idea was a button whose text would always read "<some fixed prefix>: <some mutable date>." The date changes as the containing view is visible, but the prefix is fixed early in the button's lifecycle.


I made a new class, called ScheduleButton, that subclassed UIButton. I added methods for setting the date format, the actual date, and the prefix. These call down into a single "updateButtonText" method that constructs the appropriate string and would have used the inherited UIButton methods to change the text.

So far, so good.

I dragged a UIButton object from Interface Builder into my app, and set the class to ScheduleButton (instead of UIButton, the default). I launched my app and saw the button,but without a border.

I could probably figure out the reason for this. I suspect it has something to do with the fact that UIButtons are made with a class method, buttonForType, rather than a normal alloc/initWithFrame flow (at least from a client's perspective: I imagine initWithFrame gets called in there somewhere).

But after hitting my head against several other issues during this redesign, I pretty much just said, "Fuck it." (Did I mention that I'd swear on this blog? Well, I will.) Instead, I converted ScheduleButton into a helper class that holds a reference to a UIButton. The controller still interacts with ScheduleButton, using the same methods, and they feed into one centralized method, but that one method now invokes setTitle on the contained UIButton rather than inheriting it. That means that other objects can muck with the UIButton on their own, without my helper being the wiser, though that's not likely to be a problem since I'm the sole user. (There's probably yet another solution involving Objective-C's category functionality, but I don't want to modify every instance of the UIButton class in my application: I just want to modify particular objects.)

I tried to lend credence to this implementation by convincing myself that this was an instance of the Decorator design pattern, but it's pretty clearly not. A proper Decorator shares a parent with the decorated object and implements every method by calling into the (completely contained) decorated object. It then adds extra functionality that the decorated object doesn't know about.

Ah, well. Sometimes you just need something to work.

Evolutionary Architecture and Emergent Design

IBM's DeveloperWorks has an interesting series on finding the design inherent in systems through test-driven development and other strategies.

Constant Refactoring: A Practical Benefit

As I program, I continually refactor the code I'm in. It's habit at this point. I rarely get the opportunity to do a massive change, but even the basic work I do — removing duplicated code, generalizing logic, dividing up large functions, and so on — reaps unexpected benefits.

Refactoring partly makes code more maintainable, but its main purpose is to make the code flexible enough to respond to new design requirements. I try to code on the assumption that change will come, but if I could predict the exact direction of every future change and thus craft a perfect design, I'd be writing this from my French Chateau.

Here's a simple example I recently found in my iPhone app.

One of the tabs lets you move between dates, one at a time, to see your schedule for that particular date. In my initial design, I didn't let you change dates because that view exists to show you what's going on today. But I quickly realized that I wanted to at least see what tomorrow would bring.

When I made that change, I had two buttons that would switch to either the next date or the previous date. They each called separate methods — nextDay and previousDay — that were fairly similar to each other. I was working through the basic functionality, so I didn't register the commonality. I think this is common enough: You're so focused on one thing that you don't notice something else.

On a later pass through the code, fixing something else, I realized how similar the methods were, and I wrote a new method that would just adjust the date based on some time interval. I gutted nextDay and previousDay and replaced them with calls to the new method that passed either 86400 or -86400. (The number of seconds in a day.)

After that, one of my testers suggested that I let the user swipe across the date to move to the next or previous day, depending on the direction. Once I figured out the gesture system for the iPhone, I just made each swipe call either of the two appropriate methods, which in turn called into that common method.

But then I realized that I needed to add animation to show the user what was happening. I put the animation logic into the generalized method and made it smart enough to know that a negative number meant animate one way, while a positive number meant animate the other way (animate from the left in the first case, because you're moving to the previous day, and animate from the right in the second). That handled the animation for the two cases.

A couple of my testers asked for the ability to jump to an arbitrary day. I decided to look into implementing that this morning, and once I opened the code, I realized it would be a snap. I bring up a UIDatePicker, the "choose from a list" UI in the iPhone, and then just pass the time interval between the user-selected date and the current date to that one common method. The logic is the same, and the code animates appropriately: If you jump to some arbitrary, far-future date, the screen wipes from right to left, as you'd expect. Aside from bringing up the date picker, I wrote exactly one line of code to enable the feature.

For this simple example, I probably would have found my way to the same end result: an all-purpose method for jumping to any given date. But the fact that I had refactored earlier — just because — made the solution obvious and quick.

When I started this feature, you couldn't switch dates at all. Piece by piece, I built the infrastructure so that now you can jump to any date with gestures and animations. That wasn't the original design, but constant refactoring allowed me to eventually support this radically new direction with a minimum of "from scratch" code.

iPhone Mail Reformatting

I recently wired up the ability to email lists from an iPhone app I'm writing. Setting this up was pretty straightforward (see the Apple docs on the subject).

But since I'm sending lists, and since my list might have headings with one level of subitems, I wanted to create an email that looked like this:

Heading
   Item 1
   Item 2
   Item 3

I added the appropriate spaces to my email body, URL encoded it with NSString's stringByAddingPercentEscapesUsingEncoding method, and shipped it off to Mail with [[UIApplicaton sharedApplication] openUrl:]

What I saw when Mail started was this:

Heading
Item 1
Item 2
Item 3

I assumed my indent code didn't work, so I looked at the result in the debugger. The encoded string had the three spaces in front of each item; Mail had decided it didn't like the spaces at the beginning of the line. I tried using a tab instead. No dice.

Thinking that I could trick Mail by putting a different character at the beginning, I sent it email that should have looked like this:

Heading
-   Item 1
-   Item 2
-   Item 3

What I got was:

Heading
- Item 1
- Item 2
- Item 3

Mail not only doesn't like white space at the beginning of the line; it doesn't like multiple spaces at all.

I find this very annoying. I'm sending Mail a piece of text that I want it to use, and it's making its own decisions about that text. Mail would let me type " Item 1"; it just won't take it from a passed-in URL.

Grr. In the end, I decided to just live with the last version of the list. A friend suggested "--Item 1" but I don't like the way the dashes run into the text in that form. Maybe "-- Item 1."

Technical Debt

Some time in the last year, I learned the term Technical Debt. It's a brilliant metaphor for the more hurried, less-thought-out code that crops up in any project with a ship date.

The idea is simple: Just as you actively take on financial debt to give yourself new opportunities, you take on technical debt to ship a product out the door and make money from it. Financial debt is simple money, but technical debt is more complex. Any decision you make to ship the product at the expense of a clean code base qualifies. Every test you don't write is technical debt. Every piece of duplicated code that hasn't been refactored into common functionality is technical debt. Every todo comment is technical debt. Every bug you ship with is technical debt.

But too much technical debt, and you never pay down the principal, in this case the ability to quickly add new features or fix old bugs. That duplicated code? If you never refactor it, every bug fix in that code has a cost proportional to the number of places it's used. And that's if you actually remember where all those pieces are. That todo comment you never go back to? Some day you'll need to enhance the code around it, and that todo will add to the cost of the work. Add enough of these little debts, and the cost for any given feature multiplies.

Unfortunately, while it's easy to measure financial debt, it's tough to measure technical debt, which makes it difficult to use when arguing for time to fix things up in the code base. If you never need to fix a bug in that duplicated code, for instance, you'll never experience the extra cost of fixing it in every location. Of course, the idea that a chunk of code will sit forever untouched once it's written is ludicrous, but you can't sell the idea that something might be a problem someday outside of development teams.

There's also no good way to flag technical debt. Perhaps as programmers we need a more formal way — beyond todo comments — to note down that we made a deliberate choice to take on technical debt. But then we have to advertise to the larger world that we deliberately did something not in the best possible way. I know that won't sell.

About This Blog

I'm a programmer, both professionally and passionately. This blog will contain all my thoughts, explorations, and ramblings about my programming antics.

For the most part, this blog will cover my personal programming projects, but work topics will sneak in from time to time if they don't have any confidential information.

Most people would call me a Java programmer, but that's just how my career has shaken out. I consider myself an "all-purpose" programmer. The topic itself intrigues me, regardless of the particulars of any one language.