Learning JMeter

At work recently, I set up automated load testing for some new features I'm working on. There are teams within EA that do large-scale load testing, but I have a need they can't meet. I want to know, every single day, if our performance metrics have changed in a marked way even against a development server. If they have, the change probably came from code that was checked in that day. If performance went down, our team can find the fix quickly or acknowledge the new baseline. If performance went up, we can figure out why and see if the fix can be applied in other places.

I downloaded JMeter and started giving myself a crash course on the software. I've used it in the past, but it was much simpler in those days. So were my requirements.

A simple case was pretty easy: hit a couple URLs that don't require parameters but do return dynamic data. I set up a thread group, the "how many threads should I use" required container, then I added HTTP requests that hit the URLs to that thread group, and then I added a "listener" that parses the responses. That gave me basic familiarity with the tool and allowed me to set up an automatic, nightly run of any checked-in JMeter tests. We use Hudson as a build system, and it has a nice little JMeter plugin that will graph the response times over time and also give you a simple per-run breakdown of performance.

But then I wanted a test suite that would hit a URL, parse the response, and run subsequent tests based on the data (an ID) that came back in the first response. There are plenty of hints that JMeter can do this kind of thing, but it took a lot of muddling to get it right.

I often say that I should contribute to open-source projects not as a professional programmer but as a professional writer. Documentation is often sparse and unclear, even on an established project like JMeter. Where is the "cookbook" section with "you want to do this common thing; here's how" entries? Where is the list of best practices?

You read their manual, you read the FAQ and the wiki, and you still spend a lot of time bumbling about, poking this and prodding that to see if the errors clear up. And I've barely touched the more powerful features.

Here are a few of the things I learned along the way, which I'm posting mostly for my own benefit.

• JMeter can't do anything with a response — even show it to you — without knowing the MIME type. On the one hand, that forced me to return a clean response. On the other hand, I feel like JMeter should fall back to plain text if it has no other information.



• Extract data from a response with a post-processor. If you want to use the response data from one http request in subsequent calls, you make the post-processor a child of that http request. (You can make a post-processor a sibling, in which case it runs after every request in the thread group.) I used a regular expression post-processor that matched the entire response. Even though there was no other data, I still needed to enclose the regex in (). I also needed the $1$ default template, even though I wasn't doing anything with multiple values.



• It seems like every JMeter tutorial suggests that you add a Graph Results listener to your test plan. It shows you a graph of your response time. Ooh. Aah. Pretty. Also? Useless for debugging. I moved through errors at a rapid pace after I added a View Results Tree listener to the thread group. With that listener, you can drill down on the request/response for each and every server call. I also found the Summary Report listener to be more useful than Graph Results. Keep the pretty picture in to show your boss — it doesn't add time — but add the others to make your life easier.



• Learn to love the JMeter functions. They're buried in Chapter 19 of the JMeter manual, but they're essential for making scripts that can be re-used in multiple places. I sprinkled the ${__P(name)} function throughout the text fields of my scripts so that I could fetch command-line properties such as TARGET_SERVER and TARGET_PORT. That means that running my scripts against a different environment will require nothing more than a different set of command-line arguments (prefaced with -J).

Profiling With Spring's AOP

Recently, at work, I wanted to set up an easy way to get lightweight profiling information about method calls in our system. There are lots of heavyweight profilers out there that give you great features at the cost of bogging down the machine they're profiling. Fine if it's a development box; less so if it's a production machine your users are using.

Because we're using Spring, we have access to a pretty decent aspect-oriented-programming system. (AOP is basically a way to add system-wide concerns to objects without those objects having to be aware of the systems in any way: no subclassing or encapsulation.) Since profiling is a classic cross-cutting concern — something you need throughout the system but don't want any individual object to know anything about — Spring's mechanism works pretty well.

In my final system, a programmer only needs to add a @Profile annotation on a method, and it will generate profiling information in the logs. I didn't want to profile every method all the time because of the noise it would generate and the performance hit it would cause. The programmer can refine the output of the profiling system a bit, but the default is to show method name, args, return value, and timing information.(Spring's AOP framework is limited to non-private methods that are proxied by the Spring system, so private methods and intraobject calls don't get tracked. If that becomes a problem, I'll add in true AspectJ support, but that shouldn't require any real changes in my code, since Spring's support uses AspectJ syntax.)

Here are the key parts.

First, the profiling method, with a pointcut/advice combination that says "run this code around any method annotated with a @Profile annotation.

    @Around("@annotation(profileAnnotation)") 
    public Object profileMethod(ProceedingJoinPoint pjp, 
                Profile profileAnnotation) throws Throwable {
        
        StringBuilder buf = new StringBuilder("method: " + 
                               pjp.getSignature().getName());
        if (profileArgs(profileAnnotation)) {
            buf.append("; args: ");
            if (pjp.getArgs() != null && 
                pjp.getArgs().length > 0) {
                buf.append("[");
                for (Object o : pjp.getArgs()) {
                    buf.append(o.toString() + ",");
                }
                buf.append("]");
            } else {
                buf.append(" (no args)");
            }
        }

        long currentTime = System.currentTimeMillis();
        Object retVal = pjp.proceed();
        if (profileReturnValue(profileAnnotation)) {
            buf.append("; returnValue: " + 
                        String.valueOf(retVal)); // valueof prevents NPEs
        }

        if(profileTiming(profileAnnotation)) {
            buf.append("; timing: " + (System.currentTimeMillis() - currentTime));
        }
        log.info(buf.toString());
        return retVal;
}

The Profile annotation looks like this:

/** Annotation to demark methods that should be profiled. */
@Retention( RetentionPolicy.RUNTIME )
@Target( ElementType.METHOD )

public @interface Profile {
    ProfilingType[] value() default {ProfilingType.ALL};
}

And the ProfilingType enum looks like this:
public enum ProfilingType {ARGS,RETURN,TIMING,ALL}

The default output (ProfilingType.ALL) looks something like this:
method: [methodName]; args: [each arg as String]; returnValue: [returnValue]; timing: [some number of milliseconds]

Sad, But Probably True

Reading the website for Apache Jmeter, I noticed this line: "JMeter's target audience is the testing community, which is generally not the hangout of developers or technical people."

Except for those developers who understand that the sooner you catch an issue, the cheaper it is to fix. Which, granted, is not as many as there should be.

Rounded Corners on UITableView

The iPhone UI is a soft, graceful collection of rounded corners. Your application icon is cropped and smoothed out with the ubiquitous curves. The highlight on the selected tab of a tab bar has tiny ones. Grouped table views, the corners of navigation items, the little buttons in toolbars. You can't escape the smooth arcs.

Which would be fine if Apple didn't require you to hand craft them yourself on your UI components. You might expect a "draw this with rounded corners" method. Actually, given their prevalence, you'd expect it to be a setting on UIView: cornerRadius or something like that.

But since that doesn't exist, every developer has to write his or her own take on it. So far, mine has become fairly standard. My UIView classes implement drawRect to set a curvy clip region. I have a utility method that takes any rect, with a desired radius, and returns a path for a rounded rectangle. Add that to the graphics context, call CGContextClip, and voila, I have voluptuousness. This has worked well.

Then I tried to do it on a UITableView embedded in a larger view. It's a view, right? So this should work, right? Except that UITableView's drawRect method doesn't really do anything. The corners don't get clipped if you subclass it and clip within drawRect.

Searching on this topic suggests that this is a common problem. I read various ideas, didn't really like them for one reason or another, and then came up with my own strategy. Why not overlay my table with a view that would act as a window? It would draw the part that I normally clip off and then make the interior transparent. And with a bit of work, I could make it a reusable view I could put atop other views that didn't behave the way I wanted. No more mucking around with clipping in random subviews!

I am not a graphics programmer. Not only did this take me a while, but there are probably better ways to do it. However, you may find yourself wondering how to do the same thing, and so I'm sharing my results.

Here's my init method. I used bgColor to represent the color on the outside of the corners, and I set the actual backgroundColor to clearColor I tried using clearColor as the fill color for my rounded path, but that didn't work. It was transparent over the view's background color, which meant the whole rectangle was filled in with the view's background, not the view underneath.

- (id)initWithFrame:(CGRect)frame {
    if (self = [super initWithFrame:frame]) {
        bgColor = [UIColor blackColor].CGColor;
        self.backgroundColor = [UIColor clearColor];
        self.userInteractionEnabled = YES;
    }
    return self;
}

And here's the drawing code. I used EOFillPath to ensure that my interior was empty. Yes, I hardcoded the radius. Yes, that's lame. I'll fix it in a refactor pass.

- (void)drawRect:(CGRect)rect {
    // Drawing code
    CGContextRef ctxt = UIGraphicsGetCurrentContext();
    
    CGContextSetFillColorWithColor(ctxt,bgColor);
    CGContextAddRect(ctxt,rect);
    CGPathRef curvePath = [Utils roundedCornerPathFromRect:rect withRadius:10.0];
    CGContextAddPath(ctxt, curvePath);
    CGPathRelease(curvePath);
    
    CGContextEOFillPath(ctxt);
    [super drawRect: rect];
}

That's the result of a fair amount of fiddling, and it now works. Except for one problem: You've covered the view you want the user to see with another view, effectively removing the user's ability to interact with it. To get around this, I created a "coveredView" reference within RoundedCornerOverlay and set it in the view controller managing the whole view. I overrode nextResponder in my view to return this reference. Then I overrode the various event methods you inherit from UIView. The final step was to realize I needed to convert incoming points to be relevant within the covered view.

- (void)touchesBegan:(NSSet *)touches withEvent:(UIEvent *)event {
    [[self nextResponder] touchesBegan:touches withEvent:event];
}

- (void)touchesMoved:(NSSet *)touches withEvent:(UIEvent *)event {
    [[self nextResponder] touchesMoved:touches withEvent:event];
}

- (void)touchesEnded:(NSSet *)touches withEvent:(UIEvent *)event {
    [[self nextResponder] touchesEnded:touches withEvent:event];
}

- (void)touchesCancelled:(NSSet *)touches withEvent:(UIEvent *)event {
    [[self nextResponder] touchesCancelled:touches withEvent:event];
}

- (UIView *)hitTest:(CGPoint)point withEvent:(UIEvent *)event {
    return [[self coveredView] 
             hitTest:[self convertPoint:point toView:[self coveredView]] withEvent:event];
}

- (BOOL)pointInside:(CGPoint)point withEvent:(UIEvent *)event {
    return [[self coveredView] 
            pointInside:[self convertPoint:point toView:[self coveredView]] withEvent:event];
}

Independent Developer

I assume that if anyone reads this blog — anyone? hello? — they also read my main blog, An Obsession With Food. So you've probably already seen the announcement: I launched 1.0 of my iPhone app, which helps home cooks organize the prep tasks for a dish.

What that really means is that for the first time in my professional career, I'm an independent developer. Of course, I also work for a company that can pay me. I am not planning my retirement based around sales of my application in the App Store. But I'll be the one doing the marketing — ugh — tech support, and project management. Now I'll have users. At least I hope I will.

So far, I've been the primary user of this app. In a way, this is good, because it's forced me to use the app day in and day out, finding the things that annoy me and then fixing them. But it's also bad. The app is tailored to the way I do things, even though most people probably have different menu planning strategies. I've had testers, of course, who have offered feedback. But, as of today, I have four times as many users as I did before I launched. I'm hoping they'll take the time to write me and offer suggestions, but who knows what will happen?

The Simplest Thing That Could Possibly Work

One of the maxims of Agile development — and especially Extreme Programming — is to write the simplest thing that can possibly work.

There are a few driving forces behind this motto. First, a core tenet of Agile development is to get testable, functional code up and running as soon as possible. If you don't spend a lot of time writing complex code, you can get it running sooner. And complex code is harder to maintain and thus more likely to have subtle bugs. Second, Agile assumes that requirements will change. This in part because requirements always change, but also because the mere act of showing a user a current piece of software will give them new ideas for the features that will really solve their problems. Users aren't good at articulating what they need until they see something "tangible" that isn't what they need. If you get a piece of code up sooner, you give your users the chance to steer the application in the direction they really want, not the direction they thought they wanted. And when they present this abrupt change in direction — as they inevitably will — you don't have to dismantle a complex system.

A close cousin of the "simplest thing" motto is "You're Not Gonna Need It." Experienced programmers tend to think in terms of frameworks they can build, good object-oriented principles, and so forth. But if you spend a lot of time building a nice, modular system with layers and abstraction, you may have wasted your time and effort if there will only ever be one module, one implementation of a given interface, or one class in a layer.

This is not to say that you should be sloppy, or churn out quick and dirty, copy-paste code. You are a professional and should act like one. Your "simplest thing" should be tested and it should be clean code. As the requirements change beyond what your "simplest thing" can do, you can then start to build layers on top of it, refactor to make it more modular, and so on.

All these arguments came back to me when I had to implement templatable text at work. I wanted text that a non-programmer could modify, but it needed to allow variable substitution.

Java's localization system allows this to some degree, except that its variables are numeric, not symbolic. So item zero is always the same, item one is always the same, etc. regardless of where they show up in the string. This makes sense for localization, but is fragile for more general templates. What if we wanted the text to use a different set of variables from one week to the next?

There are a number of general-purpose templating systems for Java: Probably the best-known is Velocity. As I started researching what it would take to implement it, however, the "simplest thing" motto came back to mind. Not that Velocity is difficult to implement, but we only needed to support five variables, and we didn't need any of the flow-of-control logic that Velocity offers.

So I wrote a very simple set of methods that allows me to do string replaces on the five symbolic variables I allowed. The (internal) user can write any text, using some or all of those variables once, twice, or whatever they want. And just in case we ever need to move to Velocity, I used the same variable naming syntax to define our variables.

The code is simple. It has noticeable restrictions (can't use undefined variables, can't define new ones), but in this case, it works exactly the way I need it to. It satisfied the business need, and truthfully, will probably never need to support more. When I needed to add an additional piece of templated text with the same variables, it took about 15 minutes including 10 minutes of testing.

Tutorial System, Part 2: Variadic Methods

In my last post, I mentioned a simple tutorial system that I wrote for my iPhone app: When I pass a key to the system, new users get a text box explaining the purpose of each screen, tap to close it, and then never see it again.

It worked well enough, though my testers have some usability comments that I need to address. But as I started working on version 1.1 of my app*, I realized that I had new needs. I had written, as I try to remember to do despite the temptation to overdesign, the simplest thing that could possibly work. That no longer fit the bill. (Writing the simplest thing that can possibly work assumes that design requirements will change. But since you can't predict those changes, even when you're the one creating them, you shouldn't waste time designing for requirements that don't yet exist. Address the problem in front of you, get functional code up and testable, and then refactor to simplify.)

I wanted the system to handle upgraders, so that upgraders would see text just about the new features, while new users would see a screen giving an overview of the page, including the new features. The key was that when a new user dismissed the tutorial box, s/he would never see the one for upgraders (because there would be no need).

I toyed with a few ideas, but I finally settled on one: Rather than passing a single key to the system, I'd pass a list of keys. The system would iterate through the keys until it found one the user hadn't seen, show the tutorial text for that key, register that the user had seen the key, and then register every subsequent key in the list.

In order to make this work, I used a variadic method, one with an indeterminate number arguments. It's been a while since I've been deep in a C-based language, so I had to refresh my memory about implementing them. (Curiously, the Objective-C bible, Programming in Objective-C, doesn't tell you how to implement them. Perhaps the new edition does.) They rely on the stdargs.h macros, va_start, va_arg, and va_end. In Objective-C, the convention is that the last item in the list be nil. For example, see NSArray's initWithObjects: method.

If you want to implement your own variadic method in Objective-C, here's the basic pattern (you don't need to import stdarg.h if you're importing the Foundation classes):

 - (void) method: (id) arg, ... {
        va_list argList; // special datatype for the argument list
        va_start(argList,arg); // sets up the iteration
       
       id curArg = arg;
       while (curArg != nil) {
             // do something
             curArg = va_arg(argList,id); // pass the datatype you expect to find
       }
       va_end(argList);
}

Here's the code in the context of my Tutorial system:

+ (void) showTutorialForKey: (NSString *)key, ... {
    NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
    va_list argPtr;
    va_start(argPtr,key);
    
    NSString *curArg = key;
    
    while (curArg != nil) {
        if ([TutorialSystem userHasSeenKey:curArg]) {
            curArg = va_arg(argPtr,id);
            continue;
        }
        
        NSString *titleKey = [NSString stringWithFormat:@"%@ Title",curArg];
        
        NSString* title = 
          [[NSBundle mainBundle] localizedStringForKey:titleKey value:@"" table:@"Tutorial"];
        NSString* text = 
          [[NSBundle mainBundle] localizedStringForKey:curArg value:@"" table:@"Tutorial"];
        
        TutorialController *controller = 
          [[TutorialController alloc] initWithTitle:title tutorialText:text];
        [[[self appDelegate] window] addSubview:[controller view]];
        
        // clear the user for all the rest of the args
        while (curArg != nil) {
            [TutorialSystem registerUserSeesKey:curArg];
            curArg = va_arg(argPtr,id);
        }
    }
    
    va_end(argPtr);
    [pool release];
}

* I should note that 1.0 of my app is not yet out, despite being published on the App Store, because of some bureaucracy with my employer. Assuming it's going to get resolved. I've started on 1.1.