Learning JMeter

At work recently, I set up automated load testing for some new features I'm working on. There are teams within EA that do large-scale load testing, but I have a need they can't meet. I want to know, every single day, if our performance metrics have changed in a marked way even against a development server. If they have, the change probably came from code that was checked in that day. If performance went down, our team can find the fix quickly or acknowledge the new baseline. If performance went up, we can figure out why and see if the fix can be applied in other places.

I downloaded JMeter and started giving myself a crash course on the software. I've used it in the past, but it was much simpler in those days. So were my requirements.

A simple case was pretty easy: hit a couple URLs that don't require parameters but do return dynamic data. I set up a thread group, the "how many threads should I use" required container, then I added HTTP requests that hit the URLs to that thread group, and then I added a "listener" that parses the responses. That gave me basic familiarity with the tool and allowed me to set up an automatic, nightly run of any checked-in JMeter tests. We use Hudson as a build system, and it has a nice little JMeter plugin that will graph the response times over time and also give you a simple per-run breakdown of performance.

But then I wanted a test suite that would hit a URL, parse the response, and run subsequent tests based on the data (an ID) that came back in the first response. There are plenty of hints that JMeter can do this kind of thing, but it took a lot of muddling to get it right.

I often say that I should contribute to open-source projects not as a professional programmer but as a professional writer. Documentation is often sparse and unclear, even on an established project like JMeter. Where is the "cookbook" section with "you want to do this common thing; here's how" entries? Where is the list of best practices?

You read their manual, you read the FAQ and the wiki, and you still spend a lot of time bumbling about, poking this and prodding that to see if the errors clear up. And I've barely touched the more powerful features.

Here are a few of the things I learned along the way, which I'm posting mostly for my own benefit.

• JMeter can't do anything with a response — even show it to you — without knowing the MIME type. On the one hand, that forced me to return a clean response. On the other hand, I feel like JMeter should fall back to plain text if it has no other information.



• Extract data from a response with a post-processor. If you want to use the response data from one http request in subsequent calls, you make the post-processor a child of that http request. (You can make a post-processor a sibling, in which case it runs after every request in the thread group.) I used a regular expression post-processor that matched the entire response. Even though there was no other data, I still needed to enclose the regex in (). I also needed the $1$ default template, even though I wasn't doing anything with multiple values.



• It seems like every JMeter tutorial suggests that you add a Graph Results listener to your test plan. It shows you a graph of your response time. Ooh. Aah. Pretty. Also? Useless for debugging. I moved through errors at a rapid pace after I added a View Results Tree listener to the thread group. With that listener, you can drill down on the request/response for each and every server call. I also found the Summary Report listener to be more useful than Graph Results. Keep the pretty picture in to show your boss — it doesn't add time — but add the others to make your life easier.



• Learn to love the JMeter functions. They're buried in Chapter 19 of the JMeter manual, but they're essential for making scripts that can be re-used in multiple places. I sprinkled the ${__P(name)} function throughout the text fields of my scripts so that I could fetch command-line properties such as TARGET_SERVER and TARGET_PORT. That means that running my scripts against a different environment will require nothing more than a different set of command-line arguments (prefaced with -J).