Tuesday, April 13, 2010

Ruby + Outlook + Perforce

At my work, we have a practice that I've always found curious. Usually, when people check something in, they send out an email to the whole studio with the change list.

This is, by itself, a fine practice. The curious part to me has always been that Perforce (our version control software) has the ability to send out emails for every check-in. My old boss would subscribe to every check-in in our branch, for instance, so she could keep tabs on what the group was doing. But that's not what we use.

Over time, I've come to understand the rationale of our redundant workflow. People can attach screenshots (I do work for a game company, after all) and the subject line can provide project categorization and a short description that perforce doesn't know about. Also, culturally speaking, if everyone in your company does this and you don't, people have a harder time figuring out what you do.

But I still find the workflow annoying. You submit your changes in Perforce. Then you go to the "submitted changelists" pane and double-click on the changelist you just submitted. You select all the text and then copy it (this view has more info in it — such as the list of files — than the text you wrote when submitting, so you can't just use that). You alt-tab over to Outlook, open a new message, address it to the studio email address, attach your pithy subject, and then copy and paste in the changelist info. (You also use this moment to attach screenshots, if relevant.) You send the email and then alt-tab back over to Perforce (or your IDE) to do more work.

I finally decided to automate the process a bit using Ruby to tie together Outlook and Perforce. I set it up to work with my needs, so it may be of limited use to anyone else. For instance, we have a custom of using "mini" to indicate minor, one-liner types of fixes. Otherwise, I tend to use "submit." Also, sometimes I want to aggregate a few recent changelists in one email. That's what the -count argument does. I also put all the config info (username, password, etc.) into a separate yaml file so that I can distribute the script without sending around my network password. Finally, I specified a sendMode of either Display or Send. Display opens the email for you and lets you customize it. Send just kicks it out the door. The former is useful for screenshots and the like. The sendMode and project config file options provide defaults, but they can be overridden. Sometimes I do work in another functional project and need to change the email accordingly.

You'll need p4Ruby to make this work. (I think the OLE stuff is built in to the Ruby for Windows installation.) Perforce returns information in sort of an odd way: a changelist will have the list of revisions as one field and the list of files as another. The indexes line up, but it takes a bit more work to get the info you want.

There are no doubt better ways to do this: I'm still fumbling around with Ruby.

require 'win32ole'
require 'P4'
require 'yaml'

is_mini = false
subject = ""
num_cls = 1

def assert_value(obj,message_on_fail)
if !obj
puts message_on_fail

# load the yaml settings first
assert_value((File.exists? 'p4_email.yaml'),'You must have a p4_email.yaml file in the same directory')

config = YAML.load_file 'p4_email.yaml'
sendMode = config['sendMode']
project = config['project']

assert_value(config['p4User'],'No p4User specified!')
assert_value(config['p4Password'], 'No p4Password specified!')
assert_value(config['p4Client'], 'No p4Client specified!')
assert_value(config['p4Host'],'No host specified!')

# now parse ARGs. In particular, see if the user has overridden config settings
ARGV.each_index do |index|
if ARGV[index] == '-sendMode'
sendMode = ARGV[index+1]
# check value
assert_value(sendMode == 'Display' || sendMode == 'Send',"Invalid sendMode value: #{sendMode}")

if ARGV[index] == '-project'
project = ARGV[index+1]

if ARGV[index] == '-mini'
is_mini = true

if ARGV[index] == '-subject'
subject = ARGV[index+1]

if ARGV[index] == '-count'
num_cls = ARGV[index+1].to_i

puts "count: #{num_cls}"

assert_value(project,'No project specified! Add to p4_email.yaml or use the -project command-line argument')

# set up p4 connections
p4 = P4.new
p4.client = config['p4Client']
p4.password = config['p4Password']
p4.user = config['p4User']
p4.host= config['p4Host']


# retrieve recent changelists
lists = p4.run_changes('-u',p4.user,'-m',num_cls, '-s','submitted')

#get the id
msg_body = ""
(0...(num_cls)).each_with_index do |obj,index|
cl_num = lists[index]['change']

#get the full details for that cl
cl_full = p4.run_describe(cl_num)[0]
cl_action_list = cl_full['action']
cl_rev_list = cl_full['rev']
msg_body = msg_body + "Change #{cl_num} by #{p4.user}@#{p4.client}\n\n"
msg_body = msg_body + cl_full['desc'] + "\nAffected files ...\n\n"
cl_full['depotFile'].each_index do |index|
msg_body = msg_body + cl_full['depotFile'][index] +\
"##{cl_rev_list[index]} " + cl_action_list[index] + "\n"

#compose email
outlook = WIN32OLE.new('Outlook.Application')

message = outlook.CreateItem(0)
submit_type = "submit"
if is_mini
submit_type = "mini"
message.Subject = "p4 [#{project}] #{submit_type}: #{subject}"
message.Body = msg_body
message.To = '[studioemail]'
# todo: should invoke the method by using reflection
if sendMode == 'Display'
elsif sendMode == 'Send'


  1. At the old job, we did enough automation of Perforce that we wrote a fairly extensive python module to help with this kind of thing, including rearranging the struct-of-arrays data you mentioned above into more friendly forms.

    By the end of it we had command-line scripts to roll back submissions by changelist, to heuristically determine the "owner" of a given file for bug-investigation-responsibility purposes (sum the revision numbers of each owner's checkins - if I did revision #1, #2, #3 and you did rev #4, my score is 6 and yours is 4 and the file belongs to me, but if you check in another change as rev #5, your score goes to 9 and it's yours), to assist with big integrations, etc...

  2. I like the idea of a roll back script. I don't do it often to remember the somewhat convoluted sequence one has to follow.

    And yeah, I have to wonder who designed the struct-of-arrays data type. It's very counterintuitive, though I'm sure there's some reason for it (or was at one, now entrenched, point.) If I write another p4 script of a similar nature, I'll almost certainly move all that parsing into a common file.

    The ownership concept is a neat idea. It seems like that would produce "correct enough" results most of the time.

  3. Hi Derrick,

    Nice to see P4Ruby being used for this.

    The reason for the current output, is that it's a straightforward mapping of the output given by the server: try running 'p4 -ztag describe -s somechange' at the command line and you'll see what I mean.

    For 'p4 filelog' it was so complicated to parse the different levels of index that I wrote something to make it more accessible; something similar for describe might be appropriate. I'll take a look at that for possible inclusion in a future release of P4Ruby.


  4. Tony,
    Thanks for explaining the odd structure. It's certainly counterintuitive, but once you realize what it's doing, it's easy enough to write the code to use it.

    If you were going to put them into a more readily parseable data structure, I'd guess that making depotFile a list of hashes -- {:rev = 2, :action = :edit, :file = blah} -- or something like that would make the most sense, but see above about being new to Ruby in general.

  5. Hi Derrick,

    You're welcome.

    P4Ruby already has some fairly simple classes (P4::DepotFile, P4::Revision, and P4::Integration) which could be used for this purpose and I think I'd rather p4.run_describe() returned an array of those objects because they help encapsulate things in a way that makes it easy for us to enhance things in future.