Jake Scruggs

Presenting at Lone Star Ruby Conf

2008-06-30T21:35:00.003-05:00

Word on the street is that Lone Star Ruby Conf was one of the best regional conferences last year, so I'm pretty excited to be presenting at the 2008 edition. It's happening Sept 4-6th in Austin, TX and speakers include: Matz, Evan Phoenix, Jim Weirich, The Rails Envy Guys, and too many other luminaries to name.

I'll be giving a talk entitled "Using Metrics to Take a Hard Look at Your Code" which is a distilation of my thinking over the past year about how to use numbers not as a hammer with which to beat people, but as a tool to improve your code.

Registration just opened up today, but space is limited so go sign up soon.

Disconnecting RSpec from the Database

2008-06-17T23:41:00.002-05:00

Recently I spent some time away from RSpec and I did miss it, but one thing especially liked about the testing framework I lived in for 10 months was that it had two ways to test anything you liked: One that hit the database and one that did not. We used UnitRecord to disconnect unit tests from the database (I should mention that we defined unit tests as tests that didn't hit the db and functional tests as those that did). So the vast majority of our tests didn't really need to hit the db and ran very fast. Sometimes we wanted or needed to hit the db and we did (only occasionally in model tests, more often in controller tests, and we kept logic out of the view and only tested views at the Selenium level). It was pretty cool and I'd like to thank the team who set it up before I got there.

There's been a lot of talk about whether model, view, or controller tests should hit the db but I've found that there's enough exceptions that it's worth just saying that you should try not to hit the db when possible for speed's sake and then when the test becomes 20 lines long because of all the mocking/stubbing required, go ahead and hit it. That's all well and good, but how do you disconnect the db from an RSpec example?

I, personally, have had success with NullDB. NullDB is the null object pattern applied to databases. Which is a fancy way of saying that it swallows up any call that would go to the db and returns a nil. So you mock/stub all you like, but if you forget or just don't care about a particular call then don't worry because nothing goes to the db and nil is returned (which, surprisingly, is more than enough in most situations). On the NullDB page I've linked to, the author provides a way to universally disconnect RSpec from the db, but, as I've already said, I'm a bit pragmatic when it comes to disconnection so In my spec_helper.rb I define something like this:

 1 share_as :Disconnected do
 2
 3   before :all do
 4     ActiveRecord::Base.establish_connection(:adapter => :nulldb)
 5   end
 6
 7   after :all do
 8     ActiveRecord::Base.establish_connection(:test)
 9   end
10 end

and then in any "Describe" block where I'd like to disconnect the db I do this:

1 describe User do
2   include Disconnected
3   # 'it' blocks or
4   # more describes
5 end

If I want to hit the db I just don't include Disconnected.

Now since NullDB fails silently I've heard tell of problems with it being hard to debug (although I have yet to experience these problems). Any spec/test/example that just isn't behaving properly can be exiled to the 'hit the database' area. Also I should mention that when I tried to use NullDB with Rails 1.2.6 I got a bunch of errors but it worked fine with 2.0.2. If you are looking for a more vigorous enforcement of 'don't touch the db in unit tests' then UnitRecord may be for you. I thought RSpec and UnitRecord didn't play nice, but my fellow Obtivian, Ryan Platte informs me that he's got it working. So I'm calling him out to post about disconnecting the RSpec with UnitRecord.

Breadcrumbs are Evil

2008-06-12T20:36:00.002-05:00

I've been on a few projects where I've had to mess with breadcrumbs and they tend to be much more trouble than they're worth. It all starts simply enough: put the users path through the app on the page. Which isn't too hard.

Then the bugs come.

Like: "This page is linked to from page A but it really is more of a child of page B so the breadcrumbs should show page B and not A even if they come from A"

Or: "When I come to page X from the search on the front page it should have breadcrumbs with Y and Z because that's really what it's under."

And: "Page L is listed under K but should be under M (followed a few days later by) Page L is listed under M but should be under K"

Oh, you want a hierarchy. Which is hard.

First: There's now a new job to be filled: Taxonomist. Somebody is going to have to figure out where all the pages in the app fit in some sort of tree. And this list must be maintained.

Second: There must be code that manages all this. And since there are always exceptions to the rules of hierarchy it can get quite complicated.

Third: Although your customers don't really care where the page fits in the hierarchy, everyone else in the business does. A lot. So much so that there will be many (perhaps endless) arguments about exactly where things "belong."

All this will take plenty of time that could be used doing something that, you know, your customers may actually need. I guess I really don't have too much problem with "true" breadcrumbs (aside from the fact that I never use them and they're kind of a relic from the days of "Wow the internet is so complicated people will get lost!" and they can probably be replaced with a good search box) but what I really hate is when they act as a gateway drug to hierarchy.

Syntax Highlighting for Ruby Made Very Easy

2008-05-29T17:00:00.000-05:00

So I've been making do with lame-o 'pre' tags for my code snippets and my blog looks, well, lame. But the various alternatives for highlighting ruby either didn't work for Blogger or smacked of effort (and you know how I feel about effort).

Well, recently I joined Obtiva and I was being all nosy looking at my new co-worker's blogs, when I found Tyler Jennings post about creating Spotlight. Take some ruby code, paste it into the text box, press a button, and you get some html which you copy and past into your blog. You'll have to include the default css in your css file the first time, but that's not too hard. Output looks like this:

 1 class Echo < ActionMailer::Base
 2   def copy formatted_message, incident_id
 3     users_with_echo_emails = User.find(:all, 
 4                                   :conditions => "echo_email is not NULL")
 5     users_with_echo_emails.reject! {|user| !user.receive_echo?}
 6     recipients_array = users_with_echo_emails.map { |user| user.echo_email }
 7     recipients_array << ENV['default_echo_email'] if ENV['default_echo_email']
 8 
 9     recipients recipients_array
10     subject "Globex pager incident #{incident_id} updated"
11     from 'noreply@globex.com'
12     body :message => formatted_message, :incident_id => incident_id
13   end
14 end

Cool huh?

Saving in TextMate

2008-05-27T19:22:00.004-05:00

So today, for something like the fifth time, I sat down in front of a TextMate installation that didn't have saving set up the way I like it: Auto save on loss of focus and when running a test (or spec). And, for the fifth time, I had to dig around the internet for the answer. So, as a public service to my future self:

Save files when focus is lost:
http://macromates.com/textmate/manual/saving_files
TextMate --> Preferences --> Advanced --> Saving

Save all files when running a test (or spec):
http://macromates.com/textmate/manual/commands
for specs:
Bundles --> Bundle Editor --> Edit Commands --> Rspec --> Run Single Example
Bundles --> Bundle Editor --> Edit Commands --> Rspec --> Run Examples
for tests:
Bundles --> Bundle Editor --> Edit Commands --> Ruby --> Run
Bundles --> Bundle Editor --> Edit Commands --> Rspec --> Run Focused Unit Test

Change the save drop down to "All Files in Project"

Hope this helps, future Jake.

Metric_fu Now Measures Churn

2008-05-11T21:39:00.004-05:00

Sometimes a class does too much. It's used everywhere, does everything, knows all about the nasty internals of all the other classes, and every time you change anything in the application it has to change. Or maybe you have a class that everyone loves to refactor because it's so bad. One developer changes it to be better, then another changes it to be a different kind of better, and this keeps happening without it ever getting any easier to use. There's a lot of bad reasons why a class might change all the time so I added a churn report capability to metric_fu version 0.6.0.

Now you can use metric_fu to create a report like this:

By issuing this command:

    rake metrics:churn

And use the results to find classes that might have problems. It uses 'svn log' to create this report so currently you need to be using subversion. (If anyone out there want to contribute some code to get it working with git, lemme know.) The default is to start counting changes from the start of the repository's creation. That may be too much for your project so you can change the start by adding these lines to your Rakefile:

    namespace :metrics do
     CHURN_OPTIONS = { :start_date => lambda{3.months.ago} }
   end

The Proc is there because '3.months.ago' only works after the Rails Environment is loaded (and Rails extends Fixnum) which I didn't want to do every time you run a rake task.

You can also change the minimum churn count (files with changes less than this number won't be included in the report) like so:

    namespace :metrics do
     CHURN_OPTIONS = { :minimum_churn_count => 3 }
   end

Also in this release I added the 'metrics:all_with_migrate' task because of some feedback I got that running 'metrics:all' in a cc.rb build fails if there have been migrations. So the new task migrates the test db before running all the metrics tasks.

Check out the metric_fu project home page for more information and installation instructions. Or you can jump right in by doing this in the root of your project:

script/plugin install \
 http://metric-fu.rubyforge.org/svn/tags/CURRENT/metric_fu/
sudo gem install rcov
sudo gem install flog

Dead Simple Rails Metrics with metric_fu

2008-04-27T17:45:00.004-05:00

Every time I create a new rails project I usually put off writing tasks to analyze the code's quality 'cause it takes time and time is, you know, finite. So I've decided to extract some code into a rails plugin which I call metric_fu.

It's a bunch of rake tasks that produce reports on code coverage (using Rcov), cyclomatic complexity (using Saikuro), flog scores (using Flog), and rails stats (using 'rake stats'). It knows if it's being run inside a CruiseControl.rb build and puts the output in the Custom Build Artifacts folder so when you view a build you see this:

The coverage report is your standard rcov report:

Flog output is thrown into an html file:

At the end metric_fu calculates the average flog score per method:

You might want to check out my previous posts on what to do with a Flog report: The Method Hit List and When You Should Ignore Metrics

Saikuro's output is the same as always:

(I changed the warning and error levels for this pic -- more on how I did that later)

And 'rake stats' is always useful:

So how do you get all these reports?
1. install Flog
sudo gem install flog

2. install rcov
sudo gem install rcov

3. install metric_fu
ruby script/plugin install \
http://metric-fu.rubyforge.org/svn/tags/REL_0_5_1/metric_fu/
(in the base of your rails app)

4. rake metrics:all

Which should work fine if you have standard Rails testing and you like my defaults. But what if you use a combination of RSpec and stock Rails testing? Then you can insert this into your Rakefile:


namespace :metrics do
 TEST_PATHS_FOR_RCOV = ['spec/**/*_spec.rb', 'test/**/*_test.rb']
end

The namespace isn't strictly necessary, but I like it for intentional purposes. Multiple paths are useful if, like on my last project, you need to be specific about which tests to run as some tests go after external services (and the people who manage them get cranky if you hammer 'em a lot).

If you also want Rcov to sort by lines of code (loc) and have more aggressive cyclomatic complexity settings then do this:


namespace :metrics do
 TEST_PATHS_FOR_RCOV = ['spec/**/*_spec.rb', 'test/**/*_test.rb']
 RCOV_OPTIONS =  { "--sort" => "loc" }         
 SAIKURO_OPTIONS = { "--warn_cyclo" => "3", "--error_cyclo" => "4" }
end

That's it -- hope you find it useful, lemme know if you find a bug, and check out the project home page at:
http://metric-fu.rubyforge.org

Oh, and thanks to all my co-workers who helped write the original code, in its various forms, that became this plugin.

The Bug that Took Down a Release and How I Wrote It

2008-04-15T22:25:00.001-05:00

We were disposing of some last minute bugs today and I was reminded of the time, a few projects ago, when I introduced a bug that screwed up a whole release.

Let's say that I was writing a pizza ordering web site and, after many successful releases, the Big Wigs at PizzaCo wanted to introduce a new feature that would let customers customize their pizzas even more. In production the user could select style of pizza (thin, pan, stuffed), size, and toppings. There was even some cool logic in place to make sure the toppings were available before being offered. Now, in this brave new world of pizza, the customer would be given the chance to customize each topping. If you chose peppers, you would be given the choice of green, yellow, or red (through the magic of javascript). A choice of onions would prompt the question of red or yellow. Pepperoni? -- regular or spicy?

So we wrote the feature, which was difficult, as we had to totally change how we interacted with the Topping Provider Service (TPS) in order to reserve these new toppings and check for availability. But then the Big Bosses had a thought: "What if this new extra bit of choice scares away the customer?" And their solution was to have "switch" to turn the extra customization on and off. As developers we pointed out that this would be costly as making the new system look like the old, but still work with the new under the covers (by selecting defaults for each topping), was not easy. And it would introduce extra complexity by having two different pages that do almost the same thing. But they wanted it, so we did it.

A few weeks later we found a bug with the topping selection page: If you take the last bit of pepperoni, but later come back to the toppings page to change your selections, the TPS service thinks all the pepperoni is gone (because you just reserved it) so you don't see that option on the page. We fixed it by looking at the customer's saved pizza and putting any toppings missing from the page back on the page. Except that we forgot to make the changes in the page that mimics the old behavior. Even worse, the way we fixed the new toppings page caused the pseudo-old page to blow up if you ever try to go back to it. And we found out that this happens right before a release.

And this was all my fault.

I worked on the feature to add the new customization, I also worked on the feature to hide the new feature, and (believe it or not) I worked on the bug fix. There was no one else on the team who was in a better position to remember that there were two pages and each need a change. Aside from my obvious point that I'm a terrible developer and you should never hire me, I think there's an interesting lesson here about unused functionality.

We wrote the new page and then we used it. It did all sorts of sexy Ajax and was cool. And after a few weeks we forgot about the legacy page and things that might break it. There are many times when it will be tempting to bifurcate your code, but every time you do so you create a place for a bug. This is especially true if one path is rarely used. The new code will continue to evolve and the old will sit around and collect bugs. We did a full pass through regression testing, in addition to our normal tests and QA, but we didn't find this bug until hours before the release. Which is better that finding it after, but still, if the release had gone out it would have had the new feature turned off (for "safety") and the monster bug on. We were mere hours away from disaster.

I know it's hard not to want options, but every option has its price.

Ruby Substrings and Testing Legacy Code

2008-03-31T22:11:00.004-05:00

Recently Josh Cronemeyer and I were working on writing a game in Ruby. Gosu, a 2D game library for Ruby and C++, and Chipmunk, a 2D Physics engine, do lots of the heavy lifting so we thought it would be a fun Saturday afternoon thing to do. However, the examples had no tests so when we tried to change some stuff, and of course it didn't work, we were all sorts of clueless as to why.

Well, wrapping tests around legacy code is not a lot of fun but it is an effective way of debugging. Here's how I like to do it:
Step one:
Identify potentially troublesome code (As in "What the hell is that doing?").
Step two:
Write some tests that verify the functionality of the code.
Step three:
Pull out the offending code into a method or methods.

There's a fair bit of interplay between steps two and three, testing one piece may require extracting it.

Step four:
Either you've found your problem or you've accomplished four things:
1. You really understand the troublesome code.
2. You've refactored it into a more readable version.
3. You've put some code under test.
4. You at least know where the problem isn't.

I adopted this strategy after reading Michael Feathers "Working Effectively with Legacy Code" -- which is good reading.

Eventually we found the problem to be an erroneous substring.
"hello"[0] doesn't return "h"
Ex:


irb(main):001:0> "hello"[0]
=> 104

Which is the character code of 'h', so we tried:


irb(main):002:0> "hello"[0,1]
=> "h"
irb(main):003:0> "hello"[4,5]
=> "o"

Aha! Now we thought that "string"[x,y] gives you the characters between position x and y. But no, we got some really strange behavior until we realized that "string"[x,y] gives you y number of characters starting at x position. The reason "hello"[4,5] gives "o" is because if you ask for 5 things starting at position 4 there's only one character left, which Ruby is only too happy to hand over. "hello"[0,1] is the way to get the first letter, just as "hello"[1,1] is how to get the second letter.

Now one of the things that drew me to Ruby in the first place was it's intuitiveness. Often I found that if I didn't know how something worked, I could just write what I thought would work and it did. But substrings are a rare anomaly -- I find I have to look up how they work every time. And I still make mistakes even after I check the ruby doc. Does anyone know the history behind this weird syntax?

On Inject, Complexity, and George Carlin

2008-03-18T21:33:00.007-05:00

Inject has been stalking me for over 2 years.

The year was 2005 and I had decided that I would write a website in this new fangled Rails thing. But while debugging some plugin code I ran into inject. To my java encrusted eyes, it looked like this:

weird_thing = dont_know.inject({}) {|m, o| m[o] = some_crazy_method o; m}

Holy crap. I had no freakin' clue. I looked up the docs on inject and saw that inject, from what I could understand, was used to add up numbers in an array. Which confused me even more. Of course since then I've come to understand inject, but have remained wary as it can be awful confusing to those new to Ruby. Just recently I wrote about how some teammates and I decided against using it to shorten a method, because the particular use sacrificed readability. Then Brian Dainton wrote about how he loves inject. And finally, my team got in a rather spirited discussion over its use. One side of the argument was that it should be used with care as it's hard to read, especially for Ruby newcomers (whom we will be handing this project over to sooner or later). And the other side pointed out, rightfully so, that the reason we use Ruby in the first place is precisely because of powerful methods like inject. If we have to stay away from inject, or method_missing, or metaprograming, or anything else that is deemed 'too hard' then why not just write in Java or C# or Visual Basic?

All this reminded me of one my favorite quotes form George Carlin:

Have you ever noticed that anybody driving slower than you is an idiot, and anyone going faster than you is a maniac?

The funny thing is that I can see both sides of this argument. Back in 2005 I would have thought you crazy to use such a method. Now it would take only a moment to figure out that in:

weird_thing = dont_know.inject({}) {|m, o| m[o] = some_crazy_method o; m}

dont_know is probably an array full of objects and the author of the code wants to pass each one of them into some_crazy_method, get the result, and then put that result in a hash with the original value as the key. When I was a slow driver in Ruby I thought all the other speed daemons were crazy to write such complex code. Now that I'm a budding speed racer myself, it's temping to see people who can't understand my code as stupid.

At this point it would be pretty easy to wrap up this post by saying something useless like: “Use your best judgment when adding complexity, making sure to balance blah, blah, blah...” You, like everyone else on the planet, think you know the difference between too slow (code for dummies) and too fast (code for computers only). Even if you're only wrong 10% of the time (which I doubt) you're going to write a lot of horrible code.

So how do you know when you're wrong (or right)? You don't. Let me try to explain though two examples of complexity: one good and one bad.

In our application, we have a number of different ways to go through a sign up process. So we created an object called Flow. And each flow contains a number of FlowPages that specify the order of the flow. With a little bit of Ruby magic, we made it so you can define a flow like this:

NormalFlow << FirstPage << SecondPage << ThirdPage

Which is damn cool. But there was a bunch of code needed to support this. Over time we came to realize that we weren't seeing a lot of benefit from this notation. We didn't define flows that often and even if we did, the much simpler:

NormalFlow.new(“FirstPage, SecondPage, ThirdPage”)

expresses the intent just as well. When the time came to play a story dealing with putting flows into the database so the business can try different paths, we saw that the complexity was actually making it harder to use and so we threw out the old code and went with the totally unsexy “pass in a string” constructor. Lame, but way easier to maintain.

In the other example, we had a requirement that the business should be able to change any of the words on the page quickly (without a release). This, of course, meant putting the page content into the database. We created a PageContent object that, through the magic of Ruby, gets created before any view is rendered, looks up all the content that it might need, and dynamically defines methods on itself that correspond to its contents. A given page may have a title or an address_line_one_label and to access these methods in the view you can call: content.title or content. address_line_one_label and you don't have to do a thing to set this up (other that put content in the db) -- it's available in every view. And over the months since we implemented this feature it's been easy to use, extend, and maintain.

And that's really my point here. I can't just say “use your best judgment” because you already are. Furthermore, when you've just learned about, say, metaprograming you're probably going to use it more than you should. You won't have any idea if that super cool bit of Ruby is good or bad complexity until you get back in there and try to make it do something new. If the complexity fights you at every turn, it's bad and you should get rid of it. Good code you won't even notice because the tasks that touch it go so smoothly.

A Small Rails Site and Why You Should Build One

2008-03-11T21:45:00.013-05:00

The other day I was showing Rails off to a friend of mine who wanted to know what all the fuss was about and I realized a shocking thing: I was horrible at starting a Rails site. Why? Because for the last year or so I've been working on large, established Rails codebases. Which is kinda cool as it shows how far Rails has come, but it sucks because one of the very cool things about Rails is how much you can do in a short amount of time.
So I resolved to create and deploy a small site as a way to exercise some atrophied muscles and have some good old fun with Ruby and Rails. My father, Leslie Scruggs, is a sculptor and I had written him a pure html site back in 1999. And he maintained it editing the raw html files, creating his own thumbnails, and uploading via ftp. Not the easiest of tasks for a man in his 60's. So I created lesliescruggs.com which has an admin section so he can upload pics and get them resized and thumbnailed with 10% of the effort.
Of course I used attachment_fu for image handling (resizing, saving, deleting, etc.) but I was impressed how much better it's gotten in the last year. You can now use ImageScience or MiniMagick as the back end image processor and you can save files to Amazon's S3. And, maybe it's just me, but it seems like it's easier to use than ever. Restful_authentication was also impressive in its gains. I've had mixed results with Rails plugins for authentication in the past, but everything went very smoothly. One of the things I really liked was the ability to omit the action mailer part of the plugin. Sometimes you just don't need to be able to email lost username/passwords and it's nice to be able to leave that code out. Another nice surprise was after upgrading to the latest IntelliJ I found that creating a project with RSpec was an option. Worked well, too.
And page caching, something I've been meaning to try, was pretty easy to implement and perfect for a site like this. All in all I highly recommend banging together a quick (this one took about 2 weeks of spare time) Rails website as you'll have fun and learn a bunch of things your day job won't teach you. Especially about deployment -- but that's another post.

ActiveSupport::CoreExtensions::String::Inflections underscore

2008-02-21T06:26:00.003-06:00


>> "ActiveSupport::CoreExtensions::String::Inflections".underscore
=> "active_support/core_extensions/string/inflections"

>> "ActiveSupport::CoreExtensions::String::Inflections".underscore.camelize
=> "ActiveSupport::CoreExtensions::String::Inflections"

Looks like Rails extension to the string class does more than underscore a string. Which is useful, but shouldn't the name be "relative_path" or something similar? When I see the method name "underscore" I know exactly what it means, except that's not what it does. The opposite of "underscore" is "camelize " which, again, may not have the best name.

And if you want to find the declared constant specified in a string:

>> "ActiveSupport::CoreExtensions::String::Inflections".constantize
=> ActiveSupport::CoreExtensions::String::Inflections

Useful and an excellent name.

Return to Generating a Unique Number

2008-02-19T23:09:00.003-06:00

In a previous post I discussed generating a unique id number for an order in our database. Read the original post for the full story, but to sum up, I decided that playing around with rand(100_000_000) was good enough. Martin Ankerl posted a comment pointing out that I was vulnerable to the Birthday Problem. So after a quick trip to wikipedia, I found out that if you have 23 people in a room you might think you have a 23/365 chance of having 2 people with the same birthdays, but no, your chances are closer to 50%! This is because in a group of 23 people there are 253 pairs each of which have a 1/365ish chance to have the same birthday. The formula to calculate how many are needed to give you a 50% chance of having a birthday collision is:

.5 + sqrt(.25 + 2 * 365 * ln(2) ) = 22.9999

Which means the 50% chance of collision mark for 100,000,000 is:
.5 + sqrt(.25 + 2 * 100,000,000 * ln(2) ) = 11774.600235771

Holy crap! I'm screwed if I can only have 12,000 orders before I get a 50% chance of collision. But wait, I think I'm ok because every time I insert a key into the database I check to make sure it isn't already in there. This means that when I'm at 12,000 orders the next key has only a 1 / (100,000,000 - 12,000) chance of being a collision. I think I've changed from the "Birthday problem" to the "Same birthday as you" problem, which is way more intuitive. If I'm wrong I rely on you, the blogoshpere, to tell me so.

If you haven't already, go read the wikipedia article on the Birthday problem. It's pretty cool.

When You Should Ignore Metrics

2008-01-23T06:44:00.000-06:00

Our team has been doing a bang up job of reducing our complexity through our hit list (All of our methods are ranked by Flog score and we spend some part of every iteration picking the worst methods and trying to refactor them.). But sometimes we run into a situation where we prefer a high Flog score to a low one. For instance, this bit 'o code:


def is_something?(x, y)
 x.foo == y.foo &&
 x.bar == y.bar &&
 x.blah == y.blah &&
 # and 10 more lines of the same
end

which had a Flog score in the 60s. We could have changed it to this:


def is_something?(x, y)
 ["foo", "bar", "blah", # and so on ].inject(true) do | a, b |
   a && x.send(b) == y.send(b)
 end
end

Which gave a us a much lower score. However that first method is crazy simple. Merely glance at it and you know what it's doing. The 'less complex' method? Even an experienced ruby dev would need a moment or two. As for a new dev... I can remember a time not so long ago when I found an inject in some source code and was pretty confused even after I looked up the Rdoc. It's one of those methods that's so powerful and flexible that the documentation only seems to make sense after you understand it.

After some discussion with the team we decided that while Flog is very good at telling you which methods to inspect for badness, it is not always true that a better score makes a for better method. The same can be said for all metrics, really.

Automated Javascript Rails Testing

2008-01-21T23:08:00.000-06:00

On my last project we did some javascript unit testing in browser using the unittest.js library from scriptaculous, but because it's kind of a pain to set up and integrate into your build, I didn't get it working on my current job. Which makes me a bad person. However, Dr Nic Williams has made yours and my life easier with his javascript_test plugin for rails. Now I can type:


ruby script/generate javascript_test fancy_javascript_file

And get a fancy_javascript_file_test.html to test my fancy_javascript_file.js. It's all set up to go right at the javascript file. Just open the html file in a browser and it runs the tests. The sweet doctor explains it all better than me so go check out his site.

What I want to talk about is trying to integrate this into your Cruise Control build. The first problem we faced is that the tasks that come with the plugin don't close the tabs created in the browser. So after 10-20 builds, you're going to have way too many tabs and some memory problems. My teammate Toby Tripp came up with this:


  def teardown
    applescript <<-EOS if macos?
      tell application "System Events"
        tell process "Firefox"
          set frontmost to true
          click menu item "Close Tab" of menu "File" of menu bar 1
        end tell
      end tell
    EOS
  end

This code snippet goes inside


class FirefoxBrowser < Browser

inside the plugins/javascript_test/lib/javascript_test.rb file. We could have just killed Firefox but we needed to play nice in case any other builds were using Firefox at that moment (we test with Selenium to use Firefox to click through the app).

Now all you have to do is change the position of the browser.teardown call in that same file in the 'define' method. By default, teardown is called just before the end of the block created by


@browsers.each do |browser|

But I want it to get called after every test so you can put it just before the end of the block created by:


@tests.each do |test|

So that the end result looks like this:


  def define
    trap("INT") { @server.shutdown }
    t = Thread.new { @server.start }

    # run all combinations of browsers and tests
    @browsers.each do |browser|
      if browser.supported?
        browser.setup
        @tests.each do |test|
          browser.visit(" http://localhost:4711#{test}?resultsURL=http://localhost:4711/results&t=" + ("%.6f" % Time.now.to_f))
          result = @queue.pop
          puts "#{test} on #{browser}: #{result}"
          @result = false unless result == 'SUCCESS'
          browser.teardown
        end
      else
        puts "Skipping #{browser}, not supported on this OS"
      end
      # browser.teardown
    end
     @server.shutdown
     t.join
   end

You can see by the comment where the teardown used to be. So now when you run your test the tabs will close when finished. In my next post I'll talk about how to fail the build when the javascript tests fail.

Oh, and I'd like to thank Mike Ward for turning me on to javascript unit testing in the first place.

Generating a Unique Number

2008-01-16T22:35:00.001-06:00

Weird requirement at work today: We needed a number that was alpha-numeric, exactly 11 digits, unique, and non-sequential. At first we thought of using a hash, but MD5 and Sha1 give you way too many digits. We could truncate to 11, but not knowing much about hashing, that made me pretty nervous that we'd have collisions. After a bunch of discussion, we decided on this:


MAXIMUM_FOR_10_DIGITS = 0xffffffffff
(MAXIMUM_FOR_10_DIGITS + cart_id + Time.now.to_i + rand(100_000_000)).to_s(16)

I was thinking about using hex, but I was worried about what would happen if the number got too big and went to 12 digits. So I did this calculation:


irb(main):001:0> 0xfffffffffff - 0x10000000000
=> 16492674416639

Turns out there's all sorts of room between the lowest and highest 11 digit hex number. Paul and Schubert did some quick calculations assuming lots of carts per day and figured out that we have about 250,000 years before we run out of space. Now since we use a random number, there is a chance that we could get two matching numbers so we do a database lookup to make sure that a generated number hasn't been used before. There is still a small chance that two identical numbers could be generated in the time between generation and saving to the database, but I think we can live with that.

Update: After some interesting comments, I wrote a follow up post. Check it out for more math goodness.

The Method Hit List

2008-01-09T20:01:00.000-06:00

In a previous post I talked about how my team has a daily metrics build that reports on our code quality. One of the things we measure is the Flog score of our methods (Flog is a ruby program that evaluates the complexity of your methods). We have a standing developer task to look at the top five worst methods in our application and reduce their complexity. Every iteration we try to spend some time looking at a few developer tasks, which are things that don't provide any direct business value but do make our code better, automate some manual process, or fix some technical problem not tied to a bug.

Yesterday Chirdeep and I refactored the worst method in our app and while it wasn't easy it produced a number of benefits:

First, about halfway though the refactoring we came upon some twisted logic that seemed as if it could never evaluate to false. After pulling in Ali for some consultation, we realized that indeed it would always be true and therefore was a bug. We fixed it and moved on.

Second, we noticed that the object inherited from another object, but overwrote the constructor with a strange one that took different arguments. This had the effect of making other objects interact weirdly with this class in order to deal with its 'sorta' inheritance. We managed to get rid of the custom constructor and have it rely on it's super class.

Finally, we noticed a nice clear division of labor in the complex method so we were able to extract a method that had a single responsibility. So we ended up with two methods that shared the complexity of the original one.

I would have been happy if all we accomplished was the last item. However, when you take a look at your most complex methods you often find bad stuff hiding amongst the craziness.

It's also important to note what we didn't do:
We did not simply extract out every 5 lines into a method. While that would have improved our Flog score per method, it would been the code equivalent of sweeping dirt under the rug. Methods should, as much as possible, have a single responsibility. If they don't then all you're doing is moving a bunch of complex stuff into many different arbitrary places, instead of just one place. Which is actually worse. So, like all metrics, complexity is something to pay attention to but you don't want to slavishly chase after good numbers.

Svn Merge from Trunk to Branch

2008-01-07T21:21:00.000-06:00

Every time I need to do this I try to use svn's --help which is only possible to decipher if you already know what to do. Then I spend too much time on the internet looking for the answer. And finally I break down and ask my tech lead. With much shame. I'm putting this up here so I least I'll know where to find it. Maybe you too.

In the directory of the branch:


svn merge -r 1001:1002 https://svn.dev.your/repo/trunk/src .

If you checked in some files in revision 1002, then what you're saying is that you only want the changes from that checkin with '-r 1001:1002' Which is followed by the url of the trunk (where the changes were checked in) and a '.' to say that you want to merge to the current directory. Use '--dry-run' if you want to see what would happen without actually screwing anything up. After the merge is successful, you now have some modified files in the branch you can check in.

Done and done.

Mocking Backticks and Other Kernel Methods

2007-11-14T21:20:00.000-06:00

There's a bunch of places in our build where we need to execute a system command (such as running sqlplus) but often times we've found that the command fails and our build happily churns away. It's pretty easy to check the result of system command with $?.success? but we have to remember to do that everywhere... So that means it's time to extract a method. Here's what Kurtis and I came up with:


module DatabaseHelper
 def self.command_runner(command)
   output = `#{command}`
   puts output
   fail unless $?.success?
   fail if output.include? "ERROR"
   output
 end
end

So you pass in a command as a string and the command runner:

runs the command
prints out the output
fails if the return code is bad
fails if the output includes "ERROR" (useful when running database imports with sqlplus)
and returns the output just to be a good citizen

So that's cool but when I started to write the module I thought "Hey, I can test this." Which is a cool side effect of extracting methods from your rake file. So cool that you probably want to consider it even if the method won't be re-used.

Anyway, I started to write the test and realized that I would need to mock the backtick, which is a Kernel method. I'm using mocha so I tried:


Kernel.expects(:`)

and


Kernel.any_instance.expects(:`)

But no luck. Turns out you need to mock the class in which the backtick method will be called. So here's the tests I ended up with:


unit_tests do

 test "command runner executes a command" do
   DatabaseHelper.expects(:`).returns("some output")
   $?.expects(:success?).returns(true)
 
   output = DatabaseHelper.command_runner("command")
   assert_equal "some output", output
 end

 test "command runner fails if ERROR in output" do
   DatabaseHelper.expects(:`).returns("ERROR: this is broken")
   $?.expects(:success?).returns(true)
 
   assert_raises(RuntimeError) do
     DatabaseHelper.command_runner("command")
   end
 end

 test "command runner fails if process fails" do
   DatabaseHelper.expects(:`)
   $?.expects(:success?).returns(false)
 
   assert_raises(RuntimeError) do
     DatabaseHelper.command_runner("command")
   end
 end
end

I was having such a good time mocking that I later went back and mocked out the puts so I didn't have to see the output when running the tests. Gotta keep things clean.

Btw, you may notice that I'm not using Rspec a work anymore. Yep, it's true. My new team is using DUST (well, the code that would later be part of the inspiration for DUST) which is pretty cool. I still prefer Rspec, but I can't really make the case that we should switch to Rspec mid-stream (if we were using straight Rails testing, I would make that case.).

Employee Appreciation

2007-10-24T20:21:00.001-05:00

As a consultant, I spend a lot of time at big companies and big companies generally have a morale problem. Bureaucracy, cubicles, and lax management will tend to do that. Typically they try to solve this problem by exchanging money for goodwill. A few projects ago I was consulting at one such company where they spent a ton of money having a day of games, burgers, and events during business hours. There was a big ad campaign, lots of promotional tie-ins and give-aways -- a team of people clearly worked on this for weeks (or maybe months). Now add up productivity, food, and sumo-suit rental costs and we are talking some serious money. All worth it in service of employee good will, right?

A few days after this spectacle, my team and I decided to walk to lunch. Now the most direct route to our restaurant of choice happened to be through the front door. However, the security guy stopped us when we tried to leave. Employees aren't allowed to enter or leave through the front door. Why? That's the policy.

So we backtracked, went out another door, had lunch, and came back. At which point we discovered that our badges wouldn't let us in the side doors. So we tried another side door. No luck. We went around to the front where we found out that security had turned off our badges intentionally so that we would have to go in the front door and get a lecture from the head security guy. I'm not kidding. 3 grown men got a talking to for trying to exit through the front door.

Later we heard, informally, that the company wants to keep the front entrance looking nice so that's why employees may not enter or leave through the front door. The obvious implication here is that visitors would be put off by the sight of employees walking in through the same door as them. Does anyone really believe that one day of corporate sponsored fun is likely to make up for the daily humiliation of having to enter through the back door of your own company?

Metrics for Rails

2007-09-13T23:31:00.000-05:00

Everyone thinks they write good code -- it's just part of human nature. You can't do something every day and not secretly suspect that you're good at it. Self-delusion is a powerful thing so you need to use metrics to take a hard look at your code.

On my current project, we've just added a daily metrics build (run every day at midnight by CruiseControl.rb) that takes a look at our code in three ways:

Code coverage with Rcov
Cyclomatic complexity with Saikuro
And um..., Flogging with Flog

Rcov is a code coverage tool that can be used with Rails Rcov to add a bunch of rake tasks to your build so you can figure out which lines of code are run by your tests... and which are not.

Saikuro computes cyclomatic complexity which "measures the number of linearly independent paths through a program's source code." Methods with more paths are harder to understand/debug/modify.

And Flog is cyclomatic complexity with an attitude. It scores ruby methods with an "ABC metric: Assignments, Branches, Calls, with particular attention placed on calls."

Why do we use both Saikuro and Flog? Well Flog catches Ruby specific complexities that cyclomatic complexity doesn't (for instance, calls to eval are given particular weight) and it picks up methods that Saikuro misses (we use metaprogramming to define a fair amount of methods and Saikuro seems to miss anything not defined with a 'def'). But Flog outputs a flog score which isn't very familiar to most developers, while cyclomatic complexity is a relatively well understood computer science term.

Also flog has the scariest looking software website ever. I can't believe those guys have the chutzpa to put such pictures on a site that advertises them as consultants for hire.

So how did our code look? Pretty good -- 96% code coverage overall but some methods need testing love. And some of our code had pretty high cyclomatic complexity or Flog numbers so we'll need to write some developer tasks to fix the problems uncovered. (Every iteration we work on a few developer tasks in addition to the development, bug, and production tasks.)

But the metrics build isn't done, in the future I'd like to:

Find duplicate code with CPD.
Use Heckle to see if our tests are any good. (Heckle changes your code (not permanently) and then re-runs your tests -- if they don't fail then you've got bad tests)
Fail the build on bad metrics numbers.
Figure out how to integrate the results Panopticode-style with all the cool visualizations it offers.

Why 50% Test Coverage Seems More Painfull Than No Test Coverage

2007-08-30T22:02:00.000-05:00

Recently I was on a project where a bunch of code had been written before we arrived. It was quite a struggle to get the application under test. After a number of months the team hit 50% and then we just stayed there. We had a hard time getting client developer buy-in on the push upward from 50%. I didn't really understand this attitude at first, but after talking with the devs, I realized that the tests were mostly a nuisance for them. They saw it like this:

"If I have to gut a few pages, as part of a changing requirement, now I also have to spend a day fixing the stupid tests. And the tests never really catch any bugs, so what was the point? All the tests are doing is slowing me down."

Since the coverage was low and many of the test writers were new to unit tests we didn't really have a lot of protection from bugs. But we also had a sizable suite to maintain. They were feeling all the pain of keeping a test suite running but seeing none of the benefits. Now, as a guy who'd been on projects with good test coverage I could see how the suite was making things better: First, I had a roughly 50% chance of being able to use the tests as documentation for any class I opened up. And second, the tests were catching some problems before they made it into the repository -- but not enough for the client devs to notice.

We never did convince them that writing tests allowed them to go faster in the long run and it caused me to wonder how many teams get to 30-60% coverage and give up because the benefits are too subtle and the pain is too much?

Renting American Cars

2007-08-29T20:59:00.000-05:00

As a traveling consultant I've developed a pretty poor opinion of American cars. Sliding doors that won't close, unfathomable glove boxes, ugly/boring interiors, boring/ugly exteriors, crazy placements for the radio/ac/heating/trunk release buttons, plastic doors with tails of extra material hanging off poorly drilled holes... The other day I was sitting in a Chrysler 300 talking with one of our tech leads about how he really wishes the rental company hadn't upgraded him and would've much preferred a Hyundai Elantra -- a car that costs half as much as the 300. It's a bit depressing.

Writing Your Own Custom Log Parser

2007-08-28T23:01:00.000-05:00

I spent the last week writing a log parser of all things. The web site I'm currently working on is the sign-up page for a large company so they are very concerned about how far the average joe gets into the process before they give up. If everybody bails on the address page, then perhaps it's too complicated or unresponsive or... something. Up till now we've been redesigning pages based on some data and a lot of guesswork so we thought it might be about time to get some serious data on every session every day. Now the first thing you learn about parsing rails logs is that they are designed to be human readable but not so much computer parsable. An Apache request is one line. A Rails request log has lots more info, but is multi-line. And since we run a lot of Mongrel instances at once, all these requests are interleaved, so sorting them out is an intense computation (we actually ran out of memory a few times before we did some optimization).

Our first pass was mostly a ruby script that called off to a few classes which produced a csv file that our analysts could play with. But they wanted more data -- data only Apache could provide. So we added an Apache log parser and merged the results. And there were bugs (stuff wasn't logged, sessions seemed to end at the starting page or somehow start in the middle (both of which are impossible)). And the code was complicated and hard to understand. At this point we realized that we had been treating this task like a second class citizen -- a task to be automated but not really cared about. So we bit the bullet and refactored the log parser to be more Object Oriented. Now there's a log collector that has lots of requests. Each request mostly delegates to its Apache Request or its Rails request. Along the way things became more clear and the bugs started to become obvious, including one I have to share with you:

Remember how I said many sessions seemed to start in the middle while others ended at the beginning? Well, it turns out that a typical telemarketer will sign up one user and then head back to the first page to sign up another. At which point Rails grabs the session id of their previous session and logs it before processing the request (which wipes the session, but too late for the logging). So it looks like the previous session ended at the beginning. Of course, the next session has had it's beginning stolen so it looks like it started in the middle. Since Rails does this bit of logging before it even looks at our code we needed to adjust our logger to ignore the false session and look for the real session id in a extra bit of logging we added in.

Anyway now that we have our own custom log parser we can get some awesome info on our app for debugging weird stuff that only happens under load and the analysts get a csv file then can import into Excel and generate all sorts of cool graphs and statistics. Yes it did mean a week of writing regular expressions and some crazy command line statements (like: cat access.log | grep | awk | sort | uniq -c | sort ) but we now have a very sharp, well tested, tool.

RSpec on Rails: Models

2007-08-09T23:58:00.000-05:00

In my last post I talked about using RSpec without Rails, but since just about all my Ruby programming involves Rails, I should probably get into how to specify Rails code.

RSpec on Rails: Models

So if you're new to RSpec and you want to get started quickly, head on over to rspec.rubyforge.org, get RSpec, Spec::Rails and then, after you've created a new Rails project and rspec-ed it (all detailed on the website) you can run:


ruby script/generate rspec_scaffold person name:string
        phone_number:integer
        cash:decimal

Which will get you a bunch of stuff to play with. Of you'll need to set up your database.yml and run rake db:migrate if you want thing to work.

Fire up your favorite IDE and find the person_spec.rb and you should see something like:


require File.dirname(__FILE__) + '/../spec_helper'

describe Person do
 before(:each) do
   @person = Person.new
 end

 it "should be valid" do
   @person.should be_valid
 end
end

Not the world's most rigorous set of specs, but then our model doesn't have much behavior at the moment. You'll notice that the auto-generated spec does use RSpec's built in interrogative feature. 'be_valid' makes RSpec look for 'valid?' Running the spec with spec formatting ('-fs') produces the following output:


Person
- should be valid

Finished in 0.028878 seconds

1 example, 0 failures

But what's interesting is that if I remove the string between it and do, the output remains the same. If you leave off the name of a spec, RSpec guesses a name for you by looking at what you're asserting.

But let's say I want to specify that a Person is invalid without a name. Here's how I would do that:


 it "should be invalid without a name" do
   @person.should have(1).error_on(:name)
   @person.should_not be_valid
 end

As you can see, there's some built in sugar to make checking errors more readable. Now that I have a failing spec I can add the following line to my Person model to make the it pass:


 validates_presence_of :name

Now let's do something a little more interesting:


 it "should run inside a transaction when making rich" do
   Person.should_receive(:transaction).and_yield
   @person.make_rich
   @person.cash.should == 1000000.00
 end

What I'm saying here is that a Person should have a method that makes them rich and furthermore that the rich-making should happen inside a transaction. RSpec has mature mocking/stubbing framework built into it. 'and_yield' can even specify what to yield so you could return an object that you created in the spec and then check to make sure some things happened to it. Of course, before I get to much further, I should write some code to make the spec pass:


 def make_rich
   Person.transaction do
     self.cash = 1000000.00
   end
 end

Now what if you already have a codebase that has a lot of Test::Unit tests but you want to start using rspec? Well you could just start writing specs in their own spec folder and rspec's default rails task will run both your tests and your specs. But that's not your only option. You can use RSpec inside your tests like so:


require File.dirname(__FILE__) + '/../../test_helper'
require 'spec/test_case_adapter'

class PersonTest < person =" Person.new(:name"> "Jake")
 end

 def test_person_can_have_name_set
   @person.name.should == "Jake"
 end
end

Now if you do this you won't have access to all the feature of RSpec: The reports, the mocking frameworks, and some custom matchers won't work (for instance '@person.should have(1).error_on(:name)' isn't available) but it's a low impact way of trying out RSpec before you commit.

Next time we'll look at view specing with RSpec.