I recently setup a Jenkins CI server for a client and put their Rails app in it. When someone committed a change, it ran a CI build. However, when someone committed a migration, rake would abort with the dreaded "pending migrations" error. So I just added a db:migrate step to the ci:build task, and all was well…

…or so I thought. In this particular case, the CI server builds anytime someone pushes code to either the "develop" or "master" branch (using git). Pushes to develop happen much more often than master, because master is the "stable production release branch" and develop is the "push everything here when it's ready to be shared with the rest of the team and integrated into the bleeding edge version of the app" branch. One consequence of this is that master builds will often assume an older database schema than develop builds because those migrations haven't been merged with the master branch yet.

So running "rake db:migrate" in the ci:build task broke. The migrate task doesn't understand moving the schema backwards and forwards unless you explicitly tell it what to do. However, there's a much better way to do this. The file db/schema.rb is a current snapshot of your database and it should be checked into your code repo. So if you just load that into the test DB before each CI build (via the handy "rake db:schema:load" task), then you're golden. Master has its snapshot, and develop has its snapshot. Everything's fixed! Er...

Here's the wrinkle: When you run rake spec after doing that (within the same rake ci:build über-task), it executes the following tasks:

** Invoke spec (first_time)
** Invoke db:test:prepare (first_time)
** Invoke db:abort_if_pending_migrations (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:abort_if_pending_migrations
** Execute db:test:prepare
** Invoke db:test:load (first_time)
** Invoke db:test:purge (first_time)
** Invoke environment** Execute db:test:purge
** Execute db:test:load
** Invoke db:schema:load
** Invoke environment** Execute spec

So what? It runs db:schema:load again, no big deal. Nope. Chuck Testa. If you look closely, you'll see that while db:schema:load is *invoked*, it is never actually *executed*. And that's because rake remembers that it has already been executed when we ran it manually before. So the last thing that's been done to your test database is db:test:purge. If you look at your test database at this point, you'll see that is has no tables. And your tests will fail (at least they better).

Why are there no tables? Why didn't the manual db:schema:load put all the tables into the test database? Because it put them into the development database. Remember that most tasks default to the development environment when you don't specify one. So we loaded the DB schema snapshot into the development database, and then later when rake is asked to run db:schema:load on the test database, it doesn't run it.

So what's the solution? We need to tell rake to re-run db:schema:load. (Don't be tempted here to force Rails.env = 'test' and use the test database for everything. This comes with its own issues and you'll be fighting against the way rake spec is designed to work.)

Here's what my final rake ci:build task looks like:

namespace :ci do
  desc 'Setup CI environment'
  task :env do
    ENV['CI_BUILD'] = 'true'
    ENV['RSPEC_FORMAT'] = 'progress'
    ENV['CUCUMBER_FORMAT'] = 'progress'
    ENV.delete('http_proxy') # need cukes to be able to connect to localhost
  end

  desc 'Run a continuous integration build'
  task :build => ['ci:env'] do
    Rake::Task['db:reset'].invoke # drops, re-creates, loads schema.rb
    Rake::Task['db:schema:load'].reenable # so it can be run later on the test db
    Rake::Task['spec'].invoke
    Rake::Task['cucumber'].invoke
  end
end

That line "Rake::Task['db:schema:load'].reenable" is the key. Once you add that, rake spec will happily resume loading up your test DB's schema and you'll be off to the races.

Permalink | Leave a comment  »

In the world of object-oriented software design, we talk about things we're modeling as falling into 1 of 2 categories: Entities and Value Objects. Entities have identity separate from their constituent attributes, while Value Objects do not. So I, for example, am an Entity, but the color magenta is a Value Object. Even if you list my attributes (first name: Wes, last name: Morgan, hair color: blond, eye color: hazel, height: 5'11"), that doesn't fully encompass who I am because there is probably at least one other guy out there matching that same description. Freaky Friday. But if you were trying to find me, that other guy just won't do. Magenta, however, is a color that can be represented by 3 red, green, and blue values, for example. Magenta's magenta. This is nice because you can create an immutable, throw-away magenta instance of a Color class to represent it in your code. Decide you like chartreuse better? Throw away your magenta instance and instantiate a new Color object for chartreuse. This has several handy side-effects for your code. Entities require much more care and feeding. I should know, I am one. And so is that jerk imposter whom I will crush.

Anyway! REST works very well on Entities. You have a URL like http://server.org/people/34 and you call various HTTP methods on person #34 (GET, PUT, POST, DELETE). How rude of you. But it all works. Wam, bam, thank ya ma'am.

But what about Value Objects? If you wanted to write an RGB to HTML hex value web service that converted colors from red, green, and blue values to their equivalent HTML hex value, how would make that RESTful? What's the resource? What operations do you map the HTTP verbs to? Here's a real-world example I ran into this week that highlights the conundrum and how I solved it:

I have a piece of software that converts ZIP codes into City, State pairs. This is how we in the U.S. of A. do postal codes. It can also take full street addresses and sanitize them and find out their 9-digit ZIP code. For those outside the U.S., the 9-digit ZIP code is extra special because it can tell you exactly where that address is geographically, but almost no one knows the final four digits of their code. So having software that can derive that code from their street address (which they do know) is super handy. My mission was to wrap this software in a Ruby on Rails layer to create RESTful web services out of these functions.

Step 1. ZIP -> City, State : ZIP codes are a great example of a Value Object. The ZIP code of my office in downtown Denver, CO is 80202. 80202 is the same as 80202, even though they're at different places on this page. You don't care which one I typed first, they're interchangeable. So it would be silly to create a RESTful web service that did something like this: GET /zips/89 which would return the ZIP code w/ id 89 and the associated city and state. Let's say it's 73120 in Oklahoma City, OK. Why assign ZIP codes an id number? Why not just do this: GET /zips/73120 and have that return Oklahoma City, OK? So that's what I did. The value *is* the id. That works great, moving on.

Step 2. Address -> Better Address (incl. the magical 9-digit ZIP code) : Now things get trickier. An address is still a value object. 123 Main St., Missoula, MT is the same as 123 Main St., Missoula, MT. We don't need to assign id's to these objects, they're just the sum of their parts. But unlike ZIP codes, more than one attribute is required before you can say you have an Address object. You need 1-2 lines of house number, street, apartment number, etc. information, then a city, state, and 5-9 digit ZIP code. That's a bit harder to put into a URL. Ideally we want to be able to label the different attributes so we can easily keep track of what's what. Something like this:
GET /addresses/123%20Main%20St.%20Missoula,%20MT%2059801 ...could work, but it's kind of a hassle because you then have to write a parsing routine on the server side. It would be nicer if you could just label the components of the address when you pass it in. Query parameters to the rescue! Here's what I did:
GET /addresses/1?addr1=123%20Main%20St.&addr2=&city=Missoula&state=MT&zip=59801
Now I can very easily pass this address into my sanitizer and get back the corrected version w/ the 9-digit ZIP. The "1" in the URL is a pseudo-id placeholder. If it bothers you, you could just as easily put "address" or "thisone" or "foo" there. But don't put "get" or "lookup" or any other verb there! That's not RESTful. Only the HTTP verb should define the action being taken on the resource.

So yeah, that's how I did it. I think it's still pretty RESTful. It only uses GET, but that's all you really need for read-only Value Objects. It would probably be better to do something like GET /zips/90210/city or GET /zips/80218/state, or maybe even GET /zips/73120/city_state to avoid the double call in the common case of wanting the city and state. I still don't really know of a cleaner way to do the address sanitizer piece. It works, but it feels a bit hackish.

I would like it if smarter types chimed in and critiqued my approach. Because this is all just, like, my opinion, man.

Permalink | Leave a comment  »