I recently setup a Jenkins CI server for a client and put their Rails app in it. When someone committed a change, it ran a CI build. However, when someone committed a migration, rake would abort with the dreaded "pending migrations" error. So I just added a db:migrate step to the ci:build task, and all was well…

…or so I thought. In this particular case, the CI server builds anytime someone pushes code to either the "develop" or "master" branch (using git). Pushes to develop happen much more often than master, because master is the "stable production release branch" and develop is the "push everything here when it's ready to be shared with the rest of the team and integrated into the bleeding edge version of the app" branch. One consequence of this is that master builds will often assume an older database schema than develop builds because those migrations haven't been merged with the master branch yet.

So running "rake db:migrate" in the ci:build task broke. The migrate task doesn't understand moving the schema backwards and forwards unless you explicitly tell it what to do. However, there's a much better way to do this. The file db/schema.rb is a current snapshot of your database and it should be checked into your code repo. So if you just load that into the test DB before each CI build (via the handy "rake db:schema:load" task), then you're golden. Master has its snapshot, and develop has its snapshot. Everything's fixed! Er...

Here's the wrinkle: When you run rake spec after doing that (within the same rake ci:build über-task), it executes the following tasks:

** Invoke spec (first_time)
** Invoke db:test:prepare (first_time)
** Invoke db:abort_if_pending_migrations (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:abort_if_pending_migrations
** Execute db:test:prepare
** Invoke db:test:load (first_time)
** Invoke db:test:purge (first_time)
** Invoke environment** Execute db:test:purge
** Execute db:test:load
** Invoke db:schema:load
** Invoke environment** Execute spec

So what? It runs db:schema:load again, no big deal. Nope. Chuck Testa. If you look closely, you'll see that while db:schema:load is *invoked*, it is never actually *executed*. And that's because rake remembers that it has already been executed when we ran it manually before. So the last thing that's been done to your test database is db:test:purge. If you look at your test database at this point, you'll see that is has no tables. And your tests will fail (at least they better).

Why are there no tables? Why didn't the manual db:schema:load put all the tables into the test database? Because it put them into the development database. Remember that most tasks default to the development environment when you don't specify one. So we loaded the DB schema snapshot into the development database, and then later when rake is asked to run db:schema:load on the test database, it doesn't run it.

So what's the solution? We need to tell rake to re-run db:schema:load. (Don't be tempted here to force Rails.env = 'test' and use the test database for everything. This comes with its own issues and you'll be fighting against the way rake spec is designed to work.)

Here's what my final rake ci:build task looks like:

namespace :ci do
  desc 'Setup CI environment'
  task :env do
    ENV['CI_BUILD'] = 'true'
    ENV['RSPEC_FORMAT'] = 'progress'
    ENV['CUCUMBER_FORMAT'] = 'progress'
    ENV.delete('http_proxy') # need cukes to be able to connect to localhost
  end

  desc 'Run a continuous integration build'
  task :build => ['ci:env'] do
    Rake::Task['db:reset'].invoke # drops, re-creates, loads schema.rb
    Rake::Task['db:schema:load'].reenable # so it can be run later on the test db
    Rake::Task['spec'].invoke
    Rake::Task['cucumber'].invoke
  end
end

That line "Rake::Task['db:schema:load'].reenable" is the key. Once you add that, rake spec will happily resume loading up your test DB's schema and you'll be off to the races.

Permalink | Leave a comment  »

"Be the Change you wish to see in the world" is a common quote attributed to Mahatma Gandhi. But he never said it, and in fact didn't believe the smug, apolitical sentiment it communicates. That and other awesome gems in this NYT article.

My favorite excerpt on the fake Gandhi quote:

Here, Gandhi is telling us that personal and social transformation go hand in hand, but there is no suggestion in his words that personal transformation is enough. In fact, for Gandhi, the struggle to bring about a better world involved not only stringent self-denial and rigorous adherence to the philosophy of nonviolence; it also involved a steady awareness that one person, alone, can’t change anything, an awareness that unjust authority can be overturned only by great numbers of people working together with discipline and persistence.

Hmm, great numbers of people working together with discipline and persistence. Why, that almost sounds like organizing!

Permalink | Leave a comment  »

Permalink | Leave a comment  »

Permalink | Leave a comment  »

def do_something(first_arg, second_arg)
  puts "First: #{first_arg}"
  puts "Second: #{second_arg}"
end

do_something(:second_arg => 'blah', :first_arg => 'yeah') =>
First: yeah
Second: blah

do_something('yeah', 'blah') =>
First: yeah
Second: blah

def do_something(params)
  puts "First: #{params[:first_arg]}"
  puts "Second: #{params[:second_arg]}"
end

Permalink | Leave a comment  »

find(:id => id).tap { |foo|
  foo.do_things
  something_else(foo)
  if foo.has_bar?
    this_thing(foo)
  else
    that_other
  end
}

foo = find(:id => id)
foo.do_things
something_else(foo)
if foo.has_bar?
  this_thing(foo)
else
  that_other
end
foo

Permalink | Leave a comment  »

Permalink | Leave a comment  »

Then /^it should raise (.+?) when (.+)$/ do |exception,when_step|
  lambda {
    When when_step
  }.should raise_error(eval(exception))
end

Permalink | Leave a comment  »

You get what you vote for, America.

Permalink | Leave a comment  »

I've heard so many times that Americans won't really "get" global
warming until it comes to our shores. Well, it looks like it has.
Something tells me, given the current political climate (pun very much
intended), that this won't really change many minds.

Permalink | Leave a comment  »

• Every message and thread would have a URL. Easy linking and sharing.
• Messages would be sender-stored and pulled by the recipient. Makes life harder for spammers, easier for legitimate bulk emailers.
• Since messages are stored all over the web, new message notifications will have to push realtime. Push email is no longer a luxury feature.
• Using semantic markup (not necessarily HTML), reply/quoting order is a reader preference, no matter how the author wrote it. No more top posts.
• Blogging becomes emailing "everyone." No recipient means everyone can see the message at its URL.
• Forwarding a large email just gives new recipients access to the existing message. No more re-uploading big attachments.
• Mailing lists are web archived as a free side effect. Messages can link to each other. Free wiki?
• A thread can be re-constructed from any individual message because it links to its parent via a URL Messages can be un-sent. And you can reliably determine whether the recipient has seen it yet or not.
• Email analytics becomes web analytics. No more messy embedded tracking images.

Permalink | Leave a comment  »

As more U.S. cities and states fall into the smoke-free camp (i.e. they ban smoking in workplaces, such as bars and restaurants), you hear the same refrains over and over from the non-smoker, "pro-freedom" crowd. Things along the lines of, "It should be up to the business owners whether or not to allow it, and then I can choose whether or not I want to go there," followed by some pronouncement of how the government ruins everything when it tries to be everyone's nanny and make things illegal because "some people don't like it."

Frankly, given the amount of time most people will spend thinking about issues like this, the above is a pretty compelling argument. Freedom of choice, private enterprise, and free-market capitalism are the ingredients of a big, fat America sandwich. Yum.

Let's set aside most of the compelling arguments in favor of workplace smoking bans (such as the relative difficulty for the employees to just "go somewhere else") and focus on the principle behind the anti-ban sentiment. Essentially it says that government shouldn't get involved because different businesses will adopt different policies and they'll be in line with the preferences of their customers and employees who will vote with their feet. Right?

Wrong. This exact same principle can be used to justify segregation at restaurants in the South. Since almost none of the anti-ban folks would claim to be in favor of that, the principle must be flawed. I have yet to hear another principle that does hold up, but if you have one, please comment.

"OK, fine," you say. "The principle needs work, but segregation in the South was a unique situation and deserved an exception to an otherwise sound principle. Smoking in bars and restaurants does not deserve a similar exception." Alright, I'll bite.

Here's the problem with that: "Segregation is bad and it's OK for the government to ban it" is a non-controversial standpoint because we've all lived with it being illegal for decades now and the vast majority of us were raised to believe that it's morally wrong. Many of us would be adversely affected by its return, and/or have friends, family, and neighbors who would be. But it hasn't been gone so long that we don't have valid reason to fear its return if the law were changed today.

Smoking, however, is something many of us do all the time. Almost everyone has a friend or family member who smokes. He/she who has never smoked a cigarette at least once in their lives is a rare specimen. So despite all the health warnings we're familiar with, smoking doesn't feel morally wrong or that harmful to any of us. Certainly not so much that the government should ban it.

But step back from the familiar for a second. Let's pretend humans have to abandon the Earth for one of two new life-supporting planets we've discovered. On one planet, the existing alien culture requires that humans live in slums and go to humans-only restaurants, use humans-only bathrooms, and drink from humans-only water fountains. All of these are sub-standard, dirty, and generally unpleasant compared to the ones the aliens of that world enjoy. On the second world, segregation isn't an issue but there is a strange cultural custom where the aliens like to carry around these devices that emit a green-colored gas. They enjoy watching it curl and flow in the breeze or something. Who knows, it's a crazy alien custom. Unfortunately, this green gas causes cancer in humans and aliens alike. So any humans going there are warned that they should avoid these devices. Others who have gone before you have reported that the bars and restaurants are just awful there because they're filled with the carcinogenic gas, and the aliens just don't seem to care because they enjoy the gas so much. Humans on this world are getting cancer left and right. Which world would you choose?

They're both pretty terrible, right? Neither is a place you really want to be. Now imagine there's a third world where the alien government, in an attempt to welcome the human refugees, has outlawed both of these practices in public places and private workplaces, including bars and restaurants. I don't know about you, but that's the planet I'd choose to go to.

That's not government overreach, that's government doing it's job. Sometimes we get so used to things that we forget how stupid they really are. Filling your lungs with carcinogens is your choice, but you shouldn't be allowed to do it anywhere near a place that I or anyone else has to be to make a living.

Permalink | Leave a comment  »

IMG_0371.MOV Watch on Posterous

My friends Luke & Emily's first dance as husband & wife. They're super adorable.

Permalink | Leave a comment  »

Unfortunately there were already passwords, API keys, and other things we couldn't release publicly in our git commit history.

No problem, we thought, we'll just take a snapshot of the code, remove the bits we can't open source, and then upload that to github as a new repo, devoid of all that messy history. Easy peasy.

Er, not so much. The problem is we wanted to maintain our internal branch, complete with git history, but open up development on the open source version (which comprised >99% of all the code) to github and thus outside collaborators. So we were going to be pushing and pulling to/from github, as well as merging into our internal branch. We couldn't let non-open-source code leak into github, but we also needed to merge the open sources changes into the internal version.

Git took a look at these two branches and decided they didn't have anything to do with each other because they had no common ancestry in the commit history. This was the appropriate response from git because we had purposefully removed the commit history of the github version.

I tried various methods of merging the two codebases, but git always generated conflicts left and right because it was attempting to merge two almost-but-not-quite identical codebases with no common ancestor commits.

Here's how I solved it (I hope). Let's say the internal branch is called "internal" and the open source branch is called "opensource". The commit history of the open source branch is one über-commit (X) followed by a couple small changes (Y & Z), which we'll represent as X <-- Y <-- Z. The commit history of the internal branch is pretty long, so we'll just abbreviate it as the three most recent commits, A <-- B <-- C. So here's how our two branches start out:

internal:   A <-- B <-- C
opensource: X <-- Y <-- Z


I decided to try merging the opensource branch with the internal branch using the "ours" merge strategy on the X commit. This merge strategy just discards the changes in the other branch and considers the two branches merged anyway. So I ran:


git checkout internal
git merge -s ours (sha1 of the X commit)


Then my commit history looked like this:


internal:   A <-- B <-- C <-- D
                             /
opensource:                 X <-- Y <-- Z


The new D commit was the "merge" between C and X, but a couple quick git diffs showed that my working tree was exactly the same as the C commit, and thus had discarded the changes from X. This is what I wanted because X represented almost the exact same codebase, except for the minor changes required to open source it.


Now I was in a position to merge new commits to the opensource branch into the internal branch. I ran this (still on the internal branch):

git merge opensource


Then my commit history looked like this:

Permalink | Leave a comment  »

That's a bold claim, but I'm prepared to back it up:

1. Google Maps knows nothing about Metro trains, and is generally confused about bus routes too. WMATA publishes the data that Google needs--in the format Google needs it in--to make Google Maps Metro-aware (you can find it here under "GTFS Download"). But they've published it under a draconian license that reserves the right to charge for it or remove it in the future. So, as you might imagine, that's not cool w/ Google. Way to do all the work and then drown at the shore, guys. More background on this here.

2. For a long time, WMATA didn't even provide this bullet-pre-installed-in-foot non-solution. Originally they refused to play along at all, saying:

Metro staff did explore some possibilities with Google, but ultimately we decided that forming a partnership with Google was not in our best interest from a business perspective. We do believe that Metro's newly redesigned Web site, at www.wmata.com, improves customers' access to information about the Metro system. In addition, customers may get real-time information and bus and rail schedules directly on their cell phones or PDAs.

Oh right, you mean the website whose trip planner determined that a co-worker of mine should use the following Metro rail route between Union Station and Shady Grove (two stops on the red line--the same freaking line):

Notice that round trip in the middle where they have you take the blue line from Metro Center to McPherson Square, then take the orange line back to Metro Center once you get there? Not all red line trains go to Shady Grove, but they should know that and their tool should be able to handle that extremely common situation. And apparently this sort of thing happens pretty often with the WMATA Trip Roulette Machine.

3. Notice how it also doesn't show you a map of your trip (like this other tool called GOOGLE MAPS does). They just want you to trust them that it's all correct. Inspiring a lot of confidence there, guys.

4. Google will always be better at this than you are, Metro tech team. I'm sorry, but it's true. You can't and shouldn't try to compete with their resources, experience, and technology. Feel free to continue banging out your trip planner tool, but please, for the love of Jeebus, let Google use your freaking transit data so those of us who prefer useful tools can get on with our lives. Please?

Permalink | Leave a comment  »

"Do not depend on the hope of results. When you are doing the sort of work you have taken on, essentially an apostolic work, you may have to face the fact that your work will be apparently worthless and even achieve no result at all, if not perhaps results opposite to what you expect. As you get used to this idea, you start more and more to concentrate not on the results but on the value, the rightness, the truth of the work itself."

Apparently it comes from Thomas Merton in "Letter to a Young Activist."

Sigh... Fail.

I understand that things have not been easy for those of us on the
left lately. And this quote offers a tempting salve for those feelings
of frustration. But...

Not focusing on results is a bit of a pet peeve of mine in the
progressive world. It is, in my humble opinion, how we can have
multiple, well-funded non-profits for every single issue we care
about, and yet not make much progress on them. It is how we can
control two branches of national government, and yet be unable to pass
healthcare legislation, close Guantanamo, and reign in the abuses of
large corporate-persons.

Focusing on results is exactly what activists, young and old, should
constantly be doing. The real trick is understanding how to "fail
forward" (to borrow Clay Shirky's term).

Yes, you will fail. A lot. Over and over. But if you build your power
/ movement / organization during each campaign, then you will begin
again in a better position to win than you were in the last time. And
begin again you must. This is politics, and it can be used for good
or evil. We need more progressives focused on using it to achieve good
results. Otherwise, the "work itself" has no "rightness" or "truth." It's
self-delusion.

Permalink | Leave a comment  »

1. Communication

Permalink | Leave a comment  »

In the world of object-oriented software design, we talk about things we're modeling as falling into 1 of 2 categories: Entities and Value Objects. Entities have identity separate from their constituent attributes, while Value Objects do not. So I, for example, am an Entity, but the color magenta is a Value Object. Even if you list my attributes (first name: Wes, last name: Morgan, hair color: blond, eye color: hazel, height: 5'11"), that doesn't fully encompass who I am because there is probably at least one other guy out there matching that same description. Freaky Friday. But if you were trying to find me, that other guy just won't do. Magenta, however, is a color that can be represented by 3 red, green, and blue values, for example. Magenta's magenta. This is nice because you can create an immutable, throw-away magenta instance of a Color class to represent it in your code. Decide you like chartreuse better? Throw away your magenta instance and instantiate a new Color object for chartreuse. This has several handy side-effects for your code. Entities require much more care and feeding. I should know, I am one. And so is that jerk imposter whom I will crush.

Anyway! REST works very well on Entities. You have a URL like http://server.org/people/34 and you call various HTTP methods on person #34 (GET, PUT, POST, DELETE). How rude of you. But it all works. Wam, bam, thank ya ma'am.

But what about Value Objects? If you wanted to write an RGB to HTML hex value web service that converted colors from red, green, and blue values to their equivalent HTML hex value, how would make that RESTful? What's the resource? What operations do you map the HTTP verbs to? Here's a real-world example I ran into this week that highlights the conundrum and how I solved it:

I have a piece of software that converts ZIP codes into City, State pairs. This is how we in the U.S. of A. do postal codes. It can also take full street addresses and sanitize them and find out their 9-digit ZIP code. For those outside the U.S., the 9-digit ZIP code is extra special because it can tell you exactly where that address is geographically, but almost no one knows the final four digits of their code. So having software that can derive that code from their street address (which they do know) is super handy. My mission was to wrap this software in a Ruby on Rails layer to create RESTful web services out of these functions.

Step 1. ZIP -> City, State : ZIP codes are a great example of a Value Object. The ZIP code of my office in downtown Denver, CO is 80202. 80202 is the same as 80202, even though they're at different places on this page. You don't care which one I typed first, they're interchangeable. So it would be silly to create a RESTful web service that did something like this: GET /zips/89 which would return the ZIP code w/ id 89 and the associated city and state. Let's say it's 73120 in Oklahoma City, OK. Why assign ZIP codes an id number? Why not just do this: GET /zips/73120 and have that return Oklahoma City, OK? So that's what I did. The value *is* the id. That works great, moving on.

Step 2. Address -> Better Address (incl. the magical 9-digit ZIP code) : Now things get trickier. An address is still a value object. 123 Main St., Missoula, MT is the same as 123 Main St., Missoula, MT. We don't need to assign id's to these objects, they're just the sum of their parts. But unlike ZIP codes, more than one attribute is required before you can say you have an Address object. You need 1-2 lines of house number, street, apartment number, etc. information, then a city, state, and 5-9 digit ZIP code. That's a bit harder to put into a URL. Ideally we want to be able to label the different attributes so we can easily keep track of what's what. Something like this:
GET /addresses/123%20Main%20St.%20Missoula,%20MT%2059801 ...could work, but it's kind of a hassle because you then have to write a parsing routine on the server side. It would be nicer if you could just label the components of the address when you pass it in. Query parameters to the rescue! Here's what I did:
GET /addresses/1?addr1=123%20Main%20St.&addr2=&city=Missoula&state=MT&zip=59801
Now I can very easily pass this address into my sanitizer and get back the corrected version w/ the 9-digit ZIP. The "1" in the URL is a pseudo-id placeholder. If it bothers you, you could just as easily put "address" or "thisone" or "foo" there. But don't put "get" or "lookup" or any other verb there! That's not RESTful. Only the HTTP verb should define the action being taken on the resource.

So yeah, that's how I did it. I think it's still pretty RESTful. It only uses GET, but that's all you really need for read-only Value Objects. It would probably be better to do something like GET /zips/90210/city or GET /zips/80218/state, or maybe even GET /zips/73120/city_state to avoid the double call in the common case of wanting the city and state. I still don't really know of a cleaner way to do the address sanitizer piece. It works, but it feels a bit hackish.

I would like it if smarter types chimed in and critiqued my approach. Because this is all just, like, my opinion, man.

Permalink | Leave a comment  »

I call it CopyCarrot. As in, the industry has tried the stick, now they should try the carrot. The basic idea is that you embed info in every bit of digital music (MP3 files, CDs, etc.) that contains a standardized link to the artist's PayPal account, or whatever they prefer--it shouldn't be tied to any specific payment gateway, and allows you the music fan to pay them whatever you wish for that song or album or whatever. There's no obligation, there's no digital rights management (read: restriction), just a link that allows peoples' music players to tell them, "Hey, [artist] made this. Care to throw a little money their way? (Yes / No / Stop Asking)."

It would then be tied into your iTunes / Amazon / PayPal / whatever account so that kicking the artist a buck or two would be extremely fast and easy. Obviously consumers would be able to choose if they wanted that tied together or not, it would be up to them and the software / music player they were using.

Of course many people wouldn't pay, and some people would even strip that info out of the songs they passed around the 'Net just like they do with any sort of DRM that the industry tries to add now. But I bet 2 really important things would happen:

1. Enough people would pay enough money for the music they loved that it would make life noticeably easier for artists, and it would do it with no middle man. And really, making it just a little easier for the artists would go a long way.
2. Most people would leave the info in there because it wouldn't restrict their ability to listen to the songs, and it wouldn't even nag them if they told their music player to stop pestering them about paying for their music. In general, people think artists should be compensated, and they respect systems that leave them in control of that decision. It's only when the recording industry middlemen try to impose a bunch of restrictions on how, when, and where they listen to music they have paid for that consumers try to circumvent those restrictions. CopyCarrot imposes no restrictions; it only adds information.

Software that reads this info could be configured to ask people to pay every time they start playing something new, or every X months, or every time they play a song Y+ times, or never. It could also show them what they've paid for, and what they haven't, and nifty graphs showing how much they've paid which artists, per track, per album, etc. Or how much their entire collection is worth vs. how much they've paid. But the point is, it doesn't treat fans as criminals. It recognizes that sharing music is not only a social good (it's not stealing when no one has lost anything, and great art should be shared), but also that spreading the art around helps the artist when it contains a handy link to pay them for their amazing work.

And then obviously you could start doing the same thing with movies, books, images, software, and any other kind of digital media. Artists could also offer promos to encourage people. Like, "Pay me $25 or more for my album today, and I'll get you into my next show in your town for free!" They could also embed other info like their Twitter account or their Last.fm page and then music players could link you straight to those things. It would be a direct artist-to-fan-to-artist circle of love that made everyone happy and gave fans great music and artists money in their pocket to eat food so they can make more great music. Awesome.

And then lastly, and this would be the coolest part of all, it would setup a dynamic where it was in the artists' best interests to encourage their fans to send their music far and wide, to as many people as possible, via whatever systems they prefer. The more people get those MP3's, the bigger the pie from which X% of them will pay the artist an average of $Y per track.

I think this would be pretty great. One of those things I'd like to leave my day job for awhile to go work on. But what say you, dear readers of my bloggity blog?

Permalink | Leave a comment  »