Hello! ☞ This is an archived version of Al Shaw's personal website. The current site is http://shaw.al.


TPM on Rails: Building a News App in Under Two Months

Design, Future of News, Rails, TPM, code

Last week I released my first Rails app, the TPM PollTracker. As we speak, it’s humming along on the Heroku cloud.

That, in itself isn’t as momentous as what it signifies for news. I made my first commit to the project on October 16 (and have totally scrapped most of the code at least once since then). As those at the Times, ProPublica, NPR, the Texas Tribune and others have already figured out, framework-based rapid development and agile methodologies go really well with deadline-based newsroom culture. In that sense, highly-abstracted software development can be seen as almost analogous to newswriting. Where the news story, as a form, is a rigid, highly repetitive (though highly creative) venture, so Rails’ ‘convention over configuration’ approach boils the act of creating an app down to its bare essentials. Though, as I mentioned before, it is through this rigidity that Rails affords you freedom. More freedom than any content management system can hope to achieve. Because the ‘news blob’ is such an amorphous thing— one never knows exactly what will break over the course of the day, and how one might want to present content— a content management system in a news setting will necessarily either have to be hacked apart beyond all recognition (Huffington Post, for example, ended up forking MT 3.x to create their own Frankeinstein CMS), or will need to be designed with maximum flexibility in mind (at the other end of the spectrum from HuffPo is, say, NPR, and their “Flexible Presentation Layer.”)

At the end of this initial burst of development, I see the TPM PollTracker as a number of things— how creativity can flourish out of frustration with given tools, an example of database design guiding news structure and how the concepts of DRY and Object Orientation can extend all the way from source code to editorial layers. I will talk about each of these in turn.

But first, let me start by showing you something really scary. That was PollTracker 0.0.1, a web service in a Movable Type template (with query string support for candidate filtering, no less), enough to make anyone who has ever heard the initials MVC weep tears of blood. My task was to write an API for our flash developer (the incomparable Michiko Swiggs) so she could greate the awesome graphs that now grace our app (did I mention they are embeddable). And this is what I had to work with:

MT as PollTracker

A string.

My whole “API” hinged on parsing out whitespace between words in strings, because we had to cobble something together out of Movable Type’s ‘blog entry’ structure (which isn’t only inappropriate by virtue of shoehorning data types into places they weren’t meant to go, it also causes editorial cognitive dissonance when you’re asked to put a series of poll data into a field marked Post Title, and a date into an Entry Body). Hence 355 lines of PHP. And of course one typo or missed space would throw the whole thing off (which I learned the hard way while writing rake tasks to import all this data into the new system and Rails’ find_or_create_by method would assign separate candidates to Approve, Approv, Disapprove, DIsapprove, etc).

What I learned from this exercise was the importance of a rock solid database schema, not just to fit the needed data, but to serve as a skeleton for the app’s structure and user interface and also a sanity check for editors. Shortly after deciding to build this app with Rails, Josh and I had a conversation with Charles Franklin, a polling expert at University of Wisconsin. He opened our eyes to the sheer amount of data bits needed to do poll tracking right. From there I set to work creating a schema that would minimize repetition. A foundation on which to build the app.

Here’s what I came up with:

PollTracker database and workflow diagram

Above left: PollTracker’s database structure and associations.

Above right: The DRY editorial workflow it engenders (both made in Diagrammr, so forgive the pluralization ish)

One of the biggest problems in poll tracking is attempting to approach 100% accuracy. One wrong number, misspelled name, misassigned race, can cast doubt on the whole operation. Therefore I designed this database (and data input system) to minimize repetition. This also dovetails nicely with the DRY principle. The old way was all about typing. Every poll required you to type (or mistype) a candidate’s name again and again. The new system is about choosing as much as you can. You can’t misspell something if you choose it rather than type it. So you create a candidate once, you create a pollster once, you create a contest once. Then you choose these elements over and over when creating a poll (the most basic and precarious unit of the app). Unbeknownst to the non-technosavvy inputter of data, she is practicing DRY, and Object Oriented principles within the editorial workflow. By creating a poll, she is connecting pollster, candidate and contest objects together.

Here’s what that same poll entry screen looks like in the new PollTracker app (it already knows what you want to do):

PollTracker edit poll screen

Another way the database was designed to minimize redundancy is the creation of the scores table. This is basically a join table which connects candidates to polls. The only other value in the table is poll_pct (the individual score of a given candidate). This was done in order to avoid storing candidates with their scores (because scores are inherent to polls), and allowing each poll to vary in number of candidate/score combos.

If you go to the live app, you’ll see the site is roughly organized around these data structures. There are main sections for Polls, Contests, Pollsters and Candidates. In every section, these objects interact with each other, but always in the context of the section you are in. With a flexible database structure, you can pivot your views around each model easily. But the main thing with Rails is, you define these pillars, these news blobs. And once you do, the rest of the structure falls in place nicely.

The rest is just gravy. And with the Rails community, there is a ton of gravy. To get all colophonic on you— The backend login system is powered by restful_authentication (which I patched for role-based permissions). permalink_fu makes URLs pretty. I borrowed heavily from Tim Riley’s fork of Ryan Bates’ complex form examples to do complex forms with jQuery. And there are a host of Heroku tools like taps and caches_page_via_http which make deploying and maintaining rails apps super easy and fast. There’s also a lot of cool stuff in the backend like dynamic admin bars and edit areas, AJAX-loaded pulldowns of commonly chosen candidates in poll forms and Markdown support (via BlueCloth) for longer text areas like candidate bios.

We are gearing up for another sprint on the next iteration of the PollTracker, which will have more features, will be ajaxier, and hopefully faster. With this project, I feel like I have been living in the future and things are just going to get cooler.

You should follow me on Twitter here.

Ed.note 3/10/10: Since this post was written, we have moved the PollTracker from Heroku to Engine Yard Cloud.