Archive for April, 2010

Recursive JSON to URL function

Posted on by Alex in Randomness, Software

While working on a snazzy new toy for Velocity, I needed a script to transform JSON to URL parameters. I found this clever method for doing it, but sadly, it wasn’t recursive, so it would only generate URL parameters for the first level of JSON elements.

I rewrote the function to recurse over all elements in the object tree and spit them out in a URL-safe string.

  var JSON = {
    params : function(a1){
      var u=[];
      for(x in a1){
        if(a1[x] instanceof Array)
          u.push(x+"="+encodeURI(a1[x].join(",")));
        else if(a1[x] instanceof Object)
          u.push(JSON.params(a1[x]));
        else
          u.push(x+"="+encodeURI(a1[x]));
      }
      return u.join("&");
    }
  };
var doc = {
  "env":{
    "platform":"linux",
    "browser":"Firefox%2F3.5.8"
  },
  "query":{
    "project":"user-tracking",
    "fetch_time":"0"
  },
  "user":{
    "locale":"en-US",
    "cluster":""
  },
  "last_modified":"1270872812"
};

Reloading SQLite data in Rails

Posted on by Alex in Randomness, Software

While developing a new system built on rails, I encountered a peculiar problem. I wanted to reset my database, but keep the data that was currently living it it. This turned out to be a unique problem for the following reasons:

  • Rails has the functionality to ‘seed’ the database using the seeds.rb file and running: “rake db:seed” This is great if you already have the data written out in pretty rails code. If your data lives in the database, resetting it will wipe out the data.
  • You can only seed data if it has a model associated with it. I’m working with data that uses ‘has_and_belongs_to_many’, which requires a non-modeled table in the db to hold the many-to-many association. You can’t import these associations in Rails.
  • Looking online, I found a nifty rake task to dump the SQL data from the database to a file. You can then load it into the current environment’s db after you’ve done your magic. I found that over at Agile Web Operations. The only problem is, it doesn’t work for sqlite. :(

So I did what any self-respecting developer would do… I rewrote it to do what I needed.

My new rake task implements the following changes:

  1. Works with sqlite3
  2. Only saves the ‘INSERT’ statements from the dump. No need for the schema info from sqlite since it’s already in rails.
  3. Fixes a bug where data loaded was always being loaded into the test db. I like flexibility to load into any environment.

Save the following code into /lib/tasks/seed.rake:

namespace :db do
  namespace :seed do
    require 'db/seed_tables'

    desc "dump the tables holding seed data to db/RAILS_ENV_seed.sql. SEED_TABLES need to be defined in config/environment.rb!!!"
    task :dump => :environment do
      config = ActiveRecord::Base.configurations[RAILS_ENV]
      dump_cmd = "echo '.dump #{SEED_TABLES.join(" ")}' | sqlite3 #{config['database']} | fgrep INSERT > db/#{RAILS_ENV}_seed.sql"
      system(dump_cmd)
    end

    desc "load the dumped seed data from db/RAILS_ENV_seed.sql into the test database"
    task :load => :environment do
      config = ActiveRecord::Base.configurations[RAILS_ENV]
      load_cmd = "sqlite3 #{config['database']} < db/#{RAILS_ENV}_seed.sql"
      system(load_cmd)
     end
  end
end

And, like the previous incarnation, load up the names of the tables you want to dump into a file at /db/seed_tables.rb:

SEED_TABLES = {
  "accounts",
  "products",
  "users"
]

Here’s an example of how I used it:

rake db:seed:dump
rake db:reset
rake db:seed:load