San Francisco, Sqlite3 :memory: Tests, Asteroids

San Francisco, Sqlite3 :memory: Tests, Asteroids 12 comments

posted Thursday, June 1, 2006 by topfunky

“This class got me pumped to code the right way and learn Ruby on Rails.”

“I really appreciated having someone explain some of the more basic syntactical and conceptual elements. It was a productive and fun day, well worth it!”

My workshop in New York city was a success and great reviews are pouring in already. I interviewed Sebastian Delmont for the podcast. I also got to meet the visionary Francis Hwang and other local Rubyists at Sandobe Sushi. There are a lot of brilliant developers in New York, and I hope to visit again soon.

I’ll be in San Francisco next week to teach another Carson Workshops event. It’s a great way to learn Rails, learn about Rails, or polish your skills. Even if you are not a developer it will help you get an idea of what Rails is all about and how you can use it to the max in your business. There are still seats left, so sign up!

Ruby on Rails Workshops in New York City and San Francisco

I’ll also be at Louie’s Bar and Grill for dinner at about 7pm on Wednesday, June 7. If any San Francisco Rubyists are free that night, come on down!

SQLite3 :memory:

Everyone is trying to speed up their tests. I’ve wanted to try sqlite3 memory tests and finally got them working. Here are some results on my soon-to-be-retired 1 GHz iBook G4 (768 MB RAM):

Test	Postgres	Sqlite3 :memory:
Unit Tests	57 seconds	31 seconds
Functional Tests	313 seconds	113 seconds

Yes, I understand that the MacBook runs tests at this rate without extra modification. In the meantime, here’s how you can do it:

Install my memory_test_fix plugin, based on work by Chris Roos.

./script/plugin install memory_test_fix

Install sqlite3 and the sqlite3-ruby gem. See also why and the Rails wiki.

Change your database.yml to use :memory:

test:
  adapter: sqlite3
  database: ":memory:"

Rejoice in faster tests!

Caveats

If you use database-specific queries, you might not be able to use SQLite. I use YEAR and MONTH in Rough Underbelly, so I have to use MySQL for tests in that app.
For older versions of Rails, make sure you are using config.active_record.schema_format = :ruby in environment.rb.

And finally…

(At Barcade, thanks to local Rubyists Martin and Samantha of the Met!)

12 comments

Leave a response

Luke Melia

Great to meet you and your wife in New York, Gregory. Hope to see you again out this way soon!
piggybox

I wonder how fast the Mysql in-memory table can be, comparing to SQLite :memory.
topfunky

I haven’t tried the MySQL in-memory table, but I know that the Robot Co-op uses it in production to keep the whole DB in memory.

I’d love to see a comparison.
Paul Hepworth

The SF workshop was great! Thanks again and it was nice meeting you.
mattsa

The plugin is broken by r6091 of rails trunk:

http://dev.rubyonrails.org/changeset/6091?new_path=trunk

The fix is simple, just change the reference to ActiveRecord::ConnectionAdapters::SQLiteAdapter to ActiveRecord::ConnectionAdapters::SQLite3Adapter

Thanks for wrapping this up so neatly!
tomtoday

mattsa: Thanks!

I’m going through the Peepcode on testing and I couldn’t get the :memory: database to work. I installed using _why’s sqlite3 instructions and regular, file based sqlite3 was working. Spent about 45 minutes scratching my head, then I scrolled down to the comments on this post. Whew.

topfunky: thanks for the post, plugin, and Peepcode. All great! Any chance of getting the plugin code fixed for posterity? TIA.
tomtoday

Thanks for updating the memory_test_fix plugin! That was fast. Appreciated.

Just thought I would note that ActiveRecord::ConnectionAdapters::SQLite3Adapter was added between rails 1.2.1 and 1.2.2. I had 2 apps in development tied to those 2 gem versions and saw errors on the 1.2.1 app with the newly updated plugin. So the plugin now requires rails 1.2.2 or later.

Thanks again!
topfunky

Aiee…I didn’t realize the change was between Rails 1.2.1 and 1.2.2. I’ll see if I can put a conditional in there to work with both.
topfunky

Ok…fixed. Try out the new version, which should work with current and older versions of Rails/ActiveRecord.

http://topfunky.net/svn/plugins/memory_test_fix/
Al Chou

I struggled for a couple of days to use this technique (trying numerous ways of fixing it) and just realized after looking at your code that it only accepts “database” as the specification in database.yml, whereas for SQLite³ it is also acceptable to specify “dbfile”, which is what I was doing. Can you please add a check for that setting, e.g.:

Rails::Configuration.new.database_configuration‘test’ == ’:memory:’
topfunky

Thanks…this has been fixed.
Al Chou

Thanks!

I realize I’m putting a lot of eggs in one basket, but having problems using migrations for the test environment, I would like to suggest instead of having a commented-out migration based line of code in the plugin, use the technique from the version of the Rake task in http://snippets.dzone.com/posts/show/2031 that checks the environment.rb file for the schema_format configuration setting and runs the migration version of db:test:prepare if it is set to :migration.