monogreen - Tank you very much !

ActiveRecord Unit Tests and MySQL default encoding

Recently, I’ve been trying to run the unit tests of ActiveRecord of the Rails 2-3-stable branch with the MySQL adapter – I thought I had actually found a bug in an obscure area of association handling with unsaved records.
There were two problems with getting started: First of all, the tests didn’t start because of an uninitalized constant. That was solved as so many problems nowadays by Stack Overflow:

http://stackoverflow.com/questions/1145318/getting-uninitialized-constant-error-when-trying-to-run-tests

Thanks ! One uninstall of test-unit later, some of the unit tests failed with this error:

1) Error:
test_optionally_validates_length_of_using_within_on_create_utf8(ValidationsTest):
ActiveRecord::StatementInvalid: Mysql::Error: Incorrect string value: '\xE4\xB8\x80\xE4\xBA\x8C...' for column 'title' at row 1: INSERT INTO `topics` (`group`, `title`, `author_name`, `approved`, `parent_title`, `bonus_time`, `replies_count`, `type`, `content`, `parent_id`, `author_email_address`, `written_on`, `last_read`) VALUES(NULL, '一二三四五六七八九', NULL, 1, NULL, NULL, 0, NULL, 'whatever', NULL, 'test@test.com', '2011-01-27 10:41:11', NULL)
    ./test/cases/../../lib/active_record/connection_adapters/abstract_adapter.rb:227:in `log'
    ./test/cases/../../lib/active_record/connection_adapters/mysql_adapter.rb:324:in `execute_without_query_record'
    ./test/cases/helper.rb:36:in `execute'
    ./test/cases/../../lib/active_record/connection_adapters/abstract/database_statements.rb:259:in `insert_sql'
    ./test/cases/../../lib/active_record/connection_adapters/mysql_adapter.rb:334:in `insert_sql'
    ./test/cases/../../lib/active_record/connection_adapters/abstract/database_statements.rb:44:in `insert_without_query_dirty'
    ./test/cases/../../lib/active_record/connection_adapters/abstract/query_cache.rb:19:in `insert'
    ./test/cases/../../lib/active_record/base.rb:2961:in `create_without_timestamps'
    ./test/cases/../../lib/active_record/timestamp.rb:53:in `create_without_callbacks'
    ./test/cases/../../lib/active_record/callbacks.rb:266:in `create'
    ./test/cases/../../lib/active_record/base.rb:2927:in `create_or_update_without_callbacks'
    ./test/cases/../../lib/active_record/callbacks.rb:250:in `create_or_update'
    ./test/cases/../../lib/active_record/base.rb:2577:in `save_without_validation'
    ./test/cases/../../lib/active_record/validations.rb:1090:in `save_without_dirty'
    ./test/cases/../../lib/active_record/dirty.rb:79:in `save_without_transactions'
    ./test/cases/../../lib/active_record/transactions.rb:229:in `send'
    ./test/cases/../../lib/active_record/transactions.rb:229:in `with_transaction_returning_status'
    ./test/cases/../../lib/active_record/connection_adapters/abstract/database_statements.rb:136:in `transaction'
    ./test/cases/../../lib/active_record/transactions.rb:182:in `transaction'
    ./test/cases/../../lib/active_record/transactions.rb:228:in `with_transaction_returning_status'
    ./test/cases/../../lib/active_record/transactions.rb:196:in `save'
    ./test/cases/../../lib/active_record/transactions.rb:208:in `rollback_active_record_state!'
    ./test/cases/../../lib/active_record/transactions.rb:196:in `save'
    ./test/cases/validations_test.rb:1146:in `test_optionally_validates_length_of_using_within_on_create_utf8'
    ./test/cases/../../lib/active_record/test_case.rb:57:in `with_kcode'
    ./test/cases/validations_test.rb:1137:in `test_optionally_validates_length_of_using_within_on_create_utf8'
    ./test/cases/../../../activesupport/lib/active_support/testing/setup_and_teardown.rb:62:in `__send__'
    ./test/cases/../../../activesupport/lib/active_support/testing/setup_and_teardown.rb:62:in `run'

AR tests assume that the database is set to the utf8 character encoding, and rightly so – I think anybody not using Unicode for everything needs to have very very good reasons for doing so. The problem is, though, that MySQL still ships with latin-1 as the default encoding, and nowhere in the notes on running AR tests was this mentioned. Well, here’s the fix:

mysql> drop database activerecord_unittest;
Query OK, 100 rows affected (7.28 sec)

mysql> drop database activerecord_unittest2;
Query OK, 1 row affected (0.10 sec)

mysql> create database activerecord_unittest DEFAULT CHARACTER SET 'utf8' DEFAULT COLLATE 'utf8_general_ci';
Query OK, 1 row affected (0.00 sec)

mysql> create database activerecord_unittest2 DEFAULT CHARACTER SET 'utf8' DEFAULT COLLATE 'utf8_general_ci';
Query OK, 1 row affected (0.00 sec)

I also encountered this problem:

  1) Error:
test_load_schema(AAACreateTablesTest):
ActiveRecord::StatementInvalid: ./test/cases/../../lib/active_record/connection_adapters/abstract_adapter.rb:227:in `log': Mysql::Error: PROCEDURE topics already exists: CREATE PROCEDURE topics() SQL SECURITY INVOKER
BEGIN
    select * from topics limit 1;
END

I guess AR doesn’t clean up the stored procedures after the tests ran – AFAIK, they’re not saved with the tables, so they persist after a test run. Dropping and recreating the DB works, though.
At this point I discovered that our Bug had already been fixed in the 2-3-stable branch, merged that, and went home.
Another job well done !