ricardochimal
`taps pull` from remote mysql server to local mysql server is failing
Hello. I'm not able to get a pull working. I have mysql on an Ubuntu slice version: mysql Ver 14.12 Distrib 5.0.51a, for debian-linux-gnu (x86_64) using readline 5.2
When I start up the server and hit it with a "taps pull...." from my local machine (running the same application code but with an empty database), here is the first part of the server log. My http user is "foo".
== Sinatra/1.0 has taken the stage on 5000 for production with backup from Mongrel
68.48.194.117 - foo [25/Jul/2010 21:55:23] "GET / HTTP/1.1" 200 - 0.0013
68.48.194.117 - foo [25/Jul/2010 21:55:23] "POST /sessions HTTP/1.1" 200 - 0.0647
68.48.194.117 - foo [25/Jul/2010 21:55:23] "GET /sessions/6659500699/pull/table_names HTTP/1.1" 200 - 0.0087
68.48.194.117 - foo [25/Jul/2010 21:55:23] "POST /sessions/6659500699/pull/table_count HTTP/1.1" 200 - 0.0100
68.48.194.117 - foo [25/Jul/2010 21:55:24] "POST /sessions/6659500699/pull/table_count HTTP/1.1" 200 - 0.0073
68.48.194.117 - foo [25/Jul/2010 21:55:24] "POST /sessions/6659500699/pull/table_count HTTP/1.1" 200 - 0.0133
68.48.194.117 - foo [25/Jul/2010 21:55:24] "POST /sessions/6659500699/pull/table_count HTTP/1.1" 200 - 0.0073
68.48.194.117 - foo [25/Jul/2010 21:55:24] "POST /sessions/6659500699/pull/table_count HTTP/1.1" 200 - 0.0076
68.48.194.117 - foo [25/Jul/2010 21:55:25] "POST /sessions/6659500699/pull/table_count HTTP/1.1" 200 - 0.0087
68.48.194.117 - foo [25/Jul/2010 21:55:25] "POST /sessions/6659500699/pull/table_count HTTP/1.1" 200 - 0.0073
68.48.194.117 - foo [25/Jul/2010 21:55:25] "POST /sessions/6659500699/pull/table_count HTTP/1.1" 200 - 0.0085
68.48.194.117 - foo [25/Jul/2010 21:55:25] "POST /sessions/6659500699/pull/table_count HTTP/1.1" 200 - 0.0089
68.48.194.117 - foo [25/Jul/2010 21:55:26] "POST /sessions/6659500699/pull/table_count HTTP/1.1" 200 - 0.0080
/usr/lib/ruby/gems/1.8/gems/taps-0.3.11/bin/schema:3:in `require': no such file to load -- rubygems (LoadError)
from /usr/lib/ruby/gems/1.8/gems/taps-0.3.11/bin/schema:3
68.48.194.117 - foo [25/Jul/2010 21:55:26] "POST /sessions/6659500699/pull/schema HTTP/1.1" 200 - 0.0804
/usr/lib/ruby/gems/1.8/gems/taps-0.3.11/bin/schema:3:in `require': no such file to load -- rubygems (LoadError)
from /usr/lib/ruby/gems/1.8/gems/taps-0.3.11/bin/schema:3
68.48.194.117 - foo [25/Jul/2010 21:55:27] "POST /sessions/6659500699/pull/schema HTTP/1.1" 200 - 0.0867
/usr/lib/ruby/gems/1.8/gems/taps-0.3.11/bin/schema:3:in `require': no such file to load -- rubygems (LoadError)
from /usr/lib/ruby/gems/1.8/gems/taps-0.3.11/bin/schema:3
Should it really be trying to pull the sessions table?
On the local Macbook side of things, I also have the Rails application configured with mysql in development version: mysql Ver 14.14 Distrib 5.1.41, for apple-darwin10.2.0 (i386) using EditLine wrapper
Here is a snip of the errors I get locally.
Receiving schema
/Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/schema.rb:64:in `load': undefined method `apply' for nil:NilClass (NoMethodError)
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/database/connecting.rb:73:in `connect'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/core.rb:111:in `connect'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/schema.rb:62:in `load'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/schema:44
/Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/schema.rb:64:in `load': undefined method `apply' for nil:NilClass (NoMethodError)
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/database/connecting.rb:73:in `connect'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/core.rb:111:in `connect'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/schema.rb:62:in `load'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/schema:44
/Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/schema.rb:64:in `load': undefined method `apply' for nil:NilClass (NoMethodError)
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/database/connecting.rb:73:in `connect'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/core.rb:111:in `connect'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/schema.rb:62:in `load'
I have taps 0.3.11 installed on both the slice and macbook. I have tried various combinations of deleting the database and not migrating it, since it appears to be pulling the schema and data, and trying to load the schema first without data.
Later on when the schema is done error-ing, I'll get new errors related to the data. I have a "comment" model (it is a blogging app, with article model etc.) and will see an error explaining this table is missing, but it doesn't complain about the table missing for other models.
Schema: 100% |==========================================| Time: 00:00:07
Receiving data
10 tables, 143 records
/Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/adapters/mysql.rb:169:in `query': Mysql::Error: Table 'rehash_webandy_development.comments' doesn't exist (Sequel::DatabaseError)
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/adapters/mysql.rb:169:in `_execute'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/database/logging.rb:23:in `log_yield'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/adapters/mysql.rb:169:in `_execute'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/adapters/mysql.rb:153:in `execute'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/connection_pool/threaded.rb:84:in `hold'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/database/connecting.rb:189:in `synchronize'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/adapters/mysql.rb:153:in `execute'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/dataset/actions.rb:392:in `execute'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/adapters/mysql.rb:407:in `execute'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/adapters/mysql.rb:333:in `fetch_rows'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/dataset/actions.rb:101:in `each'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/dataset/actions.rb:248:in `map'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/dataset/actions.rb:248:in `map'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/adapters/shared/mysql.rb:235:in `schema_parse_table'
from /Library/Ruby/Gems/1.8/gems/sequel-3.13.0/lib/sequel/database/query.rb:129:in `schema'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/utils.rb:140:in `single_integer_primary_key'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/data_stream.rb:214:in `factory'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/operation.rb:259:in `pull_data'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/operation.rb:257:in `each'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/operation.rb:257:in `pull_data'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/operation.rb:221:in `run'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/cli.rb:164:in `clientxfer'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/cli.rb:33:in `pull'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/cli.rb:21:in `send'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/../lib/taps/cli.rb:21:in `run'
from /Library/Ruby/Gems/1.8/gems/taps-0.3.11/bin/taps:6
from /usr/bin/taps:19:in `load'
from /usr/bin/taps:19
I'd really love to be able to use taps. This is the last step I have pulling down my remote database in preparation for moving my site over to Heroku. I suppose I could just take a regular mysqldump file and import that, but I wanted to see if any of these issues were valid issues first. I'm assuming I'll have an easier time on the db:push side of things once I get the database set up locally ok.
The blog post I've followed so far is this one: http://adam.heroku.com/past/2009/2/11/taps_for_easy_database_transfers/
Thanks!
Actually I just used mysqldump. The good news is taps worked perfectly with heroku db:push :)
Here is what I did that I was trying to use taps for from a pull perspective.
mysqldump -h localhost -u root --password=password dbname > dumpfilename
scp -P myport [email protected]:/home/user/public_html/site/current/dumpfilename .
mysql -u user local_db_name < dumpfilename
MySQL imported everything, and a simple heroku db:push and all the data was live. Is it fair to assume since you guys work with PostgreSQL maybe MySQL is not supported as well? Feel free to nuke this issue or break it into sub-issues if that is more appropriate. I just wanted to share how I accomplished the same thing with mysqldump in case anyone else was struggling with this. Thanks.
I was able to fix the require 'rubygems' errors in the schema bin by changing the way Utils.schema_bin works:
def schema_bin(*args)
bin_path = File.expand_path("#{File.dirname(__FILE__)}/../../bin/#{bin('schema')}")
`schema #{args.map { |a| "'#{a}'" }.join(' ')}`
#`"#{bin_path}" #{args.map { |a| "'#{a}'" }.join(' ')}`
end
