in_threads
in_threads copied to clipboard
A way to wrap the default thread executor
a ruthlessly handy library thank you,
I make use of it to speed up generating database seeds in a rails app (db/seeds.rb
) file.
as such:
users = [
{
display_name: 'John McTest',
email: '[email protected]',
password: password,
},
# ...
].in_threads.with_progress('users').map do |entry|
User.create!(entry)
end
In order to make use of database connections across threads, one needs to wrap each block with ActiveRecord::Base.connection_pool.with_connection
.
For readability concerns, I would like to avoid sprinkling ActiveRecord::Base.connection_pool.with_connection
throughout my seeds file.
Right now I use the following heavy handed global workaround:
require 'in_threads'
InThreads::Pool.class_eval do
def run(&block)
wrapped = ->{ ActiveRecord::Base.connection_pool.with_connection { block.call } }
@queue.push(wrapped)
end
end
I was wondering if there is a better, recommended way of handling this situation? Maybe a way to configure (or wrap) the default thread execution? Or if I could easily write a new in_db_threads
helper that would behave like in_threads
, but wrap with activerecord, but that seems impossible without monkey patching this gem.
Thank you 😄
Currently library doesn't have any supported way to hook into the process.
The workaround looks fine, only problem it being a monkey patch if/when something changes.
I checked introducing another wrapper (.in_threads.with_progress('users').with_db.
), but looks like it will not be simple, so best way would be to introduce hooks like .in_threads(around_each: proc{ |&block| ….with_connection(&block) })
and then you could add a method like in_db_threads
to wrap it.
Thanks for the response.
If comes a stage when I need to get rid of the patch, I'll happily try implementing that hook idea. I'll let you decide if you want to keep this issue open to track the idea.