carrierwave_backgrounder icon indicating copy to clipboard operation
carrierwave_backgrounder copied to clipboard

Published 20 hours ago • verified publisher icon lardawge

Asset being uploaded before backgrounder gets to it on update

Open jvsidler opened this issue 12 years ago • 25 comments

ruby 2.0.0-p0 Carrierwave 0.8.0 Carrierwave_backgrounder 0.2.1 Rails 4.0.0.rc1 Sidekiq 2.12.0 Redis 3.0.4 Fog 1.11.1

Using store_in_background

When I attempt to update a video:

File is uploaded to cache Record gets the tmp column updated File is uploaded to Rackspace Cloudfiles as if backgrounder wasn't being used Then sidekiq attempts to update file on Cloudfiles but can't find the resource since the file now has a new name from the upload that happened before backgrounder attempted to update the resource.

Controller update action:

def update
  if @video.update(video_params)
    redirect_to edit_admin_video_path(@video), notice: 'Video was successfully updated.'
  else
    render action: 'edit'
  end
end

Uploader class:

class VideoUploader < CarrierWave::Uploader::Base
  include ::CarrierWave::Backgrounder::Delay

  storage :fog

  def store_dir
    "#{@model.class.to_s.underscore.pluralize}/#{@model.id}/#{@mounted_as.to_s}"
  end

  def extension_white_list
    %w(mp4)
  end

end

Video class:

require 'carrierwave/orm/activerecord'

class Video < ActiveRecord::Base
  ...
  mount_uploader :file_url, VideoUploader
  store_in_background :file_url
  ...
end

jvsidler avatar Jun 05 '13 19:06 jvsidler

@jvsidler have you considered using the process_in_background option that backgrounder provides? I believe that I needed to enable both, to solve my problems.

Are you using any kind of custom workers around backgrounder (as I was in my issue)?

ekampp avatar Jun 05 '13 20:06 ekampp

@ekampp I get a NoMethodError on 'body' for whatever reason when I enable both. Switching to process_in_background fixes the problem I'm having but times out my browser when uploading. Shouldn't it pass that to the background?

Thanks so much for your suggestion.

jvsidler avatar Jun 05 '13 21:06 jvsidler

@jvsidler, store_in_background is the method you want to use if you want to defer processing and uploading to s3. The real question is, why is the upload not getting deferred until sidekiq processes the job.

Can you try with master?

lardawge avatar Jun 05 '13 21:06 lardawge

@lardawge I'll try with master right now

jvsidler avatar Jun 05 '13 21:06 jvsidler

@lardawge same issue still occurs. When I put a debugger in the store_dir method on my uploader it gets called three times after submitting ...

jvsidler avatar Jun 05 '13 22:06 jvsidler

@lardawge I've put together a test app at https://github.com/jvsidler/carryup. You can verify that it is happening on a brand new app. Please let me know if there is anything I can do to help.

jvsidler avatar Jun 05 '13 23:06 jvsidler

I'm working with resque, and I don't have any of these problems. Have you considered swapping out your queue, to determine if it's backgrounder it self, or a problem with backgrounder/sidekiq?

ekampp avatar Jun 06 '13 11:06 ekampp

@jvsidler,

It looks like you have a couple issues here...

  1. You should name the file field without the _url suffix. It is possibly confusing carrierwave and is causing CB to not behave correctly.
  2. The field for storing the tmp directory should not be passed into the form. It is for CB internal use.

Something like this:

class Video < ActiveRecord::Base
  mount_uploader :attachment, VideoUploader
  store_in_background :attachment
end

Migration

class CreateVideos < ActiveRecord::Migration
  def change
    create_table :videos do |t|
      t.string :attachment
      t.string :attachment_tmp

      t.timestamps
    end
  end
end

If your intention is to pass in the video as a url, the field in the form would need to be :remote_attachment_url.

lardawge avatar Jun 06 '13 15:06 lardawge

@lardawge I changed the column name as you suggested but to no avail. Also, I did not have a _tmp form field in my app, I just missed that when rapidly prototyping the test app.

@ekampp I swapped sidekiq out for resque and it seems to be working ... at least on my test app.

jvsidler avatar Jun 06 '13 16:06 jvsidler

@lardawge I'm still getting the same behavior. I've changed the column names and removed the tmp field from the form and switched to resque. Even on the test app. Can you reproduce locally with my test app?

jvsidler avatar Jun 06 '13 18:06 jvsidler

@lardawge some more info step by step:

  1. I turned my worker queue OFF and created a record and, as expected, it did not upload
  2. I turned the queue worker ON and it uploaded successfully as expected
  3. I turned the queue worker OFF and updated the record. IMMEDIATELY on Rackspace Cloudfiles the whole directory from record_id down (I'm saving to model_name/record_id/mounted_on/file_name) was deleted and nothing was uploaded
  4. I turned my queue worker ON and the worker reports a fail because it can't find the resource BUT uploads the new file correctly. Since the job reports an error, the database record is not updated with the new file name and doesn't remove the info in the _tmp column

jvsidler avatar Jun 06 '13 18:06 jvsidler

Can you try just assigning file_attachment?

@video.file_attachment = params[:file_attachment]
if @video.save
...

lardawge avatar Jun 06 '13 18:06 lardawge

Saves immediately to rackspace and then the background job fails.

def update
  @video.file_attachment = video_params[:file_attachment]
  if @video.save
    redirect_to @video, notice: 'Video was successfully updated.'
  else
    render action: 'edit'
  end
end

jvsidler avatar Jun 06 '13 19:06 jvsidler

If it works with resque, but not sidekiq, that suggests to me, that the issue is in the interface between sidekiq and backgrounder. @lardawge is there a central place, where the different worker queues are defined and handled?

ekampp avatar Jun 06 '13 19:06 ekampp

@ekampp my test app is currently using resque

jvsidler avatar Jun 06 '13 19:06 jvsidler

Ok. And the problem still persists? Strange. I don't have that problem with my app. I will try and download yours tomorrow.

ekampp avatar Jun 06 '13 19:06 ekampp

@lardawge step number five from above is more accurately stated as follows:

  1. I turned my queue worker ON and the worker reports a failure because it can't find the resource BUT THE FILE IS UPLOADED CORRECTLY. The job reports an error so the database record is not updated with the new file name and doesn't remove the info in the _tmp column.

This is weird since the file correct file gets up there just fine ...

jvsidler avatar Jun 06 '13 19:06 jvsidler

I'll try and have a look at it tonight...

lardawge avatar Jun 06 '13 21:06 lardawge

Thanks so much!

jvsidler avatar Jun 06 '13 22:06 jvsidler

What is the error you are seeing from the worker?

lardawge avatar Jun 07 '13 18:06 lardawge

CarrierWave::Workers::StoreAsset Arguments "Video" "5" "file_attachment" Exception Fog::Storage::Rackspace::NotFound Error [HTTP 404] resource not found in ord region

I can also confirm that the upload happens before the background job attempts it because if I remove the asset from Rackspace I get the error right away. This shows me that the upload is attempted immediately. When make sure the asset is on Rackspace as my application expects it to be, the upload works fine and then the background job fails.

jvsidler avatar Jun 07 '13 19:06 jvsidler

@jvsidler, I think there is some funkiness with rackspace cdn. I get the error just trying to submit the update form. It creates it fine but can't fins the resource on update.

lardawge avatar Jun 10 '13 17:06 lardawge

But it works fine when I take carrierwave_backgrounder out ... where does backgrounder intercept the create/update calls?

jvsidler avatar Jun 11 '13 15:06 jvsidler

I have the same problem - I believe this a bug in carrierwave: https://github.com/carrierwaveuploader/carrierwave/issues/1333

johnhamelink avatar Aug 31 '14 23:08 johnhamelink

I am also using process_in_background and I am getting a lot of errors from backgrounders about not able to find a temp file, perhaps the same problem? Since it make sense if the files has been uploaded before sidekiq get a chance to process it.

E, [2015-02-21T21:17:02.258660 #16795] ERROR -- : Actor crashed!
CarrierWave::ProcessingError: Failed to manipulate with MiniMagick, maybe it is not an image? Original Error: Command ("mogrify -resize 200x200 /tmp/mini_magick20150221-16795-4ox14p.jpg") failed: {:status_code=>nil, :output=>""}
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/processing/mini_magick.rb:273:in `rescue in manipulate!'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/processing/mini_magick.rb:259:in `manipulate!'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/processing/mini_magick.rb:153:in `resize_to_fit'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/processing.rb:84:in `block in process!'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/processing.rb:76:in `each'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/processing.rb:76:in `process!'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave_backgrounder-0.4.1/lib/backgrounder/delay.rb:14:in `process!'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/callbacks.rb:18:in `block in with_callbacks'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/callbacks.rb:18:in `each'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/callbacks.rb:18:in `with_callbacks'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/cache.rb:122:in `cache!'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave-0.10.0/lib/carrierwave/uploader/versions.rb:226:in `recreate_versions!'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/carrierwave_backgrounder-0.4.1/lib/backgrounder/workers/process_asset.rb:12:in `perform'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/processor.rb:75:in `execute_job'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/processor.rb:52:in `block (2 levels) in process'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:127:in `call'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:127:in `block in invoke'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.3.241/lib/new_relic/agent/instrumentation/sidekiq.rb:33:in `block in call'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.3.241/lib/new_relic/agent/instrumentation/controller_instrumentation.rb:365:in `perform_action_with_newrelic_trace'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/newrelic_rpm-3.9.3.241/lib/new_relic/agent/instrumentation/sidekiq.rb:29:in `call'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-unique-jobs-3.0.11/lib/sidekiq_unique_jobs/middleware/server/unique_jobs.rb:17:in `call'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/server/retry_jobs.rb:74:in `call'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-failures-0.4.3/lib/sidekiq/failures/middleware.rb:9:in `call'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/server/logging.rb:11:in `block in call'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/logging.rb:22:in `with_context'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/server/logging.rb:7:in `call'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:132:in `call'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:132:in `invoke'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/processor.rb:51:in `block in process'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/processor.rb:98:in `stats'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/processor.rb:50:in `process'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `public_send'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `dispatch'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:122:in `dispatch'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in `block in invoke'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in `block in task'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in `block in task'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in `block in initialize'
        /var/www/wwm/releases/37/vendor/bundle/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in `block in create'

pencilcheck avatar Feb 22 '15 19:02 pencilcheck