Doesn't call :on_success

[ngw]

Hi, I need to upload some files on S3, and I would like to cleanup the tmp structure I created before on_success.

def upload_file dir, file
  f = dir + '/' + file
  on_error = Proc.new { |http| puts "An error occured: #{http.response_header.status}" }
  cleanup = Proc.new { |http| puts "\nDELETE!\n"; FileUtils.rm( f ) }
  item = Happening::S3::Item.new(
    @config[ 'portfolios_bucket' ], @subdomain + "/" + mime_dir( file ) + file,
    :aws_access_key_id => @config[ 's3_access_key' ], 
    :aws_secret_access_key => @config[ 's3_secret_key' ],
    :permissions => 'public-read' )
  item.put( File.read( f ), :on_error => on_error, :on_success => cleanup,
    :headers => { "Content-Type" => File.mime_type?( f ) } ) do |response|
  end
end

Files are correctly uploaded, but I never see "DELETE". The app is built using Goliath.io, I'm not entirely sure how it integrates with EM, but considering that it's pretty big and everything seems to work very well ... Should I call EM.stop? Suggestions on how to fix?

[jweiss]

Are you only not seeing the puts or does the cleanup also not happen? I do not know where Goliath is outputting SDTOUT.

EM.stop is not needed and would cause problems with Goliath

[ngw]

Honestly I'm a bit confused now, I don't understand the 'meaning' of on_success ...

def upload_file dir, file
  f = dir + '/' + file
  on_error = Proc.new { |http| puts "An error occured: #{http.response_header.status}" }
  item = Happening::S3::Item.new(
    @config[ 'portfolios_bucket' ], @subdomain + "/" + mime_dir( file ) + file,
    :aws_access_key_id => @config[ 's3_access_key' ], 
    :aws_secret_access_key => @config[ 's3_secret_key' ],
    :permissions => 'public-read' )
  item.put( File.read( f ), :on_error => on_error, :on_success => cleanup( f ),
    :headers => { "Content-Type" => File.mime_type?( f ) } ) do |response|
    puts "UPLOADED: #{f}"
  end
end

def cleanup file
  unless file =~ /assets/
    FileUtils.rm file
    puts "RM #{ file }"
  end    
end

Output is:

RM /Users/ngw/kenji/tmp/nofeed/css/main.css RM /Users/ngw/kenji/tmp/nofeed/css/reset.css RM /Users/ngw/kenji/tmp/nofeed/index.html RM /Users/ngw/kenji/tmp/nofeed/project.zip [46807:INFO] 2011-06-16 14:36:32 :: Status: 200, Content-Length: 2, Response Time: 279.25ms UPLOADED: /Users/ngw/kenji/tmp/nofeed/css/main.css UPLOADED: /Users/ngw/kenji/tmp/nofeed/css/reset.css UPLOADED: /Users/ngw/kenji/assets/mediator/mediator.html UPLOADED: /Users/ngw/kenji/tmp/nofeed/index.html UPLOADED: /Users/ngw/kenji/tmp/nofeed/project.zip

The files aren't removed at all .... When is on_success called?

[ngw]

Damn, and that was the result of an uncleaned dir. So, the code is suboptimal, I still need to refactor it, but I will include the full source. It downloads something from s3, process it, and upload the results on s3. The problem is that I'm apparently bitten by concurrency problems, all related to happening and on_success.

https://gist.github.com/79abf5219627d4db814e

What happens basically is that it starts processing (a processing that is entirely external) before it downloaded all files, apparently tries to cleanup before it finished uploading. I'm not able to understand if the problem is in Goliath or Happening, and I'm definitely not sure how to fix this ....

[leoc]

Is this still an issue?

I'd suggest not giving a block to the put function, because that overrides the :on_success option.