s3sync sync shouldn't append source directory path into target path

[krunk4ever]

I figured out why s3sync was always uploading everything every time. It turns out that it was appending the source directory path into the target path.

e.g. if I run:

s3sync sync source aws.example.com:target

• source/README.txt => aws.example.com:target/source/README.txt

Using the full path doesn't work either:

s3sync sync /Users/guest/source aws.example.com:target

• /Users/guest/source/README.txt => aws.example.com:target/Users/guest/source/README.txt

Using the current working directory doesn't work either:

s3sync sync . aws.example.com:target

• ./README.txt => aws.example.com:target/./README.txt

I thought the above would work, but apparently target/README.txt is a different path than target/./README.txt

Any guidance would be appreciated.

[krunk4ever]

After looking at the source code, I was able to determine that by appending / after the source path, it wouldn't be appended to the destination path.

There are still 2 problems with the source path:

• If there's no trailing /, only the folder name should be appended:

s3sync sync /Users/guest/source aws.example.com:target would

• /Users/guest/source/README.txt => aws.example.com:target/source/README.txt

• s3sync doesn't know how to handle when ./ is passed in as the source giving the following error:

/Users/guest/.rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/find.rb:40:in `block in find': No such file or directory (Errno::ENOENT)

That was why I had initially dropped the trailing /

[clarete]

Thanks for the investigation @krunk4ever! That's a pretty annoying/serious bug! However, I don't think I'll have time to fix that in the source code for now. If you're interested in fixing this problem, I'd definitely be able to help you in anything you might need, including applying the patch! Otherwise this bug will have to wait a little bit to be closed!

Thank you so much!

[clarete]

I was debugging something close to this issue today and I fixed another occurrence of this ENOENT error. Do you have the full traceback? Thanks! :)

[krunk4ever]

Here's the full stacktrace:

/Users/thon/.rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/find.rb:40:in `block in find': No such file or directory (Errno::ENOENT)
from /Users/thon/.rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/find.rb:40:in `collect!'
from /Users/thon/.rvm/rubies/ruby-2.0.0-p353/lib/ruby/2.0.0/find.rb:40:in `find'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/gems/s3sync-2.0.0/lib/s3sync/sync.rb:114:in `list_files'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/gems/s3sync-2.0.0/lib/s3sync/sync.rb:298:in `read_trees'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/gems/s3sync-2.0.0/lib/s3sync/sync.rb:171:in `run'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/gems/s3sync-2.0.0/lib/s3sync/cli.rb:397:in `run'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/gems/s3sync-2.0.0/lib/s3sync/cli.rb:456:in `block (2 levels) in run'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/gems/cmdparse-2.0.5/lib/cmdparse.rb:464:in `parse'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/gems/s3sync-2.0.0/lib/s3sync/cli.rb:469:in `run'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/gems/s3sync-2.0.0/bin/s3sync:66:in `<top (required)>'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/bin/s3sync:23:in `load'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/bin/s3sync:23:in `<main>'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/bin/ruby_executable_hooks:15:in `eval'
from /Users/thon/.rvm/gems/ruby-2.0.0-p353@ota/bin/ruby_executable_hooks:15:in `<main>'

[clarete]

It is for sure the bug that I fixed in the last commit debugging your first message! Now we just have to deal with the slash craziness in the function that builds the source/target paths. Thanks!

[krunk4ever]

No, thank you!

[clarete]

Paths are still a problem, as we can see in the issue #19. I closed the oldest issue because we made more progress here, but both could be addressed together.