s3-sftp-proxy icon indicating copy to clipboard operation
s3-sftp-proxy copied to clipboard

Support for Folders (that do not exist)

Open Morl99 opened this issue 6 years ago • 0 comments

We have a Backup Script that tries to backup to sftp, and we are using this proxy to get the files to s3. While this works quite well, we have a specific setup that seems to fail.

The client tries to access a folder (in the logs you will see that this is /cassandra) and the proxy tries to fetch this folder from s3. S3 returns a 404, and for me it looks like, the proxy now crashes/throws an error.

time="2018-09-18T15:09:54Z" level=info msg="connected from client 10.192.16.1:41664"
time="2018-09-18T15:09:54Z" level=info msg="user user01 logged in"
time="2018-09-18T15:09:54Z" level=info msg="channel: session"
2018/09/18 15:09:54 list objects method STAT
2018/09/18 15:09:54 Session credentials: &{creds:{AccessKeyID: SecretAccessKey: SessionToken: ProviderName:} forceRefresh:true m:{w:{state:0 sema:0} writerSem:0 readerSem:0 readerCount:0 readerWait:0} provider:0xc4201bea20}
2018/09/18 15:09:54 Get Session Credentials Provider: EC2RoleProvider
2018/09/18 15:09:54 awsCfg credentials: &{creds:{AccessKeyID:ASIAZJISBX7TTO7PFKVQ SecretAccessKey:uG0BXmkVUyNNOdmgc5OEeC7ReBOCk8dBD0/d0raM SessionToken:FQoGZXIvYXdzEKD//////////wEaDPOzTMi9wev1scKFoCKkAovP9ohQ2wnnhon2ZxVATWipma3Aaxm3I5EfQEz+Fwd8aKY4KJyvRXpuMO9LY3G1uIv7WRSAKRy/rgbjTjCXTYVGJQrNCGpQIrNUR56gX9318jHIxOKoWynM0a+BJC2RD/75Xties3p97+npzgFkPHpXQy4oqMajpC4ppxkHeybTTNPl6TpKd9kLb/Bl75JiDnFcxms7FAofAppCeNPlHRj46crjpV0Ldy3UrKtA+WjXdYDCuMHiWtSKFbCzr3M7xJnDHGjQ4JF2z47HSJmvn99GiIJ0RuYmxdvYDb6/tzixh/o+SQHzu7R+7eaMdxg9+V8S6IKj9pGg91VFdkOKCBmrLFa2LBoHbm9n/56n40Rs9MH5DO1juY9ye+fPPoSXLdlNF9sog6aE3QU= ProviderName:EC2RoleProvider} forceRefresh:false m:{w:{state:0 sema:0} writerSem:0 readerSem:0 readerCount:0 readerWait:0} provider:0xc4201bea20}
2018/09/18 15:09:54 Current Creds Provider: EC2RoleProvider
2018/09/18 15:09:54 HEAD return: {

}
time="2018-09-18T15:09:54Z" level=debug msg="S3ObjectStat.ListAt: len(result)=1 offset=0"
time="2018-09-18T15:09:54Z" level=debug msg="GetObjectAclWithContext(Bucket=bucketname, Key=cassandra)"
time="2018-09-18T15:09:54Z" level=debug msg="GetObjectAclWithContext(Bucket=bucketname, Key=cassandra) with error: NoSuchKey: The specified key does not exist.\n\tstatus code: 404, request id: F3F7A22A3B327A58, host id: 2mdjuqaGga9AjhJe3cFVCZChlqJ4YjmJ0ec8EQS4MOPrrFgJzbZdsaK89nYmRqW7Hd/KIpfTmD8="
time="2018-09-18T15:09:54Z" level=debug msg="=> NoSuchKey: The specified key does not exist.\n\tstatus code: 404, request id: F3F7A22A3B327A58, host id: 2mdjuqaGga9AjhJe3cFVCZChlqJ4YjmJ0ec8EQS4MOPrrFgJzbZdsaK89nYmRqW7Hd/KIpfTmD8="
time="2018-09-18T15:09:54Z" level=debug msg="ListObjectsV2WithContext(Bucket=bucketname, Prefix=cassandra)"
time="2018-09-18T15:09:55Z" level=debug msg="=> <nil>"
time="2018-09-18T15:09:55Z" level=debug msg="HandleChannel.serve ended"
time="2018-09-18T15:09:55Z" level=debug msg="HandleChannel.discardRequest ended"
time="2018-09-18T15:09:55Z" level=debug msg="HandleChannel ended"
time="2018-09-18T15:09:55Z" level=debug msg="HandleClient.requestHandler ended"
time="2018-09-18T15:09:55Z" level=debug msg="HandleClient.channelHandler ended"
time="2018-09-18T15:09:55Z" level=info msg="connection from client 10.192.16.1:41664 closed"
time="2018-09-18T15:09:55Z" level=debug msg="HandleClient ended"

This should probably be handled a little better, although I admit, that I am not 100% sure, what the sftp client is issuing.

What is the expected behavior of accessing a folder that does not exist at the moment? Does it behave like a real sftp server in this case?

Morl99 avatar Sep 18 '18 16:09 Morl99