s3proxy icon indicating copy to clipboard operation
s3proxy copied to clipboard
verified publisher icon gaul

Upload not possible

Open TrueKillRob opened this issue 1 year ago • 5 comments

Hi,

I'm using the s3proxy in Azure. It runs in a container instance. The instance listens on port 8080 and 443. The certificate is created by the startup script using the commands you provided in your wiki. The storage is an Azure StorageAccount (BlobStorage).

I'm using curl and S3Browser (https://s3browser.com/) to test the proxy.

  • It works to create buckets and directories.
  • It works to delete the buckets and directories.
  • It works to download a file uploaded directly to the StorageAccount (without the proxy via the Azure portal).
  • I can't upload any files. It doesn't matter if authentication is enabled or not and if I use https or http.

I have enabled the proxy's debug mode. From the many lines it generates, I found that this might be the error:

[s3proxy] W 11-25 14:11:47.970 S3Proxy-Jetty-20 o.g.s.o.e.j.server.HttpChannel:793 |::] handleException /test2/local-identity.txt java.io.IOException: com.azure.storage.blob.models.BlobStorageException: Status Code 400, "<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidMetadata</Code><Message>The metadata provided is invalid. It contains characters that are not allowed.

My Configuration:

S3PROXY_AUTHORIZATION=none S3PROXY_CORS_ALLOW_ALL=false S3PROXY_CORS_ALLOW_CREDENTIAL= S3PROXY_CORS_ALLOW_HEADERS= S3PROXY_CORS_ALLOW_METHODS= S3PROXY_CORS_ALLOW_ORIGINS= S3PROXY_CREDENTIAL=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX X S3PROXY_ENCRYPTED_BLOBSTORE= S3PROXY_ENCRYPTED_BLOBSTORE_PASSWORD= S3PROXY_ENCRYPTED_BLOBSTORE_SALT= S3PROXY_ENDPOINT=http://0.0.0.0:8080 S3PROXY_IDENTITY=06291a9a-XXXX-XXXX-XXXX-140160e6a711 S3PROXY_IGNORE_UNKNOWN_HEADERS=false S3PROXY_KEYSTORE_P ASSWORD=XXXXXXXXXX S3PROXY_KEYSTORE_PATH=/opt/s3proxy/keystore.p12 S3PROXY_READ_ONLY_BLOBSTORE=false S3PROXY_VIRTUALHOST= JCLOUDS_AZUREBLOB_AUTH=azureKey JCLOUDS_CREDENTIAL=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX JCLOUDS_ENDPOINT=https://XXXXXXXXX XXX.blob.core.windows.net/ JCLOUDS_FILESYSTEM_BASEDIR=/data JCLOUDS_IDENTITY=XXXXXXXXXX JCLOUDS_KEYSTONE_PROJECT_DOMAIN_NAME= JCLOUDS_KEYSTONE_SCOPE= JCLOUDS_KEYSTONE_VERSION= JCLOUDS_PROVIDER=azureblob-sdk JCLOUDS_REGION= JCLOUDS_REGIONS=us-east-1

Thanks for any help Robert

TrueKillRob avatar Nov 25 '24 14:11 TrueKillRob

It appears that the S3 client added metadata with characters that Azure does not support. Consider remapping them via https://github.com/gaul/s3proxy/wiki/Middleware-user-metadata-replacer.

gaul avatar Nov 25 '24 16:11 gaul

Thank you for your reply. I have added the parameters from the link you gave me. But nothing changes :-( It could be that the tool I am using is faulty. CURL works now (without authentication). I am looking for another tool. Can you recommend an S3 tool for Windows?

TrueKillRob avatar Nov 26 '24 08:11 TrueKillRob

Can you check which characters your client is sending? You can do this by running S3Proxy with more logging:

java -DLOG_LEVEL=trace -jar /path/to/s3proxy

gaul avatar Nov 26 '24 21:11 gaul

I set it to trace but I'm to blind to se the corret line. Can you find the character? Alternative: Should I disable SSL and use WireShark on the client?

Caused by: com.azure.storage.blob.models.BlobStorageException: Status code 400, "<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidMetadata</Code><Message>The metadata specified is invalid. It has characters that are not permitted.
RequestId:a41d3f3c-101e-0001-12a3-409dff000000
Time:2024-11-27T08:11:11.9891638Z</Message></Error>"
	at com.azure.storage.blob.implementation.util.ModelHelper.mapToBlobStorageException(ModelHelper.java:483)
	at reactor.core.publisher.Mono.lambda$onErrorMap$30(Mono.java:3797)
	at reactor.core.publisher.Mono.lambda$onErrorResume$32(Mono.java:3887)
	at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94)
	... 82 common frames omitted
[s3proxy] W 11-27 08:11:11.999 S3Proxy-Jetty-18 o.g.s.o.e.j.server.HttpChannel:793 |::] handleException /test2/localidentity.txt java.io.IOException: com.azure.storage.blob.models.BlobStorageException: Status code 400, "<?xml version="1.0" encoding="utf-8"?><Error><Code>InvalidMetadata</Code><Message>The metadata specified is invalid. It has characters that are not permitted.
RequestId:a41d3f3c-101e-0001-12a3-409dff000000
Time:2024-11-27T08:11:11.9891638Z</Message></Error>"
[s3proxy] D 11-27 08:11:12.003 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] onStateChange(PUT{uri=/test2/localidentity.txt, connection=PooledConnection{channel=[id: 0xe498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443]}}, [response_completed])
[s3proxy] D 11-27 08:11:12.004 reactor-http-epoll-1 reactor.netty.ReactorNetty:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Removed handler: azureSdkHandler, pipeline: DefaultChannelPipeline{(reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
[s3proxy] D 11-27 08:11:12.007 reactor-http-epoll-1 reactor.netty.ReactorNetty:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Non Removed handler: azureSdkHandler, context: null, pipeline: DefaultChannelPipeline{(reactor.left.sslHandler = io.netty.handler.ssl.SslHandler), (reactor.left.httpCodec = io.netty.handler.codec.http.HttpClientCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
[s3proxy] D 11-27 08:11:12.007 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] onStateChange(PUT{uri=/test2/localidentity.txt, connection=PooledConnection{channel=[id: 0xe498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443]}}, [disconnecting])
[s3proxy] D 11-27 08:11:12.008 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:254 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Releasing channel
[s3proxy] D 11-27 08:11:12.008 reactor-http-epoll-1 r.n.r.PooledConnectionProvider:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Channel cleaned, now: 0 active connections, 1 inactive connections and 0 pending acquire requests.
[s3proxy] D 11-27 08:11:13.061 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:302 |::] request: Request(PUT https://s3proxy.westeurope.azurecontainer.io/test2/localidentity.txt)@737f3ad8
[s3proxy] T 11-27 08:11:13.062 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: Authorization: AWS4-HMAC-SHA256 Credential=0995ee04-0c58-4d83-ad56-de3d75dbc3ad/20241127/us-east-1/s3/aws4_request,SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date;x-amz-meta-s3b-last-modified;x-amz-meta-sha256, Signature=f1e17baa4a76707ca506611d36e471918ca994f88fac9b444fc718ab6cfb1762
[s3proxy] T 11-27 08:11:13.062 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: x-amz-content-sha256: b8e8590bad590f7fc37152f54faabbcf49c884b8ef84c8d7ed37636afe636841
[s3proxy] T 11-27 08:11:13.062 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: x-amz-meta-sha256: b8e8590bad590f7fc37152f54faabbcf49c884b8ef84c8d7ed37636afe636841
[s3proxy] T 11-27 08:11:13.063 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: x-amz-date: 20241127T081113Z
[s3proxy] T 11-27 08:11:13.063 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: x-amz-meta-s3b-last-modified: 20241125T105436Z
[s3proxy] T 11-27 08:11:13.064 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: User-Agent: S3 Browser/12.1.5 (https://s3browser.com)
[s3proxy] T 11-27 08:11:13.065 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: Host: s3proxy.westeurope.azurecontainer.io
[s3proxy] T 11-27 08:11:13.065 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: Content-Length: 34
[s3proxy] T 11-27 08:11:13.066 S3Proxy-Jetty-19 o.gaul.s3proxy.S3ProxyHandler:327 |::] header: Content-Type: text/plain
[s3proxy] D 11-27 08:11:13.085 reactor-http-epoll-1 r.n.r.PooledConnectionProvider:259 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Channel acquired, now: 1 active connections, 0 inactive connections and 0 pending acquire requests.
[s3proxy] D 11-27 08:11:13.085 reactor-http-epoll-1 r.n.h.client.HttpClientConnect:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Handler is being applied: {uri=https://blobstoragename.blob.core.windows.net/test2/localidentity.txt, method=PUT}
[s3proxy] D 11-27 08:11:13.085 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] onStateChange(PUT{uri=/test2/localidentity.txt, connection=PooledConnection{channel=[id: 0xe498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443]}}, [request_prepared])
[s3proxy] D 11-27 08:11:13.086 reactor-http-epoll-1 reactor.netty.ReactorNetty:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Added decoder [azureSdkHandler] at the end of the user pipeline, full pipeline: [reactor.left.sslHandler, reactor.left.httpCodec, azureSdkHandler, reactor.right.reactiveBridge, DefaultChannelPipeline$TailContext#0]
[s3proxy] D 11-27 08:11:13.087 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] onStateChange(PUT{uri=/test2/localidentity.txt, connection=PooledConnection{channel=[id: 0xe498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443]}}, [request_sent])
[s3proxy] D 11-27 08:11:13.101 reactor-http-epoll-1 r.n.h.c.HttpClientOperations:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Received response (auto-read:false) : RESPONSE(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 400 The metadata specified is invalid. It has characters that are not permitted.
Content-Length: <filtered>
Content-Type: <filtered>
Server: <filtered>
x-ms-request-id: <filtered>
x-ms-client-request-id: <filtered>
x-ms-version: <filtered>
x-ms-error-code: <filtered>
Date: <filtered>
[s3proxy] D 11-27 08:11:13.108 reactor-http-epoll-1 r.n.r.DefaultPooledConnectionProvider:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] onStateChange(PUT{uri=/test2/localidentity.txt, connection=PooledConnection{channel=[id: 0xe498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443]}}, [response_received])
[s3proxy] D 11-27 08:11:13.112 reactor-http-epoll-1 r.netty.channel.FluxReceive:259 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] [terminated=false, cancelled=false, pending=0, error=null]: subscribing inbound receiver
[s3proxy] D 11-27 08:11:13.113 reactor-http-epoll-1 r.n.h.c.HttpClientOperations:254 |::] [e498f3c1-7, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Received last HTTP packet
[s3proxy] T 11-27 08:11:13.113 reactor-http-epoll-1 r.n.channel.ChannelOperations:244 |::] [e498f3c1, L:/192.168.0.134:53922 - R:blobstoragename.blob.core.windows.net/52.239.213.100:443] Disposing ChannelOperation from a channel
java.lang.Exception: ChannelOperation terminal stack
	at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:485)
	at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:789)
	at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:114)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at com.azure.core.http.netty.implementation.AzureSdkHandler.channelRead(AzureSdkHandler.java:224)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
	at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
	at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1349)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1389)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:799)
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:501)
	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:399)
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Unknown Source)
[s3proxy] D 11-27 08:11:13.115 S3Proxy-Jetty-19 o.g.s.S3ProxyHandlerJetty:164 |::] Unknown exception:

TrueKillRob avatar Nov 27 '24 09:11 TrueKillRob

I disabled SSL and used WireShark to capture the upload. Since all the sesitive data is in the file, I deleted the container and the StorageAccount and created a new one with a different IP and credentials... I hope you can work with pcap files. If you want a different format, let me know.

I have also used another commercial product. It works with the same client application. Please don't get me wrong: I want to use your application. It was just a test and the commercial application uses a Maria-SQL and a WEB-GUI and Automation is not possible...

I hope you can help! Upload-Error.zip

TrueKillRob avatar Nov 27 '24 15:11 TrueKillRob