[BUG] S3.client settings missing from list of repository-s3 settings
Describe the bug
I have a problem right now using Oracled Bucket with the remote storage
Exception occurred while uploading remote index paths for
indexes=[top_queries-2025.11.30-85662/OlMiRO3eQDmMREEf0_gBQg],[.ql-datasou....e3-15363/hCmCFo-gS22nIaqfCUnpnA]
Suppressed: org.opensearch.core.common.io.stream.NotSerializableExceptionWrapper: completion_exception:
software.amazon.awssdk.services.s3.model.S3Exception: AWS chunked encoding not supported. (Service: S3, Status Code: 501,
Request ID: fra-1:qd07EryGbB2TqNOyZ_YhB5w9EK5Kz34JZtDo_U12-cEFjiJFUr6x67MWO8DUoSrB) (SDK Attempt Count: 1)
Caused by: org.opensearch.core.common.io.stream.NotSerializableExceptionWrapper: s3_exception: AWS chunked encoding not
supported. (Service: S3, Status Code: 501, Request ID:
fra-1:qd07EryGbB2TqNOyZ_YhB5w9EK5Kz34JZtDo_U12-cEFjiJFUr6x67MWO8DUoSrB) (SDK Attempt Count: 1)
at software.amazon.awssdk.services.s3.model.S3Exception$BuilderImpl.build(S3Exception.java:113) ~[?:
at software.amazon.awssdk.services.s3.model.S3Exception$BuilderImpl.build(S3Exception.java:61) ~[?:
So I tried to disable the encoding as per the setting here in my opensearch.yml (helm chart)
https://github.com/opensearch-project/OpenSearch/blob/b7f013f27f18fea78b1aecd90dfbd58ecbfde722/plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3ClientSettings.java#L246
but got an error
SettingsException[unknown setting [s3.client.default.disable_chunked_encoding] please check that any required plugins are installed, or check the breaking changes documentation for removed settings]
it seems similar to the problem fixed by
https://github.com/opensearch-project/OpenSearch/pull/19788
Is it possible to expose all settings available in the client from the opensearch.yml file as tehre is no other way to modify them with the helm chart ?
Related component
Plugins
To Reproduce
- add s3.client.default.disable_chunked_encoding: true to your opensearch.yml
Expected behavior
Clean start of the app
Additional Details
Plugins repository-s3
Host/Environment (please complete the following information):
- Kubernetes 1.34.1
- Helm chart version 3.3.2
Hi , I’d like to work on this issue as a first contribution.
What I understand so far:
The setting s3.client.
What I plan to do: Add DISABLE_CHUNKED_ENCODING into the CLIENT_SETTINGS list so it becomes a valid client-setting. Update documentation to reflect that this can be configured via opensearch.yml .
If this approach looks good to maintainers/owners of the repository-s3 plugin, I will submit a PR soon. Thanks!
Imho it would maybe benefit from a larger fix and expose all the settings currently missing ( unless there is a problem with modifying one of them )
- REQUEST_TIMEOUT_SETTING
- CONNECTION_TIMEOUT_SETTING
- CONNECTION_TTL_SETTING
- MAX_CONNECTIONS_SETTING
- MAX_SYNC_CONNECTIONS_SETTING
- CONNECTION_ACQUISITION_TIMEOUT
- MAX_PENDING_CONNECTION_ACQUIRES
- DISABLE_CHUNKED_ENCODING