Using an s3 compatible storage - fails to connect on error upload Failed reqwest:

[sanasz91mdev]

Why is my request failing in dump handler pods:

[2024-01-04T06:45:20Z INFO  core_dump_agent] INotify Starting...
[2024-01-04T06:45:20Z INFO  core_dump_agent] INotify Initialised...
[2024-01-04T06:45:20Z INFO  core_dump_agent] INotify watching : /var/mnt/core-dump-handler/cores
[2024-01-04T06:45:35Z INFO  core_dump_agent] Uploading: /var/mnt/core-dump-handler/cores/0f4c0a44-58d3-46db-9c83-5fb7f2aa45fe-dump-1704350735-segfaulter23-segfaulter-1-4.zip
[2024-01-04T06:45:35Z INFO  core_dump_agent] zip size is 29122
[2024-01-04T06:45:35Z ERROR core_dump_agent] Upload Failed reqwest: error sending request for url (https://68.218.153.165/dumps-bucket/0f4c0a44-58d3-46db-9c83-5fb7f2aa45fe-dump-1704350735-segfaulter23-segfaulter-1-4.zip): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (self signed certificate)

y does it fail to uplaoad to my valid s3 compatible storage? i can upload things via S3 browser on same URL

[No9]

HI @sanasz91mdev Self signed certs are currently not supported as the underlying S3 library only supports them as a build definition rather than a runtime definition https://github.com/durch/rust-s3/blob/32a5a69363cb74c86ed8b68f713ca14d8f5a2798/s3/src/request.rs#L58C28-L58C41

If you must have support for self signed certs can I suggest building a version of the agent using this Dockerfile https://github.com/IBM/core-dump-handler/blob/main/Dockerfile

with the "no-verify-ssl" feature enabled here https://github.com/IBM/core-dump-handler/blob/main/core-dump-agent/Cargo.toml#L21-L29

Then modifying the chart to use your self signed cert enabled image https://github.com/IBM/core-dump-handler/blob/main/charts/core-dump-handler/values.yaml#L4-L6

Hope this helps

[sanasz91mdev]

@No9 i did the exact steps ... i am still getting error:

[2024-01-04T14:39:44Z ERROR core_dump_agent] Upload Failed reqwest: error sending request for url (https://flexifyapp.australiaeast.cloudapp.azure.com/dumps-bucket/a03b4dfe-0761-4105-bb76-89e80f7a4a68-dump-1704376803-segfaulter51-segfaulter-1-4.zip): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (self signed certificate)

[No9]

That's strange - it looks like it is passed through the reqwest layer now but is getting thrown out by OpenSSL. This is surprising as we should be using rstls Can you confirm the agents Cargo.toml?

[sanasz91mdev]

That's strange - it looks like it is passed through the reqwest layer now but is getting thrown out by OpenSSL. This is surprising as we should be using rstls Can you confirm the agents Cargo.toml?

yup i edited agent's Cargo.toml

[target.x86_64-unknown-linux-musl.dependencies.rust-s3]
version = "0.31.0"
default-features = false
features = ["no-verify-ssl"]

[target.aarch64-unknown-linux-musl.dependencies.rust-s3]
version = "0.31.0"
default-features = false
features = ["no-verify-ssl"]

[No9]

Ah that explains why you were getting the SSL errors. If you are using musl you need to keep the "tokio-rustls-tls" features too Not sure if you missed the standard build so including it for completeness.

[target.x86_64-unknown-linux-musl.dependencies.rust-s3]
version = "0.31.0"
default-features = false
features = ["tokio-rustls-tls", "no-verify-ssl"]

[target.aarch64-unknown-linux-musl.dependencies.rust-s3]
version = "0.31.0"
default-features = false
features = ["tokio-rustls-tls", "no-verify-ssl"]

[target.x86_64-unknown-linux-gnu.dependencies.rust-s3]
version = "0.31.0"
features = ["no-verify-ssl"]

[sanasz91mdev]

got fixed with this ... thanks. can you also tell why it does not work with self signed certificates ... i even used a non self signed certificate today but it gave error @No9 ...

(https://20.253.24.196/dumps-bucket/cd025bb8-d1af-4cbf-ade3-71e91d2c2bcc-dump-1704735065-segfaulter-segfaulter-1-4.zip): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (unable to get local issuer certificate) [2024-01-08T17:31:21Z INFO core_dump_agent] Uploading: /var/mnt/core-dump-handler/cores/694eaf1b-15e6-402c-bb7a-e8e20dad4c00-dump-1704735081-segfaulter2-segfaulter-1-4.zip [2024-01-08T17:31:21Z INFO core_dump_agent] zip size is 29716 [2024-01-08T17:31:21Z ERROR core_dump_agent] Upload Failed reqwest: error sending request for url (https://20.253.24.196/dumps-bucket/694eaf1b-15e6-402c-bb7a-e8e20dad4c00-dump-1704735081-segfaulter2-segfaulter-1-4.zip): error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1915: (unable to get local issuer certificate)

Why is this so? ...

Eventually i had to build a new image with ssl-no-verify

[No9]

ssl-no-verify isn't currently supported because of the static build item mentioned in this comment. https://github.com/IBM/core-dump-handler/issues/152#issuecomment-1876734976