Adding ca-certificates in runtime does not work

[paulc0te]

Expected Behavior

Adding certificates at runtime must not break work of certificates added during build.

Current Behavior

The certificates that I added during build stop working after I add the certificate at runtime.

Possible Solution

If I add the same certificates only during build, then everything works

Steps to Reproduce

I'm building a spring boot application image using a gradle task that runs paketo-buildpacks. After build, the application works in k8s.

Directory with certificates when building an image

bindings/ca-certificates  
cert1.pem  
...  
cert12.pem  
type <- (ca-certificates)  

gradle task with which the application image is built

bootBuildImage {
    builder = "${dockerRepoUrl}/paketobuildpacks/builder:0.3.158-base"
    runImage = "${dockerRepoUrl}/paketobuildpacks/run:1.2.49-base-cnb"
    bindings = [
            "${project.rootDir}/bindings/ca-certificates:/platform/bindings/ca-certificates",
            "${project.rootDir}/bindings/depmapping:/platform/bindings/depmapping"
    ]
}

The command I run the build with

gradle clean bootBuildImage

Part of the Buildlog that I see when building

...
[creator]     Paketo Buildpack for CA Certificates 3.5.1
[creator]       https://github.com/paketo-buildpacks/ca-certificates
[creator]       Launch Helper: Contributing to layer
[creator]         Creating /layers/paketo-buildpacks_ca-certificates/helper/exec.d/ca-certificates-helper
[creator]       CA Certificates: Contributing to layer
[creator]         Added 12 additional CA certificate(s) to system truststore
[creator]         Writing env.build/SSL_CERT_DIR.append
[creator]         Writing env.build/SSL_CERT_DIR.delim
[creator]         Writing env.build/SSL_CERT_FILE.default
...

Attached is the full build_log

The env variable which points to the directory in which the ca-certificates directory is located

SERVICE_BINDING_ROOT: /paketo-certs/platform/bindings

Directory with certificates in a container

ls /paketo-certs/platform/bindings/ca-certificates
cert13.pem
type <- (ca-certificates)

Application start log

Added 1 additional CA certificate(s) to system truststore
Setting Active Processor Count to 12
Calculated JVM Memory Configuration: -XX:MaxDirectMemorySize=10M -XX:MaxMetaspaceSize=345042K -XX:ReservedCodeCacheSize=240M -Xss1M (Total Memory: 4G, Thread Count: 250, Loaded Class Count: 58504, Headroom: 0%)
Enabling Java Native Memory Tracking
Adding 125 container CA certificates to JVM truststore
Spring Cloud Bindings Enabled
2024-07-10T17:01:26+03:00	Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 -Djava.locale.providers=COMPAT,SPI,CLDR -Xms1024m -Xmx3200m -Djava.security.properties=/layers/paketo-buildpacks_bellsoft-liberica/java-security-properties/java-security.properties -XX:+ExitOnOutOfMemoryError -XX:ActiveProcessorCount=12 -XX:MaxDirectMemorySize=10M -XX:MaxMetaspaceSize=345042K -XX:ReservedCodeCacheSize=240M -Xss1M -XX:+UnlockDiagnosticVMOptions -XX:NativeMemoryTracking=summary -XX:+PrintNMTStatistics -Dorg.springframework.cloud.bindings.boot.enable=true
2024-07-10T17:01:36+03:00	2024-07-10 17:01:36.380 -  INFO 1 --- [main] [MDC : ] com.ats.arm.AtsArmApplication           :Starting AtsArmApplication using Java 11.0.18 on arm-bb9d9646-msvkx with PID 1 (/workspace/BOOT-INF/classes started by cnb in /workspace)
...

After the application starts, I try to use certificates, but they don't work. I am getting the following error

[BackgroundTask-0-psz-admin] [MDC : jmixUser=psz-admin] c.a.minio.filestorage.MinioFileStorage  :Error saving file to minio storage
javax.net.ssl.SSLHandshakeException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

JAVA_HOME points to the correct cacerts location

JAVA_HOME=/layers/paketo-buildpacks_bellsoft-liberica/jre

ls /layers/paketo-buildpacks_bellsoft-liberica/jre/lib/security
blocked.certs  cacerts  default.policy  public_suffix_list.dat

All certificates are recorded in cacerts and are working. I checked this by exporting and using curl.

I noticed that the certificates added in cacerts during build look like this

/layers/paketo-buildpacks_ca-certificates/ca-certificates/ca-certificates/569e89cc.0-0, Jan 1, 1980, trustedCertEntry,
Certificate fingerprint (SHA-256): 03:5B:7E:AD:50:4C:7C:17:29:50:35:54:DA:A7:14:15:87:3B:BA:49:0A:3C:60:56:42:C7:12:B6:AF:9F:DC:C1

Certificates added at runtime look like this

/tmp/ca-certificates1702841895/8b90f171.0-0, Jan 1, 1980, trustedCertEntry,
Certificate fingerprint (SHA-256): A6:14:07:AE:1D:23:6C:05:CE:9A:91:96:EE:64:5C:8E:A7:2A:BF:61:A2:08:3E:14:8D:54:E4:75:E0:32:D4:D8

I tried updating paketo-buildpacks/ca-certificates to the latest version but it did not solve the problem

Motivations

I need to use 2 of these methods - because during the build phase paketo-buildpacks accesses the https repository for its dependencies. Accordingly, he should already have a certificate. What is the reason for this behavior of paketo-buildpacks/ca-certificates ?