boto3 >= 1.36 causes issues with some s3 providers

Hello,

we're using barman through https://cloudnative-pg.io/plugin-barman-cloud. When upgrading from 0.4.1 to 0.5.0, barman version bumped from 3.13.3 to 3.14.0.

One changes is to remove pining on boto3 for version <= 1.46 (BAR-637).

Several S3 provider are actually not compatible with this version (see Outscale for information but there are others) and then barman 1.34 cannot work with them :/

according to AWS, we can workaround it by setting two environment variables:

export AWS_REQUEST_CHECKSUM_CALCULATION=when_required
export AWS_RESPONSE_CHECKSUM_VALIDATION=when_required

I've tested it with plugin-barman-cloud (for information, this is how I did:

apiVersion: barmancloud.cnpg.io/v1
kind: ObjectStore
metadata:
  name: my-os
spec:
  configuration:
    data:
      compression: bzip2
    destinationPath: s3://backup/
    endpointURL: https://my-old-s3-service
    s3Credentials:
      accessKeyId:
        key: S3_ACCESS_KEY
        name: s3-backup-pg
      secretAccessKey:
        key: S3_SECRET_KEY
        name: s3-backup-pg
    wal:
      compression: snappy
  instanceSidecarConfiguration:
    env:
    - name: AWS_REQUEST_CHECKSUM_CALCULATION
      value: when_required
    - name: AWS_RESPONSE_CHECKSUM_VALIDATION
      value: when_required

)

maybe it would be useful to add it in the documentation this workaround as all people using barman with "old" s3 services may be impacted

regards

Thanks, Sylvain, for the tip. We'll add something to the documentation.

Have you also shared this with the CNPG team?

CC: @mnencia

Hi Martin, not for the moment, I wanted first to have your feedback here and then do roughly the same issue on their side

Currently facing this issue with BackBlaze B2

postgres {"level":"info","ts":"2025-06-19T11:00:36.273447343Z","logger":"barman-cloud-wal-archive","msg":"2025-06-19 11:00:36,270 [1531] ERROR: Barman cloud WAL archiver exception: An error occurred (InvalidArgument) when calling the PutObject operation: Unsupporte │
│ d header 'x-amz-sdk-checksum-algorithm' received for this API call.}

Setting the env vars worked:

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: cluster-pg
spec:
  instances: 1
  imageCatalogRef:
    apiGroup: postgresql.cnpg.io
    kind: ClusterImageCatalog
    name: postgresql
    major: 14
  storage:
    pvcTemplate:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 20Gi
      storageClassName: nfs-main
      volumeMode: Filesystem
  env:
    - name: AWS_REQUEST_CHECKSUM_CALCULATION
      value: when_required
    - name: AWS_RESPONSE_CHECKSUM_VALIDATION
      value: when_required

We are going to document this, but we want it be clear that it's a tip, and we don't guarantee it will work. Just need to pick the right wording 😄