retina icon indicating copy to clipboard operation
retina copied to clipboard

GCP Bucket Support through S3 compatible - SignatureDoesNotMatch 403 Error

Open MasterExecuson opened this issue 5 months ago • 0 comments

Describe the bug As S3 compatible buckets are supported, I expected that GCP Buckets would also work. Tutorial

But uppon implementing, we receive the following error:

level=error caller=outputlocation/s3.go:111 msg="Couldn't upload file" srcFilePath=/tmp/capture-node-blobupload-gke-nodepool-1-sdfa234-4445-20240902052745UTC.tar.gz bucketName=tmp-packet-captures objectKey=retina/captures/tmp/capture-node-blobupload-gke-nodepool-1-sdfa234-4445-20240902052745UTC.tar.gz error="failed to upload file to S3: operation error S3: PutObject, https response error StatusCode: 403, RequestID: , HostID: , api error SignatureDoesNotMatch: Access denied."

To Reproduce Version v0.0.14 through Helm Chart and CRD

  1. Follow all the steps on tutorial, configure the HMAC key, region and secret
  2. Add IAM permission for Bucket write access
  3. Initialize Capture

Expected behavior Upload to GCP Bucket should just work.

Platform (please complete the following information):

  • OS: Linux
  • Kubernetes Version: v1.28.11
  • Host: GCP
  • Retina Version: V0.0.14 Through Helm Chart

Additional context I found this Stackoverflow link, seems like the issue is due to the v2 library: According to https://github.com/aws/aws-sdk-go-v2/issues/1816 the v2 SDK doesn't support GCP GCS.

Not sure how to fix, but would be a great plus if it just worked.

Configuration file:


apiVersion: retina.sh/v1alpha1
kind: Capture
metadata:
  name: capture-node-blobupload
spec:
  captureConfiguration:
    captureOption:
      duration: 10s
    captureTarget:
      nodeSelector:
        matchExpressions:
          - {
              key: kubernetes.io/hostname,
              operator: In,
              values: [gke-nodepool-1-sdfa234-4445],
            }
  outputConfiguration:
    hostPath: "/tmp/retina"
    s3Upload:
      endpoint: https://storage.googleapis.com
      bucket: tmp-packet-captures
      region: auto # I also tried us-central1 for clear of mind.
      path: retina/captures
      secretName: capture-s3-upload-secret
---
apiVersion: v1
kind: Secret
metadata:
  name: capture-s3-upload-secret
stringData:
  s3-access-key-id: <ID>
  s3-secret-access-key: <SECRET>

MasterExecuson avatar Sep 02 '24 17:09 MasterExecuson