retina
retina copied to clipboard
GCP Bucket Support through S3 compatible - SignatureDoesNotMatch 403 Error
Describe the bug As S3 compatible buckets are supported, I expected that GCP Buckets would also work. Tutorial
But uppon implementing, we receive the following error:
level=error caller=outputlocation/s3.go:111 msg="Couldn't upload file" srcFilePath=/tmp/capture-node-blobupload-gke-nodepool-1-sdfa234-4445-20240902052745UTC.tar.gz bucketName=tmp-packet-captures objectKey=retina/captures/tmp/capture-node-blobupload-gke-nodepool-1-sdfa234-4445-20240902052745UTC.tar.gz error="failed to upload file to S3: operation error S3: PutObject, https response error StatusCode: 403, RequestID: , HostID: , api error SignatureDoesNotMatch: Access denied."
To Reproduce Version v0.0.14 through Helm Chart and CRD
- Follow all the steps on tutorial, configure the HMAC key, region and secret
- Add IAM permission for Bucket write access
- Initialize Capture
Expected behavior Upload to GCP Bucket should just work.
Platform (please complete the following information):
- OS: Linux
- Kubernetes Version: v1.28.11
- Host: GCP
- Retina Version: V0.0.14 Through Helm Chart
Additional context
I found this Stackoverflow link, seems like the issue is due to the v2 library: According to https://github.com/aws/aws-sdk-go-v2/issues/1816 the v2 SDK doesn't support GCP GCS.
Not sure how to fix, but would be a great plus if it just worked.
Configuration file:
apiVersion: retina.sh/v1alpha1
kind: Capture
metadata:
name: capture-node-blobupload
spec:
captureConfiguration:
captureOption:
duration: 10s
captureTarget:
nodeSelector:
matchExpressions:
- {
key: kubernetes.io/hostname,
operator: In,
values: [gke-nodepool-1-sdfa234-4445],
}
outputConfiguration:
hostPath: "/tmp/retina"
s3Upload:
endpoint: https://storage.googleapis.com
bucket: tmp-packet-captures
region: auto # I also tried us-central1 for clear of mind.
path: retina/captures
secretName: capture-s3-upload-secret
---
apiVersion: v1
kind: Secret
metadata:
name: capture-s3-upload-secret
stringData:
s3-access-key-id: <ID>
s3-secret-access-key: <SECRET>