google-cloud-storage not working proper - missing settings gcsManagedLedgerOffloadBucket | gcsManagedLedgerOffloadRegion

[ton3r]

Hi,

I've set up the cluster. During configuration I see that the offloading to GCP Buckets seems not to be working. I think it relates to missing properties.

I've followed the steps to use the k8s secret. But the json inside of the pod didn't contain any data. So I tried it with

helm upgrade pulsar datastax-pulsar/pulsar \
--set storageOffload.gcsServiceAccountJsonFile=bucket-pulsar-001-topic-offload.json \
--set storageOffload.gcsServiceAccountJsonFileContent=$(cat bucket-sa/bucket-pulsar-001-topic-offload.json | base64) \
--namespace pulsar \
--values values.yaml

Now the json contains the required content. But the UI shows me :

So I connect to a broker pod and do

I have no name!@pulsar-broker-677dbffbdc-jbkzw:/pulsar$ env | grep -i off
gcsManagedLedgerOffloadBucket=
managedLedgerOffloadDriver=google-cloud-storage
gcsManagedLedgerOffloadRegion=
gcsManagedLedgerOffloadServiceAccountKeyFile=/pulsar/gcp-service-account/bucket-pulsar-001-topic-offload.json

Values for gcsManagedLedgerOffloadBucket and gcsManagedLedgerOffloadRegion are not set. So I tried with

--set storageOffload.gcsManagedLedgerOffloadBucket=pulsar-001-topic-offload 
--set storageOffload.gcsManagedLedgerOffloadRegion=europe-west4 

but this is also not taken.

How to fix this?

[ton3r]

After adding the values manual to config-map pulsar-broker the offloading takes place. A manual "Offload" on topic level offloades the things.

But the "False" from the screenshot in the "Namespace" Details is still there.