solr-operator icon indicating copy to clipboard operation
solr-operator copied to clipboard

Backup not working with Service accounts using Web identity Tokens

Open Joshuaariolu opened this issue 8 months ago • 5 comments

I have noticed that the backup functionality does not work when I use a service account with web identity token. I get the error message below.

ERROR (OverseerThreadFactory-54-thread-3) [c:Collection s: r: x: t:] o.a.s.c.a.c.CollectionHandlingUtils Collection Collection, operation backup failed => org.apache.solr.s3.S3Exception: An AmazonServiceException was thrown! [serviceName=S3] [awsRequestId=56HQCK517V8WD2EE] [httpStatus=403] [s3ErrorCode=null] [message=null] I also get a 400 error on some occassions.

I am currently running on solr version 9.8.1 and solr operator v0.8.1

I saw a similar thread on https://github.com/apache/solr-operator/issues/475 and apparently it has been resolved on versions 8.11 and higher.

Is there any part of the back up process I may be missing when working with Web Identity token?

Joshuaariolu avatar May 13 '25 22:05 Joshuaariolu

Required parameters parsed on values.yaml

solrModules: ["scripting","s3-repository"]

serviceAccountName: "solr-svc-acct"

backupRepositories: - name: "s3-backups-1" s3: region: "us-east-1" bucket: "dev-main-ue1"

Backup definition apiVersion: solr.apache.org/v1beta1 kind: SolrBackup metadata: name: local-backup spec: repositoryName: "s3-backups-1" solrCloud: solr collections: - Collection recurrence: # Store one backup daily, and keep a week at a time. schedule: "@every 5m" maxSaved: 7

Joshuaariolu avatar May 13 '25 22:05 Joshuaariolu

Make sure to whitelist the path of the token. -Dsolr.allowPaths=/var/run/secrets/eks.amazonaws.com/serviceaccount/token

https://github.com/apache/solr-operator/issues/775

elangelo avatar May 15 '25 08:05 elangelo

Thanks for the response @elangelo Please where did you define that in your values.yaml?

Joshuaariolu avatar May 15 '25 20:05 Joshuaariolu

I'm not using the helm chart but i think solrOptions.javaOpts should be set. something like --set solrOptions.javaOpts="-Dsolr.allowPaths=/var/run/secrets/eks.amazonaws.com/serviceaccount/token" if you want to directly put it in the helm install command

elangelo avatar May 16 '25 06:05 elangelo

Thanks alot @elangelo that worked for me

Joshuaariolu avatar May 16 '25 13:05 Joshuaariolu