netbox-chart icon indicating copy to clipboard operation
netbox-chart copied to clipboard

StorageBackends not working with GoogleCloudStorage

Open pimvandenbroek opened this issue 1 year ago • 2 comments

The Helm chart version

netbox-5.0.0-beta.132

Environment Versions

Kubernetes 1.29
GKE

Custom chart values

# Use Google Cloud Storage backend
storageBackend: storages.backends.gcloud.GoogleCloudStorage
# Provide the necessary configuration for GCS
storageConfig:
  GS_BUCKET_NAME: 'your-gcs-bucket-name'
  GOOGLE_APPLICATION_CREDENTIALS: '/path/to/your/credentials.json'

Current Behavior & Steps to Reproduce

  • Create a storage bucket
  • Create a serviceaccount
  • Mount the serviceaccount in the deployment to '/path/to/your/credentials.json'
  • Add the config above

When uploading a file, you'll get:

There was a problem with your request. Please contact an administrator.

The complete exception is provided below:

<class 'google.api_core.exceptions.Forbidden'>

403 POST https://storage.googleapis.com/upload/storage/v1/b/your-gcs-bucket-name/o?uploadType=multipart: {
  "error": {
    "code": 403,
    "message": "Access denied.",
    "errors": [
      {
        "message": "Access denied.",
        "domain": "global",
        "reason": "forbidden"
      }
    ]
  }
}
: ('Request failed with status code', 403, 'Expected one of', <HTTPStatus.OK: 200>)

Python version: 3.12.3
NetBox version: 4.1.4
Plugins: None installed
If further assistance is required, please post to the NetBox discussion forum on GitHub.

Expected Behavior

I expect that files were uploaded to GCS.

Multiple tests were done:

  • Local test using export GOOGLE_APPLICATION_CREDENTIALS='/path/to/your/credentials.json', and then using google.cloud python package to upload a file. This worked fine
  • Jumped into the netbox-pod, and did the same. This also worked fine
  • Created a django app, purely for uploading a file to gcloud storage. I had to add GOOGLE_APPLICATION_CREDENTIALS instead of GS_CREDENTIALS. After that file upload worked perfectly.

NetBox Logs

No response

pimvandenbroek avatar Oct 23 '24 13:10 pimvandenbroek

When using GS_CREDENTIALS: service_account.Credentials.from_service_account_file('/path/to/credentials.json') I see this error:

Server Error
There was a problem with your request. Please contact an administrator.

The complete exception is provided below:

<class 'ValueError'>

This library only supports credentials from google-auth-library-python. See https://google-auth.readthedocs.io/en/latest/ for help on authentication with this library.

Python version: 3.12.3
NetBox version: 4.1.4
Plugins: None installed

D1StrX avatar Oct 23 '24 13:10 D1StrX

Thanks for filing this issue, @pimvandenbroek.

  • Jumped into the netbox-pod, and did the same. This also worked fine
  • Created a django app, purely for uploading a file to gcloud storage. I had to add GOOGLE_APPLICATION_CREDENTIALS instead of GS_CREDENTIALS. After that file upload worked perfectly.

As far as I can understand from here, this is not related to the Helm chart per se, right? If so, I'd recommend opening a case to the related project, NetBox in that case, I guess.

LeoColomb avatar Oct 23 '24 17:10 LeoColomb

I partially agree. I'm using the provided options given by the Helm-chart, which aren't working as intended. And of course I understand it is still in beta, but that's exactly why we're trying to help by submitting issues.

I'm currently unable to setup a complete working Netbox instance (that's why we chose the helm-chart). For now I will close this issue, until I have some time to properly setup a Netbox instance.

pimvandenbroek avatar Oct 24 '24 06:10 pimvandenbroek