Why doesn't gcsfs allow projects different from the default project in google_default?

[yokomotod]

With account which has $ gcloud config get-value project # => project-a,

fs = gcsfs.GCSFileSystem(project='project-b')

raise validation error:

ValueError: User-provided project 'project-b' does not match the google default project 'project-a'. Either

  1. Accept the google-default project by not passing a `project` to GCSFileSystem
  2. Configure the default project to match the user-provided project (gcloud config set project)
  3. Use an authorization method other than 'google_default' by providing 'token=...'

However, I do want to use another project. Why gcsfs doesn't allow this?

My google_default account has correct permission to access project-b so it's no problem to access from the account. (I believe the account's default project doesn't matter.)

For option2, I use multiple fs objects for multiple projects with single google_default account, so I can't switch default project (gcloud config set project) to match.

For option3, I would like to use the Application Default Credential mechanism to make it work conveniently both on cloud and local.

[Martin4R]

I just came across the same issue. Does anyone know why gcsfs restricts it to the default-project? As a short test, I commented out the following lines and everything worked perfectly when I provide project='project-b' https://github.com/fsspec/gcsfs/blob/d97099ab40138745569fff98a098defcd7693270/gcsfs/credentials.py#L91-L93

Also the documentation of google.auth.default states:

Returns: Loaded credentials and the project ID. Authorized user credentials do not have the project ID information. External account credentials project IDs may not always be determined.

which indicates that the credentials are not bound to any project-id.

I think the function should be changed to:

def _connect_google_default(self):
        credentials, project = gauth.default(scopes=[self.scope])
        self.project = self.project or project
        self.credentials = credentials

[martindurant]

I would be happy to see a PR! This is probably only testable against real GCP as opposed to in CI, but I am prepared to take your word for it :)

[martindurant]

@Martin4R , ping