Cannot mount GCS bucket

[luisremis]

I have been using goofys with AWS S3 without issues, working great.

When I try to run experiments on GCS, I am not able to mount the bucket.

This is the commands I tried:

luis@server:$ goofys -o allow_other --uid=1001 --gid=1002 --dir-mode=0755 --file-mode=0644 --debug_fuse --profile=gcs -f --endpoint https://storage.googleapis.com  goofys-testing-us-west-2 aperturedb/db/images
2022/11/04 21:38:46.597680 s3.ERROR code=NoCredentialProviders msg=no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors, err=<nil>

2022/11/04 21:38:46.597754 main.ERROR Unable to access 'goofys-testing-us-west-2': NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2022/11/04 21:38:46.597767 main.FATAL Mounting file system: Mount: initialization failed

luis@server:/$ goofys -o allow_other --debug_fuse --profile=gcs -f --endpoint https://storage.googleapis.com  goofys-testing-us-west-2 aperturedb/db/images
2022/11/04 21:41:02.043650 s3.ERROR code=NoCredentialProviders msg=no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors, err=<nil>

2022/11/04 21:41:02.043751 main.ERROR Unable to access 'goofys-testing-us-west-2': NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2022/11/04 21:41:02.043764 main.FATAL Mounting file system: Mount: initialization failed

What am I missing?

[kahing]

could you try the GCS backend instead? goofys ... gs:/goofys-testing-us-west-2 (no need to pass --endpoint)

[govekk]

Hi, I'm having the same errors when trying to use goofys with GCS, with either version of the command:

govekk@server$ ./goofys --profile=gcs -f --endpoint https://storage.googleapis.com omero-bucket omero-bucket-here
2023/02/06 11:35:01.490860 s3.ERROR code=NoCredentialProviders msg=no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors, err=<nil>

2023/02/06 11:35:01.490970 main.ERROR Unable to access 'omero-bucket': NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2023/02/06 11:35:01.490984 main.FATAL Mounting file system: Mount: initialization failed

govekk@server$ ./goofys -f gs://omero-bucket omero-bucket-here
2023/02/06 11:35:12.946584 s3.ERROR Unable to access 'gs': invalid argument
2023/02/06 11:35:15.952680 s3.ERROR code=NoCredentialProviders msg=no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors, err=<nil>

2023/02/06 11:35:15.952712 main.ERROR Unable to access 'gs': NoCredentialProviders: no valid providers in chain. Deprecated.
	For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2023/02/06 11:35:15.952724 main.FATAL Mounting file system: Mount: initialization failed

I get the same errors either on a Google Compute VM where I have given "full access to all Cloud APIs" to the default service account, or on a separate CentOS 7 Linux machine where I use gcloud auth application-default login with my user account.

[govekk]

As an update, I got the old command (with --profile=gcs --endpoint https://storage.googleapis.com) working by creating access keys for service accounts under Cloud Storage > Settings > Interoperability on the Google Cloud console, and setting those as the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

The gs:// command still does not work, I believe because the most recent Linux binaries available under Releases do not contain the update that allows goofys to work with gs://.

[davegold24]

I've tested gs:// on both RHEL 8.7 and CentOS 7.9 The hardest part was to figure out the rights.

I seeyou used: ./goofys -f gs://omero-bucket omero-bucket-here The last argument should be a mount point (/mnt/gctest)

Also, use --debug_s3 and --debug_fuse to get better output, if you didn't try that already.

This pointed me to make sure the json is in /root/.config/gcloud/application_default_credentials.json

[luisremis]

As an update, I got the old command (with --profile=gcs --endpoint https://storage.googleapis.com) working by creating access keys for service accounts under Cloud Storage > Settings > Interoperability on the Google Cloud console, and setting those as the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables.

thanks @govekk, that really help.

After creating access keys for service accounts under Cloud Storage > Settings > Interoperability, this is what worked for me:

export AWS_ACCESS_KEY_ID=GOOGXXXXXXXXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXX
goofys -o allow_other --profile=gcs --endpoint https://storage.googleapis.com my-bucket local-folder