disable uniform bucket-level access for GCS buckets
Without getting into too much detail here, this is possibly a fix that's only required for Google corp users. However, it should also not negatively impact non-Googler users. Happy to chat directly for any questions.
It's hard to tell exactly where a fine grained ACL is required by butler.py create_config from the stack trace alone, but for the sake of posterity, this is what prompted the change request:
$> python3.7 butler.py create_config --oauth-client-secrets-path=$CLIENT_SECRETS_PATH --firebase-api-key=$FIREBASE_APK_KEY --project-id=$CLOUD_PROJECT_ID $CONFIG_DIR
...
Running: bower install --allow-root
| bower test-fixture extra-resolution Unnecessary resolution: test-fixture#^2.0.0
| bower webcomponentsjs extra-resolution Unnecessary resolution: webcomponentsjs#^v1.0.2
| bower iron-location extra-resolution Unnecessary resolution: iron-location#1 - 2
| bower mocha extra-resolution Unnecessary resolution: mocha#^3.1.2
| bower iron-collapse extra-resolution Unnecessary resolution: iron-collapse#^2.0.0
Installed chromedriver at: ./ENV/bin/chromedriver
Created symlink: source: /usr/local/google/home/bookholt/Documents/chrome/clusterfuzz_config, target /usr/local/google/home/bookholt/Documents/chrome/clusterfuzz/src/appengine/config.
Created symlink: source: /usr/local/google/home/bookholt/Documents/chrome/clusterfuzz/src/clusterfuzz, target /usr/local/google/home/bookholt/Documents/chrome/clusterfuzz/src/appengine/clusterfuzz.
Revision: 20220526220743-utc-747e109f-bookholt
deployment/linux-3.zip is ready.
Deploying <redacted> to prod.
Running: gsutil cp deployment/windows-3.zip gs://deployment.<redacted>.appspot.com/windows-3.zip
| Copying file://deployment/windows-3.zip [Content-Type=application/zip]...
- [1 files][ 39.2 MiB/ 39.2 MiB]
| Operation completed over 1 objects/39.2 MiB.
Running: gsutil cp deployment/macos-3.zip gs://deployment.<redacted>.appspot.com/macos-3.zip
| Copying file://deployment/macos-3.zip [Content-Type=application/zip]...
- [1 files][ 40.0 MiB/ 40.0 MiB]
| Operation completed over 1 objects/40.0 MiB.
Running: gsutil cp deployment/linux-3.zip gs://deployment.<redacted>.appspot.com/linux-3.zip
| Copying file://deployment/linux-3.zip [Content-Type=application/zip]...
- [1 files][ 35.7 MiB/ 35.7 MiB]
| Operation completed over 1 objects/35.7 MiB.
Running: gsutil cp -a public-read src/appengine/resources/clusterfuzz-source.manifest gs://deployment.<redacted>.appspot.com/clusterfuzz-source.manifest.3
| Copying file://src/appengine/resources/clusterfuzz-source.manifest [Content-Type=text/cache-manifest]...
BadRequestException: 400 Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access
| Return code is non-zero (1).
| Exit.
Traceback (most recent call last):
File "butler.py", line 319, in <module>
main()
File "butler.py", line 305, in main
command.execute(args)
File "src/local/butler/create_config.py", line 307, in execute
deploy_zips(args.new_config_dir)
File "src/local/butler/create_config.py", line 207, in deploy_zips
'--config-dir', config_dir
File "/usr/local/google/home/bookholt/.pyenv/versions/3.7.13/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', 'butler.py', 'deploy', '--force', '--targets', 'zips', '--prod', '--config-dir', '/usr/local/google/home/bookholt/Documents/chrome/clusterfuzz_config']' returned non-zero exit status 1.
Thanks! I see https://github.com/google/clusterfuzz/issues/2683 from someone else, and it seems like this is due to -a public-read in the command above. This can be safely removed, would be preferable to your PR, assuming that unblocks you.
I've uploaded https://github.com/google/clusterfuzz/pull/2687 for this.
Closing from lack of activity.