google
[FR] Migrate from legacy ACLs to uniform bucket-level access control
Use of legacy fine grained ACLs is being actively discouraged for use by Googlers in favor of uniform bucket-level access control.
PR https://github.com/google/clusterfuzz/pull/2682 was needed to disable uniform bucket-level access control for projects where it is enabled.
This is a tracker/reminder to evaluate the cost of migrating to UBLA.
I took a very quick look to see specifically what CF does that requires the legacy ACLs. There is probably behavior that I missed, but these two cases were apparent:
1: Creating buckets: https://github.com/google/clusterfuzz/search?q=create_bucket
Buckets can probably be created with UBLA enabled, otherwise creation will fail when fine grained ACLs are prohibited by org security policy.
2: Setting ACLs: https://github.com/google/clusterfuzz/search?q=acl
Refactoring will be necessary where CF current tries to set fined grained ACLs. However, this query shows only setting ACLs to a publich-ish value, which seem pretty compatible with UBLA.
PR https://github.com/google/clusterfuzz/pull/2682 was needed to disable uniform bucket-level access control for projects where it is enabled.
Is this still needed to unblock you after https://github.com/google/clusterfuzz/pull/2687 ?
2: Setting ACLs: https://github.com/google/clusterfuzz/search?q=acl Refactoring will be necessary where CF current tries to set fined grained ACLs. However, this query shows only setting ACLs to a publich-ish value, which seem pretty compatible with UBLA.
The main issue here is the corpus backup part which needs this level of fine grained control. By default on some of our deployments (e.g. OSS-Fuzz) the corpus is marked as private by default. After 90 days, we mark a corpus as public. However, this won't be needed by most deployments (if any apart from OSS-Fuzz), so we can likely get away with not worrying about this.
Is this still needed to unblock you after https://github.com/google/clusterfuzz/pull/2687 ?
Not 100% sure and I don't have a an easy way to test, but I think so. In cases where UBLA is mandatory, the UBLA permission model must be selected when a bucket is created otherwise bucket creation will fail.
The main issue here is the corpus backup part which needs this level of fine grained control. By default on some of our deployments (e.g. OSS-Fuzz) the corpus is marked as private by default. After 90 days, we mark a corpus as public.
Would it work to move corpus content from a private bucket to a public bucket instead of toggling ACLs on the corpus directly?
I should also add that I'm no longer blocked because my GCP project was granted an exception to use fine-grained ACLs, but this refactoring is still worth adding to the backlog.
As a non-Googler, I can confirm that groups with UBLA enabled cannot perform a new deployment of Clusterfuzz due to the legacy ACL requirements.
...
Running: gsutil cp -a public-read src/appengine/resources/clusterfuzz-source.manifest gs://deployment.<redacted>.appspot.com/clusterfuzz-source.manifest.3
| Copying file://src/appengine/resources/clusterfuzz-source.manifest [Content-Type=application/x-ms-manifest]...
BadRequestException: 400 Cannot insert legacy ACL for an object when uniform bucket-level access is enabled. Read more at https://cloud.google.com/storage/docs/uniform-bucket-level-access
| Return code is non-zero (1).
| Exit.
Traceback (most recent call last):
File "butler.py", line 319, in <module>
main()
File "butler.py", line 305, in main
command.execute(args)
File "src/local/butler/create_config.py", line 307, in execute
deploy_zips(args.new_config_dir)
File "src/local/butler/create_config.py", line 207, in deploy_zips
'--config-dir', config_dir
File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', 'butler.py', 'deploy', '--force', '--targets', 'zips', '--prod', '--config-dir', '<redacted>']' returned non-zero exit status 1.
