dashboard,syz-ci: implement asset storage

[a-nogikh]

Asset is a file/attachment relevant for debugging a kernel crash or the syzbot itself.

TODO list:

• Fix lint errors and unit tests
• Deploy and debug a test instance.
• Generate the new index.yaml file.
• Write some documentation.

Dashboard keeps track of the uploaded assets and manages their lifetime. Some of the assets are attached to the bug reports sent by syzkaller, some (like html coverage report) are displayed on the web portal.

Two new dashboard API requests:

• add_build_asset -- let dashboard remember one more build-related asset.
• needed_assets -- query the list of assets that are still needed by the dashboard.

The syz-ci side of the asset storage functionality is responsible for the following:

• Intercept assets at various stages of syz-ci operation.
• Compress and upload assets to GCS.
• Report assets to the dashboard.
• Remove no longer needed assets.

[dvyukov]

Re assets.AssetStorage check out: https://go.dev/blog/package-names#naming-package-contents https://rakyll.org/style-packages/#package-naming

[codecov[bot]]

Codecov Report

Merging #3223 (aaa4cd3) into master (da70065) will decrease coverage by 0.1%. The diff coverage is 48.7%.

:exclamation: Current head aaa4cd3 differs from pull request most recent head afad233. Consider uploading reports for the commit afad233 to get more accurate results

Impacted Files	Coverage Δ
dashboard/app/entities.go	79.1% <ø> (ø)
pkg/asset/backend_gcs.go	0.0% <0.0%> (ø)
syz-ci/manager.go	0.9% <0.0%> (-0.2%)	:arrow_down:
syz-ci/syz-ci.go	19.3% <0.0%> (-4.0%)	:arrow_down:
pkg/asset/config.go	20.7% <20.7%> (ø)
pkg/asset/type.go	27.3% <27.3%> (ø)
dashboard/app/api.go	58.4% <46.9%> (-0.5%)	:arrow_down:
pkg/asset/storage.go	65.6% <65.6%> (ø)
dashboard/app/asset_storage.go	70.7% <70.7%> (ø)
dashboard/app/jobs.go	63.6% <75.0%> (+0.2%)	:arrow_up:
... and 18 more

[a-nogikh]

Merged the helper commits as a separate PR and rebased this one.

[a-nogikh]

@dvyukov PTAL

I addressed your comments (for the non-implemented ones see my replies).

There's a all: code review fixes commit that is still there only for convenience purposes. I'll split and squash it when we get to merging this PR.

I deployed this branch on a dev syz-ci and it seems to be working fine.