pkg/cover: cover.html is too big

[tarasmadan]

Is your feature request related to a problem? Please describe. Our cover.html is too big. Sometimes Chrome can't process it and fails.

Describe the solution you'd like We can compress the source code embedded into cover.html

JS zlib may be the way to go. The idea is to compress all the files we're embedding during the cover.html generation and decompress the data on-demain in the browser.

[a-nogikh]

I assume that majority of its size are the syzlang programs, maybe we can remove them or offload them somewhere else (e.g. replace by the links to some web endpoint that serves them).

[tarasmadan]

+1. I forgot about the programs. To zlib "files + programs" looks easier. We'll continue to emit the self-containing cover.html files.

[ramosian-glider]

(dumping my thought from an offline discussion here)

I am afraid that gzipping all programs together won't work, because we'll have to unpack them when the page is opened (or the first time the user wants to see a program), and the browser will still explode.

It would be nice to compress individual programs somehow, perhaps using a Shared Compression Dictionary.

[ramosian-glider]

For what it's worth, here's an example of a crashing cover.html page: https://storage.googleapis.com/syzbot-assets/abc0c3aab895/ci-upstream-kasan-gce-root-f315296c.html

[a-nogikh]

Uncompressed cover.html for ci-snapshot-upstream-root now weights 917 (!) MB.

$ cat ci-snapshot-upstream-root-4f79eaa2.html | wc -c
917700104

But if we drop all syz_mount_image calls from there, it's already 293 MB.

$ cat ci-snapshot-upstream-root-4f79eaa2.html | grep -v "syz_mount_image" | wc -c
293592599

@dvyukov @tarasmadan wdyt about either not including big reproducers in there or not encoding the images themselves? I doubt anyone would ever click on the line in the coverage file to find out what exact fs image was mounted to reach it.

[tarasmadan]

+1 to limit the fs size to a few lines.

[dvyukov]

I doubt anyone would ever click on the line in the coverage file to find out what exact fs image was mounted to reach it.

All programs can be used to recreate the coverage with syz-execprog. But do we have any other options on the table to make the reports open-able? I guess we could add a mode to export reports w/o reproducers. But these are used by somebody, right?

[tarasmadan]

At least @sirdarckcat parsed them out from the reports.

[sirdarckcat]

You can have some JavaScript code at the beginning stop the loading of the html page and then use HTTP Range requests to load specific sections. However that requires you to save an index with the offsets at the end of the file (and have the JS do a range request to get them).

Or do what we do for the kernel dashboard or kernel.dance which is just generate all output as a SQLite database and have JS code fetch it.