Use the full dataset for accuracy validation in the integration test

[anhappdev]

In our integration test we have a check to validate the accuracy results of the benchmarks. Currently, the app uses a tiny subset of the full dataset, so its result may not be reliable.

We should use the full dataset instead. Since this test is run exclusively on our CI, the full dataset can be stored privately on Google Cloud.

[freedomtan]

@Mostelk to check if we can get space from what Bruno is planning to buy. 20 GiB should be a safe bet for this purpose.
access frequency: maybe 50 times per month.

[freedomtan]

@Mostelk to check with with Bruno for the progress.

for public data, it's pretty easy.

• Zenodo: https://zenodo.org/
• HuggingFace: https://huggingface.co/, provides git-lfs, for public one

The main issue: ImageNet

What we need: 8.x GiB for all the full validation.

[anhappdev]

Another solution would be using GitHub Release if each file is under 2 GB. The SNUSR dataset was released this way: https://github.com/mlcommons/mobile_models/releases

[freedomtan]

@freedomtan check if we can put datasets other than ImageNet to GitHub Release.

[freedomtan]

• MS COCO: It seems only images with CC licenses were collected and annotations are in CC
• ADE20K:
  • images: https://groups.csail.mit.edu/vision/datasets/ADE20K/terms/
  • annotations: BSD
• [SQuAD]: CC BY-SA 4.0
• SR: supposedly OK.

[freedomtan]

Let's use github release for datasets other than ImageNet.

[anhappdev]

Waiting until https://github.com/mlcommons/mobile_app_open/pull/707 is merged so we save time and bandwidth downloading dataset.