Koan-Sin Tan

let's try to implement FID and CLIP (with images / distribution of output tensor pre-generated on a host machine, I mean not trying to do Inception V3 and CLIP image...

For [CLIP score](https://github.com/mlcommons/inference/blob/master/text_to_image/tools/clip/clip_encoder.py): it's to calculate the cosine similarity between text features and image features. Where text features are features from sending captions to the *CLIP text encoder*. That is,...

For [FID score](https://github.com/mlcommons/inference/blob/master/text_to_image/tools/fid/fid_score.py#L314), we need to compare two sets of distributions by sending groundtruth images and generated images into Inception V3. The former one could be generated offline; the second...

Let's try discuss in the mobile working group meeting. - do we want to have two or only one score? - if two scores are used, do we need to...

For CLIP score: it turns out to be quite straightfward. Convert an OpenAPI CLIP model to tflite and run it with TFLite interpreter, then we can get CLIP scores. See...

For the output to LoadGen: the `::mlperf::QuerySamplesComplete()` is called to return processed outputs https://github.com/mlcommons/mobile_app_open/blob/09e4b41f72714c84fb0cf844433da47ff726f62f/flutter/cpp/mlperf_driver.cc#L83 for non-offline case: https://github.com/mlcommons/mobile_app_open/blob/09e4b41f72714c84fb0cf844433da47ff726f62f/flutter/cpp/mlperf_driver.cc#L66-L80 What returned in the `QuerySampleResponse`, which uses uintptr data https://github.com/mlcommons/inference/blob/9e2c9f642e6e12b74e7c08d2e099c8af0e542873/loadgen/query_sample.h#L49-L76 My understand...

we may need to or extend the `ComputeAccuracy()` if we use 2 (FID and CLIP) scores. However, this has nothing to do with LoadGen interface. https://github.com/mlcommons/mobile_app_open/blob/09e4b41f72714c84fb0cf844433da47ff726f62f/flutter/cpp/dataset.h#L68

> > For CLIP score: it turns out to be quite straightfward. Convert an OpenAPI CLIP model to tflite and run it with TFLite interpreter, then we can get CLIP...

ask @nathanw-mlc for the App Store account

- privacy policy statement: how user-generated data are treated (where are they stored, how they are to be used by MLCommons), etc. https://developer.apple.com/app-store/app-privacy-details/ - app description: short and long description...