Add stories ci for qnn

[cccclai]

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/4662

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:x: 1 New Failure

As of commit 5174d8fe90f8949240d17eee3a4573ebb402a140 with merge base 99e1ae1f5a5e8f8ef81c2a36a492c053128b75fb ():

NEW FAILURE - The following job has failed:

• docker-builds / docker-build (executorch-ubuntu-22.04-qnn-sdk) (gh) name unknown: The repository with name 'executorch/executorch-ubuntu-22.04-qnn-sdk' does not exist in the registry

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[chiwwang]

This looks ok for me. @haowhsu-quic @shewu-quic @chunit-quic @winskuo-quic , could you guys find a chance to give the script a try?

I think this depends on Hutton's PRs. We need those PRs to lower the examples/models/llama2, right?

[shewu-quic]

Yes, I think the most of PRs have been merged. Thanks for cclai's effort. But we also need the proper calibration for quantized llama. I think this PR should be good.

In the following, I will create a PR for model sharding to support lager llama lowering.

[chiwwang]

Yes, I think the most of PRs have been merged. Thanks for cclai's effort. But we also need the proper calibration for quantized llama. I think this PR should be good.

In the following, I will create a PR for model sharding to support lager llama lowering.

Then let's find a chance to sunset qualcomm static_llama.py example.

[winskuo-quic]

This looks ok for me. @haowhsu-quic @shewu-quic @chunit-quic @winskuo-quic , could you guys find a chance to give the script a try?

I think this depends on Hutton's PRs. We need those PRs to lower the examples/models/llama2, right?

Hi Kiwi, I have tested locally with the 3 scripts in this PR, and it works out pretty smooth for me.

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[cccclai]

As a note, this pr just test fp, and quantized version need to be added later

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[cccclai]

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

[guangy10]

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

[kirklandsign]

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

[guangy10]

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86. The wrapper script .ci/scripts/setup-qnn-deps.sh is reusable.

[kirklandsign]

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86.

Oh we want a separate docker like executorch-ubuntu-22.04-android which sets QNN as well, plus other android components? If not other components, we can use executorch-ubuntu-22.04-qnn-sdk. Otherwise .ci/scripts/setup-qnn-deps.sh is reusable.

[guangy10]

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86.

Oh we want a separate docker like executorch-ubuntu-22.04-android which sets QNN as well, plus other android components? If not other components, we can use executorch-ubuntu-22.04-qnn-sdk. Otherwise .ci/scripts/setup-qnn-deps.sh is reusable.

If I understand @cccclai 's comment correctly, executorch-ubuntu-22.04-qnn-sdk is for Linux x86 not for android. If we extend executorch-ubuntu-22.04-qnn-sdk by adding additional android deps to build runners and apps, it's kind of duplicate to executorch-ubuntu-22.04-android. A better option seems to be extending executorch-ubuntu-22.04-android with QNN SDK for benchmarking and leave executorch-ubuntu-22.04-qnn-sdk is for Linux x86 w/ emulator only. @cccclai @kirklandsign what do you guys think?

[kirklandsign]

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86.

Oh we want a separate docker like executorch-ubuntu-22.04-android which sets QNN as well, plus other android components? If not other components, we can use executorch-ubuntu-22.04-qnn-sdk. Otherwise .ci/scripts/setup-qnn-deps.sh is reusable.

If I understand @cccclai 's comment correctly, executorch-ubuntu-22.04-qnn-sdk is for Linux x86 not for android. If we extend executorch-ubuntu-22.04-qnn-sdk by adding additional android deps to build runners and apps, it's kind of duplicate to executorch-ubuntu-22.04-android. A better option seems to be extending executorch-ubuntu-22.04-android with QNN SDK for benchmarking and leave executorch-ubuntu-22.04-qnn-sdk is for Linux x86 w/ emulator only. @cccclai @kirklandsign what do you guys think?

Oh I guess @cccclai means the CI is for linux x86. The docker is pulling in QNN SDK so it can be used for everything which requires QNN SDK (including Android)

[guangy10]

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86.

Oh we want a separate docker like executorch-ubuntu-22.04-android which sets QNN as well, plus other android components? If not other components, we can use executorch-ubuntu-22.04-qnn-sdk. Otherwise .ci/scripts/setup-qnn-deps.sh is reusable.

If I understand @cccclai 's comment correctly, executorch-ubuntu-22.04-qnn-sdk is for Linux x86 not for android. If we extend executorch-ubuntu-22.04-qnn-sdk by adding additional android deps to build runners and apps, it's kind of duplicate to executorch-ubuntu-22.04-android. A better option seems to be extending executorch-ubuntu-22.04-android with QNN SDK for benchmarking and leave executorch-ubuntu-22.04-qnn-sdk is for Linux x86 w/ emulator only. @cccclai @kirklandsign what do you guys think?

Oh I guess @cccclai means the CI is for linux x86. The docker is pulling in QNN SDK so it can be used for everything which requires QNN SDK (including Android)

I understand that part, and @kirklandsign what's your actionable suggestion? Like expending executorch-ubuntu-22.04-qnn-sdk for android benchmarking and delete executorch-ubuntu-22.04-android, or something else?

[kirklandsign]

I think it's more intuitive to make it part of the existing executorch-ubuntu-22.04-clang12-android docker image, otherwise have to switch to load different dockers to build QNN runtime and android app

no this is x86, not android

I see. So this work enables running QNN with the emulator on x86_64, the android build is not covered in this PR correct?

That's correct. The reusable part is the docker image with QNN SDK set up.

Thanks for the clarification. BTW, IIUC that's not reusable because we will have to set up the QNN SDK for the android docker as this is for Linux x86.

Oh we want a separate docker like executorch-ubuntu-22.04-android which sets QNN as well, plus other android components? If not other components, we can use executorch-ubuntu-22.04-qnn-sdk. Otherwise .ci/scripts/setup-qnn-deps.sh is reusable.

If I understand @cccclai 's comment correctly, executorch-ubuntu-22.04-qnn-sdk is for Linux x86 not for android. If we extend executorch-ubuntu-22.04-qnn-sdk by adding additional android deps to build runners and apps, it's kind of duplicate to executorch-ubuntu-22.04-android. A better option seems to be extending executorch-ubuntu-22.04-android with QNN SDK for benchmarking and leave executorch-ubuntu-22.04-qnn-sdk is for Linux x86 w/ emulator only. @cccclai @kirklandsign what do you guys think?

Oh I guess @cccclai means the CI is for linux x86. The docker is pulling in QNN SDK so it can be used for everything which requires QNN SDK (including Android)

I understand that part, and @kirklandsign what's your actionable suggestion? Like expending executorch-ubuntu-22.04-qnn-sdk for android benchmarking and delete executorch-ubuntu-22.04-android, or something else?

I see. I would say we can keep two docker images for now. Just in case -qnn-sdk and -android diverged further next.

A quick patch for -android is in https://github.com/pytorch/executorch/pull/4662/files#diff-e6d9e0419b8aac6f9b7664fe9d2398da07ab195b26f0dce338e13968511daba5R46, add QNN_SDK=yes We get QNN SDK set up for android image for free

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.