Add device type (public, private) and id

[huydhn]

And also enable Apple benchmark there. This depends on https://github.com/pytorch/test-infra/pull/6579

(Revert https://github.com/pytorch/executorch/commit/b415af07be46569903d4d03bd66195b9afa85a96 for testing as it currently breaks building Apple benchmark app, I'll put it back before landing cc @shoumikhin )

Testing

https://hud.pytorch.org/benchmark/llms?startTime=Sat%2C%2019%20Apr%202025%2007%3A11%3A18%20GMT&stopTime=Sat%2C%2026%20Apr%202025%2007%3A11%3A18%20GMT&granularity=day&lBranch=enable-apple-benchmark-private-device&lCommit=46b6eab37da51c83304353df4d403ccbb9991ec4&rBranch=enable-apple-benchmark-private-device&rCommit=46b6eab37da51c83304353df4d403ccbb9991ec4&repoName=pytorch%2Fexecutorch&benchmarkName=&modelName=All%20Models&backendName=All%20Backends&modeName=All%20Modes&dtypeName=All%20DType&deviceName=All%20Devices&archName=All%20Platforms

where the private device is marked private.

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10496

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[zingo]

Regarding (Revert https://github.com/pytorch/executorch/commit/b415af07be46569903d4d03bd66195b9afa85a96 for testing as it currently breaks building Apple benchmark app, I'll put it back before landin...

We discovered in some internal flow that we where missing --recursive on git submodule update in some of our scripts and got into problems with that PR, maybe it might help you to to add it if not there.

e.g. change git submodule update --init to git submodule update --init --recursive

I do not know if helps you in your case but could be a thing to check.

[huydhn]

We discovered in some inetrnal flow that we where missing --recursive on git submodule update in some of our script maybe it might help you to to add it if not there.

e.g. change git submodule update --init to git submodule update --init --recursive

I do not know if helps you in your case but could be something to check.

I see. It looks like there is already a PR for this https://github.com/pytorch/executorch/pull/10472, but let me check if it misses some spots

[guangy10]

We discovered in some inetrnal flow that we where missing --recursive on git submodule update in some of our script maybe it might help you to to add it if not there. e.g. change git submodule update --init to git submodule update --init --recursive I do not know if helps you in your case but could be something to check.

I see. It looks like there is already a PR for this #10472, but let me check if it misses some spots

Yes, --recursive is required otherwise execturoch runtime will complain about missing CMakeLists.txt in the third-party library.

[huydhn]

I think the failed build-benchmark-app job already checkout the submodule recursively https://github.com/pytorch/executorch/blob/main/.github/workflows/apple.yml#L282, but it's still failing to build after the tokenizer bump https://github.com/pytorch/executorch/actions/runs/14672135868/job/41180874589

2025-04-25T20:07:18.5106110Z Undefined symbols for architecture arm64:
2025-04-25T20:07:18.5109390Z   "tokenizers::create_regex(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)", referenced from:
2025-04-25T20:07:18.5110360Z       tokenizers::(anonymous namespace)::_create_regex(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) in tiktoken.o

[guangy10]

I think the failed build-benchmark-app job already checkout the submodule recursively https://github.com/pytorch/executorch/blob/main/.github/workflows/apple.yml#L282, but it's still failing to build after the tokenizer bump https://github.com/pytorch/executorch/actions/runs/14672135868/job/41180874589

2025-04-25T20:07:18.5106110Z Undefined symbols for architecture arm64:
2025-04-25T20:07:18.5109390Z   "tokenizers::create_regex(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&)", referenced from:
2025-04-25T20:07:18.5110360Z       tokenizers::(anonymous namespace)::_create_regex(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&) in tiktoken.o

If it fails on main, we should tag the folks in the team to take a look