Introduce PTEFile class

[lucylq]

Stack from ghstack (oldest at bottom):

• #15801
• -> #15800

PTEFile class holds the components of a PTE file: the program, mutable constants and named data.

Currently, the program definition does not contain mutable constants and named data; they are always stored in segments and not inline. This means when we deserialize, they are lost, because we only deserialize into the program concept.

Now, segment data is included in the PTEFile class.

Differential Revision: D86814175

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15800

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:x: 6 New Failures, 2 Unrelated Failures

As of commit 6c8e9e4eb6235fb7fe33ea4af1b59ea34a5c3f01 with merge base b1e3e28bb611e06d484138be27221faffd89f565 ():

NEW FAILURES - The following jobs have failed:

• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, non-quantized) / linux-job (gh) RuntimeError: Command docker exec -t 82630ac29d815c4828976b31cc2c9cf790a0323f1b151774b86c0d2c57339dcb /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, quantized-int4-tile-packed) / linux-job (gh) RuntimeError: Command docker exec -t 81d93dc54bc9a2beeeb35b4221befa0184b1d3cbaf26853660e7ebe61a54ea68 /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, quantized-int4-weight-only) / linux-job (gh) RuntimeError: Command docker exec -t d85cfe17cc5bee703e1e5cbbfe8f795a6561e111c6ba1f05bae3d1a89cc53f42 /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, non-quantized) / linux-job (gh) RuntimeError: Command docker exec -t 72910975926e90f72cb469277a7e3801631028b411d6b7866705ceb355f9bf04 /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, quantized-int4-tile-packed) / linux-job (gh) RuntimeError: Command docker exec -t dea75d85fc51e58e2a7749c0d351ad2040489dd4d6f01fe3b60d9d6e7bb25d6a /exec failed with exit code 1
• Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, quantized-int4-weight-only) / linux-job (gh) RuntimeError: Command docker exec -t 1e49b4e29bb6e68795feb5f467587bc6beb4b5758f015e7a597bc5c7f89a09c8 /exec failed with exit code 1

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• Test Metal Backend / test-model-metal-e2e (openai, whisper-small, non-quantized) / macos-job (gh) (similar failure) RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Test Metal Backend / test-model-metal-e2e (openai, whisper-large-v3-turbo, non-quantized) / macos-job (gh) (trunk failure) RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions[bot]]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example @pytorchbot label "release notes: none"

For more information, see https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[lucylq]

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

[zingo]

It might be nice it bundleIO BPTE files could be mixed into this in some way and handled similarly.

@zingo thanks for the suggestion - could you comment a bit more on what you're looking for?

Im thinking the it might be god to have if you made tools parsing the different files. As Im not writing those kind of tools right now Im not sure its fully applicable but I feel the bpte are sometimes forgotten or need special handling.