Run test_models.sh with strict=False flag

[mergennachin]

This is the recommended anyway, so let's gradually start migrating to strict=False

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12368

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:x: 10 New Failures, 2 Unrelated Failures

As of commit c16b6458d4f75616e6c37489240588bef90e296a with merge base a1e3d48feefac850a79f4afc34eb3879cde513a3 ():

NEW FAILURES - The following jobs have failed:

• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh) RuntimeError: Command docker exec -t f0690dc18a065741b96ed07983b0226cf10fafe8fc21576096b6c846c2e5d8a6 /exec failed with exit code 1
• pull / test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job (gh) RuntimeError: Command docker exec -t 9d0537caf3cc4f5d4014fc030caf0decca684c32504e113084a5b4c18cf3a765 /exec failed with exit code 134
• pull / test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job (gh) RuntimeError: Command docker exec -t 500f0d5251de44fcdddb011009a0a43facdfeaff9234cd0996a40da4df017925 /exec failed with exit code 134
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh) RuntimeError: Command docker exec -t f649801d72f78b08d1c47773ac30b8a7c45a0c61611a07485273e0510dad3664 /exec failed with exit code 1
• trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh) RuntimeError: Command docker exec -t 0a57d2970f46d5d4af1d3675c7bf70186973fa268eb11d9e99e4db280e5f07c9 /exec failed with exit code 1
• trunk / test-models-linux-aarch64 (ic3, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh) RuntimeError: Command docker exec -t 2c66cea1d40d4e50c2972ed7b8d36d314dddf0c6e35a3017d32f444af39f781d /exec failed with exit code 134
• trunk / test-models-linux-aarch64 (ic4, xnnpack-quantization-delegation, linux.arm64.2xlarge) / linux-job (gh) RuntimeError: Command docker exec -t 31309720598a2bf8316324387a744ebb5c9605ef709676bc2c5b3f4ca53d536c /exec failed with exit code 134
• trunk / test-models-linux-aarch64 (llama3_2_vision_encoder, portable, linux.arm64.2xlarge) / linux-job (gh) RuntimeError: Command docker exec -t 65adc97abb20c496bd5e11cc9c10732986855c459624542b4b00c9c0187d32d9 /exec failed with exit code 1
• trunk / test-models-macos (ic4, xnnpack-quantization-delegation) / macos-job (gh) RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 134
• trunk / test-models-macos (llama3_2_vision_encoder, portable) / macos-job (gh) RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• trunk / unittest-release / linux / linux-job (gh) (trunk failure) [ FAILED ] TensorPtrMakerTest.FailedCreateTensorUsingFromBlobWithIllegalStrides
• trunk / unittest-release / macos / macos-job (gh) (trunk failure) [ FAILED ] TensorPtrMakerTest.FailedCreateTensorUsingFromBlobWithIllegalStrides

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions[bot]]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example @pytorchbot label "release notes: none"

For more information, see https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[mcr229]

@digantdesai yea we should probably check that model size isn't bloated. We don't have any CI here that checks for this so right now we would have to do so manually.

[digantdesai]

@digantdesai yea we should probably check that model size isn't bloated. We don't have any CI here that checks for this so right now we would have to do so manually.

Can we add a CI to make sure it doesn't double or something, just a guard against regressing.

[github-actions[bot]]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

[github-actions[bot]]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.