NXP Backend: Add eIQ Neutron Backend

[robert-kalmar]

Summary

Initial implementation for the NXP eIQ Neutron Backend for Neutron-N3-64 (i.MX RT700)

Test plan

Functionality tested by python unit tests:

pytest -c /dev/null/ backend/nxp/tests

cc @digantdesai @JakeStevens , @JakeStevens , @skywall

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/10196

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:x: 10 New Failures

As of commit 8935fa549c593f7f06d12819b8a83d4da45d4fac with merge base 4559a618c66853d1a8b20e0b1b776013c4a54146 ():

NEW FAILURES - The following jobs have failed:

• Lint / link-check / lint-urls / linux-job (gh)
• Lint / link-check / lint-xrefs / linux-job (gh) RuntimeError: Command docker exec -t fd1de1edaba220a32fc6cfc657b8e6bd9e4ab5a4659cc2b3a80ccc00dfd6eada /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh) RuntimeError: Command docker exec -t cb7ab171ac001b837c484fd99a8d91a9c50d085cf1f4b58f687b257db1e55ec0 /exec failed with exit code 139
• pull / test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job (gh) RuntimeError: Command docker exec -t e6738379819dbdc59c2bdd872e5da1b36657298c30b22886b603f948be2dde8b /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh) RuntimeError: Command docker exec -t f30762d1fd463103a18abc479e981d774fd94668d5e121de5927e162a386c2b3 /exec failed with exit code 1
• pull / unittest / macos / macos-job (gh) RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh) RuntimeError: Command docker exec -t 6b401c100f9981f8e256027ec527e1aed7d3076a74200294bbe5a9361e9fa336 /exec failed with exit code 127
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh) RuntimeError: Command docker exec -t 0d2720d469d5ca1114f98afbd574f52eb1568639a68b7f28b42536b0ee5d494e /exec failed with exit code 127
• pull / unittest-editable / linux / linux-job (gh) RuntimeError: Command docker exec -t 84f77ca6d60309df7e9657bc21f780f4068415a515c8b611f67dde35a07f1080 /exec failed with exit code 1
• pull / unittest-editable / macos / macos-job (gh) RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[robert-kalmar]

@pytorchbot label "module: nxp" , label "release notes: nxp"

[pytorch-bot[bot]]

Didn't find following labels among repository labels: ,,label

[digantdesai]

wow this is a huge PR! I will try to go through this bit by bit and leave comments, expect a few days before we can finish reviewing this. Thanks though :)

[robert-kalmar]

wow this is a huge PR! I will try to go through this bit by bit and leave comments, expect a few days before we can finish reviewing this. Thanks though :)

Unfortunately, it was hard to break it down as most of it is the infrastructure for conversion from Edge Dialect to format suitable for Neutron Converter (LiteRT/tflite flatbuffer). But let me give you some insight into the code, what hopefully help you with review:

• the backends/nxp/backend/neutron_partitioner.py and backends/nxp/backend/nxp_backend.py are the main files for the eIQ Neutron Backend
• the backends/nxp/backend/neutron_node_extraction.py contains logic to extract the microcode for the Neutron NPU from the Neutron Converter invocation.
• the backends/nxp/backend/ir/lib/tflite is automatically generated from LiteRT flatbuffers schema
• the backends/nxp/backend/ir/converter/builder is the LiteRT flatbuffer generation logic
• the backend/nxp/backedns/ir/converter/node_conversion/ contains the conversion logic from Edge Dialect to LiteRT

[JakeStevens]

@robert-kalmar please make sure to run the linter and re-submit

[JakeStevens]

@robert-kalmar please make sure to run the linter and re-submit

Still failing after latest push.

[robert-kalmar]

Linting errors should be fixed now.

[JakeStevens]

Think you need just one more run of the linter, misplaced import

[robert-kalmar]

Think you need just one more run of the linter, misplaced import

Hopefully fixed in last push.

[JakeStevens]

Failed tests seem unrelated but will let @digantdesai take a look

[digantdesai]

Yeah looks unrelated. Let's merge, worst case if this causes some instability we can revert it.

[facebook-github-bot]

@digantdesai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.