relax check on bf16

[lucylq]

Now require kernel support for bf16.

Export: python3 torchchat.py export llama3 --dtype [fp16 | bf16] --output-pte-path llama3.pte

(.venv) (base) [[email protected] /data/users/lfq/torchchat (lfq.export-bf16)]$ python3 torchchat.py generate llama3 --device cpu --pte-path llama3.pte --prompt "Hello my name is"
Warning: checkpoint path ignored because an exported DSO or PTE path specified
Warning: checkpoint path ignored because an exported DSO or PTE path specified
Using device=cpu Intel Core Processor (Broadwell)
Loading model...
Time to load model: 0.11 seconds
I 00:00:00.000905 executorch:program.cpp:129] InternalConsistency verification requested but not available
E 00:00:51.744419 executorch:method.cpp:936] Overriding output data pointer allocated by memory plan is not allowed.
I 00:00:51.744460 executorch:pybindings.cpp:196] Cannot set_output_data_ptr(): this likely means the outputs were MemoryPlanned inspect the error code to know for sure, but likely this is not an issue. 0x2
F 00:00:51.747880 executorch:op_index.cpp:87] In function operator()(), assert failed (false): Unhandled dtype BFloat16 for index.Tensor_out
Aborted (core dumped)

fp16 runs well!

(.venv) (base) [[email protected] /data/users/lfq/torchchat (lfq.export-bf16)]$ python3 torchchat.py generate llama3 --device cpu --pte-path llama3_f16.pte --prompt "Hello my name is" 2>/dev/null
Warning: checkpoint path ignored because an exported DSO or PTE path specified
Warning: checkpoint path ignored because an exported DSO or PTE path specified
Using device=cpu Intel Core Processor (Broadwell)
Loading model...
Time to load model: 0.10 seconds
Hello my name is Elsie, I'm a 28 year old woman who loves to be outdoors and try new things. I'm a bit of a thrill-seeker and love to push myself out of my comfort zone. I'm also a big fan of nature and love spending time in the wilderness.
I'm a bit of a foodSerializedName and love trying new recipes and experimenting with different flavors and ingredients. I'm also a big fan of coffee and love a good cup of joe in the morning.
I'm a bit of a homebody and love spending time at home with my family and friends. I'm also a big fan of movies and TV shows and love getting lost in a good story.
 Injector

I'm a bit of a hopeless romantic and love the idea of finding true love and building a life with someone. I'm looking for someone who is kind, honest, and genuine, and who also loves the outdoors and trying new things. If you're a fellow thrill-seeker who loves nature and<|begin_of_text|>Hello my name is Elsie, I'm a 28 year old woman who loves to be outdoors and try new things. I'm a bit of a thrill-seeker and love to push myself out of my comfort zone. I'm also a big fan of nature and love spending time in the wilderness.
I'm a bit of a foodSerializedName and love trying new recipes and experimenting with different flavors and ingredients. I'm also a big fan of coffee and love a good cup of joe in the morning.
I'm a bit of a homebody and love spending time at home with my family and friends. I'm also a big fan of movies and TV shows and love getting lost in a good story.
 Injector

I'm a bit of a hopeless romantic and love the idea of finding true love and building a life with someone. I'm looking for someone who is kind, honest, and genuine, and who also loves the outdoors and trying new things. If you're a fellow thrill-seeker who loves nature and<|eot_id|>
Max Sequence Length Reached. Ending Conversation.
==========

[pytorch-bot[bot]]

:link: Helpful Links

:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/3460

• :page_facing_up: Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

:white_check_mark: No Failures

As of commit 245ffd8613f1206c515724167f58a57259e710bf with merge base 1fd80feff8f22479f650b53b10ad4aada49e9d71 (): :green_heart: Looks good so far! There are no failures yet. :green_heart:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[facebook-github-bot]

@lucylq has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[lucylq]

LGTM. @lucylq could you create an issue for bf16 kernel support? Thanks!

https://github.com/pytorch/executorch/issues/3488

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D56849464

[facebook-github-bot]

@lucylq merged this pull request in pytorch/executorch@0909c5ac58b67d3f902fc888b3449f520a50e23c.