llm-foundry
                                
                                 llm-foundry copied to clipboard
                                
                                    llm-foundry copied to clipboard
                            
                            
                            
                        make triton attn req pre-mlri tagged triton
I fork triton and rename it at triton_pre_mlri, triton diff here
llmfoundry/models/layers/flash_attn_triton.py is copy pasta from HazyResearch flash_attn_triton where I modify imports to be
import triton_pre_mlir as triton
import triton_pre_mlir.language as tl
This PR will be used as a springboard to test torch2 updt
~~blocking composer pr (waiting for updt img): https://github.com/mosaicml/composer/pull/2229~~ merged
This is integrated into https://github.com/mosaicml/llm-foundry/pull/149 closing in favor of that PR.