Jonathan Marchand comments

Results 4 comments of


                                            Jonathan Marchand

Blank output in the Inference while using a customize trained T5 model

Hi @gaurav21s By blank, you mean all the outputs are empty-zeroed tensors ? Do you have any warning / error messages ?

Blank output in the Inference while using a customize trained T5 model

Thanks for your reports, I can confirm this bug and we're investigating it. Simple code to reproduce it: ``` py import torch from transformers import AutoModelForSeq2SeqLM, AutoTokenizer from kernl.model_optimization import...

Blank output in the Inference while using a customize trained T5 model

T5 weights are in BF16, triton 2.0 does not support fully BF16, we're waiting for the fix to be propagated. https://github.com/openai/triton/pull/1306

bug: Trying to run T5 tutorial and getting `free(): invalid pointer` error.

Thanks for reporting this @gilljon. Could you give the content of `test_kernl.py` ? We have some issues with t5 and bf16 but I don't think it's related.