Liu Jun

The customized model is not in your "Supported Models" list. Can it benefit from Deepspeed chat?

Flash is not flash

1

I test Flash attention vs HF GPT2 with pytorch lightning warp. But it is slow than transformers.GPT2LMHeadModel with same config parameters. Not sure where I am going wrong?  Purple...

Where is InferenceEngine definition?

2

I can't find it in energonai/engine.py "from energonai.engine import InferenceEngine"

@MogicianXD 想用于NLP研究。不需要用户数据。

charref() maybe throw OverflowError: Python int too large to convert to C int

1

in charref(): ` if name[0] in ["x", "X"]: c = int(name[1:], 16) else: c = int(name) ` if name like "xabcdeabcdeabcdeabcde" will cause chr(c) throw OverflowError.

能否增加一个从头预训练的例子？

1

例如从头训练一个1B的llama2架构的模型.

Have you tried other values?