DouJS

icon indicating a website link dujiangsu.github.io

icon company Sun Yat-sen University && NSCC-GZ icon location Guangzhou, Guangdong, China icon location I am Jiangsu Du, postdoctoral researcher at Sun Yat-sen University.

Results 5 comments of DouJS

Connection refused on docker exposed port

Thanks for your correction

OPT inference generate example

Hi， you can follow https://github.com/dujiangsu/EnergonAI/blob/feature/opt-175b/energonai/utils/checkpointing_opt.py to load parameters of opt model. As for the generate example, you can refer to the hf_gpt2 example.

Missing energonai_linear_func in setup.py

I push this commit by mistake.😂 EnergonAI will have a big update recently.

num_beams for beam search

Since we are working on imporving a large-scale model demo recently, the requirement might be delayed for one or two weeks. You can use [beam search](https://github.com/hpcaitech/EnergonAI/blob/a21430b8b09685810fbf496b39ea762e6c3c702b/energonai/model/model_factory.py#L19) in transformer and add...

inference of pre-trained model

Yes, the model trained by ColossalAI can be easily transferred to EnergonAI, achieving the deployment. We are preparing the demo for serving a GPT-3 level model these days.