EnergonAI

EnergonAI copied to clipboard

_pickle.UnpicklingError: invalid load key, '{'.

4


RundongCao

OPT inference

2

hello, I want to just inference of pre-trained model in the terminal, but I don't want to run a HTTP server. How could I do that?

Joanna-0421

Where is InferenceEngine definition?

2

I can't find it in energonai/engine.py "from energonai.engine import InferenceEngine"

liujuncn

an error caused by running the example of the opt

4

(pytorch) root@USER-20211001RA:~/EnergonAI-main/examples/opt# python opt_fastapi.py opt-125m --checkpoint ./restored.pt /usr/local/anaconda3/envs/pytorch/lib/python3.9/site-packages/torch/library.py:130: UserWarning: Overriding a previously registered kernel for the same operator and the same dispatch key operator: aten::index.Tensor(Tensor self, Tensor?[] indices) -> Tensor...

LemonSqi

polish code and add setup script

2

- polish code - add setup script - upgrade version to 0.0.2

juncongmoo

How to use dynamic batch features

1

Hello, I have launched the opt-125M inference, and send request to server with locust. but what ever config the max_batch_size, the InferenceEngine always run in batch_size =1. how can i...

hudengjunai

OPT demo TEST

2

========================================================================================= **No pre-built kernel is found, build and load the layernorm kernel during runtime now** ========================================================================================= **No modifications detected for re-loaded extension module layernorm, skipping build step...** [W tensorpipe_agent.cpp:682] RPC...

Batizhao8899

fail to install EnergonAI

3

I use anaconda, python 3.10 and pytorch 1.13.1 . When I ran the following Installation command: _pip install ._ an error happened. Part of the error message is: ``` Processing...

NewDriverLee

Is there any examples of using offload feature in GPT/BLOOM/OPT inference?

1

Hi, currently in the examples, only `linear` describes a naive example of offload, in other projects such as `opt`, `bloom`, `gpt`, there is no option for offload. I am wondering...

YJHMITWEB

miss cache error when pose generation opt

2

how to use this demo, could u provide any detail example

tycallen