OPT demo TEST

[Batizhao8899]

========================================================================================= No pre-built kernel is found, build and load the layernorm kernel during runtime now

No modifications detected for re-loaded extension module layernorm, skipping build step...

[W tensorpipe_agent.cpp:682] RPC agent for master encountered error when reading incoming request from worker0: eof (this error originated at tensorpipe/transport/shm/connection_impl.cc:259) [W tensorpipe_agent.cpp:682] RPC agent for worker0 encountered error when reading incoming request from master: pipe closed (this error originated at tensorpipe/core/pipe_impl.cc:356) INFO: Finished server process [158599] Process SpawnProcess-1: ERROR: Exception in ASGI application

asyncio.exceptions.CancelledError INFO: 111.192.91.34:6974 - "POST /generation HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last):

ImportError: /home/ubuntu/.cache/colossalai/torch_extensions/torch1.11_cu11.3/layernorm.so: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

RuntimeError: eof (this error originated at tensorpipe/transport/shm/connection_impl.cc:259)

[W tensorpipe_agent.cpp:863] RPC agent for master encountered error when sending outgoing request #9 to worker0: eof (this error originated at tensorpipe/transport/shm/connection_impl.cc:259)