Hi, could you please help with resolve below issue
i have already install the serathi-serve, but still meet this error
python vidur/profiling/attention/main.py \
--models codellama/CodeLlama-34b-Instruct-hf \
--num_gpus 4
Traceback (most recent call last):
File "/app/software1/vidur/vidur/profiling/attention/main.py", line 10, in
from sarathi.model_executor.attention import AttentionBackend
File "/app/software1/vidur/sarathi/model_executor/init.py", line 1, in
from sarathi.model_executor.model_loader import get_model
File "/app/software1/vidur/sarathi/model_executor/model_loader.py", line 11, in
from sarathi.model_executor.models import * # pylint: disable=wildcard-import
File "/app/software1/vidur/sarathi/model_executor/models/init.py", line 1, in
from sarathi.model_executor.models.falcon import FalconForCausalLM
File "/app/software1/vidur/sarathi/model_executor/models/falcon.py", line 32, in
from sarathi.model_executor.layers.rotary_embedding import get_rope
File "/app/software1/vidur/sarathi/model_executor/layers/rotary_embedding.py", line 30, in
from sarathi import pos_encoding_ops
ImportError: cannot import name 'pos_encoding_ops' from 'sarathi' (/app/software1/vidur/sarathi/init.py)
microsoft
