FasterTransformer icon indicating copy to clipboard operation
FasterTransformer copied to clipboard

Can't run translate_example.py found in T5 model

Open EmanElrefai12 opened this issue 2 years ago • 0 comments

Hello,

I am trying to run this file like mentioned here. https://github.com/NVIDIA/FasterTransformer/blob/main/docs/t5_guide.md. I got these errors. One of these is from building the project and I created an issue about it. Any help?

[The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'T5Tokenizer'. 
The class this function is called from is 'PreTrainedTokenizerFast'.
2023-08-15 13:20:29,342 __main__ [INFO] Run gemm test: ./bin/t5_gemm 32 4 128 512 8 64 2048 512 8 64 2048 32128 0 1 0 > .tmp_gemm.log
2023-08-15 13:20:44,859 __main__ [INFO] Run gemm test: ./bin/t5_gemm 32 1 128 512 8 64 2048 512 8 64 2048 32128 0 1 1 > .tmp_gemm.log
[INFO] MPI is not available in this PyTorch build.
[INFO] MPI is not available in this PyTorch build.
[INFO] MPI is not available in this PyTorch build.
Traceback (most recent call last):
  File "/content/FasterTransformer/examples/pytorch/t5/translate_example.py", line 507, in <module>
    translate(vars(args))
  File "/content/FasterTransformer/examples/pytorch/t5/translate_example.py", line 310, in translate
    ft_encoder = FTT5Encoder(ft_encoder_weight.w, lib_path, encoder_config.num_heads,
  File "/content/FasterTransformer/examples/pytorch/t5/../../../examples/pytorch/t5/utils/ft_encoder.py", line 380, in __init__
    torch.classes.load_library(lib_path)
  File "/usr/local/lib/python3.10/dist-packages/torch/_classes.py", line 51, in load_library
    torch.ops.load_library(path)
  File "/usr/local/lib/python3.10/dist-packages/torch/_ops.py", line 643, in load_library
    ctypes.CDLL(path)
  File "/usr/lib/python3.10/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /content/lib/libth_transformer.so: cannot open shared object file: No such file or directory]

EmanElrefai12 avatar Aug 15 '23 14:08 EmanElrefai12