[rank2]: AttributeError: 'Collator' object has no attribute 'tokenizer'

[lis-kp]

hi, nice project, thanks for sharing it! i have been trying to run the classifier fine-tuning code, but i keep getting this error:

.... [rank2]: Original Traceback (most recent call last): [rank2]: File "/home/liskanashiro/miniforge3/lib/python3.12/site-packages/torch/utils/data/_utils/worker.py", line 349, in _worker_loop [rank2]: data = fetcher.fetch(index) # type: ignore[possibly-undefined] [rank2]: ^^^^^^^^^^^^^^^^^^^^ [rank2]: File "/home/liskanashiro/miniforge3/lib/python3.12/site-packages/torch/utils/data/_utils/fetch.py", line 55, in fetch [rank2]: return self.collate_fn(data) [rank2]: ^^^^^^^^^^^^^^^^^^^^^ [rank2]: File "/home/liskanashiro/llava_factory/Mantis/mantis/train/data.py", line 1531, in call [rank2]: batch_encoding = self._right_pad_inputs_with_attention_mask(model_inputs=batch) [rank2]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ [rank2]: File "/home/liskanashiro/llava_factory/Mantis/mantis/train/data.py", line 1399, in _right_pad_inputs_with_attention_mask [rank2]: pad_token_id = self.tokenizer.pad_token_id [rank2]: ^^^^^^^^^^^^^^ [rank2]: AttributeError: 'Collator' object has no attribute 'tokenizer' ...

has anyone else encounter the same? thanks!

[ElegantLin]

https://github.com/ElegantLin/Mantis/pull/1