MoE-Infinity run on the mutiple gpus

Is there an unquantized version that can run on multiple GPUs?

Apr 19 '24 12:04 YLSnowy

Feel free to replace the example model with "mistralai/Mixtral-8x7B-Instruct-v0.1". If there's further error in running multi-GPU version, please feel free to post here

Apr 19 '24 13:04 drunkcoding

when I run interface_example.py, I meet the errors below: Traceback (most recent call last): File "/home/admin/Documents/MoE-Infinity-main/examples/interface_example.py", line 32, in names = datasets.get_dataset_config_names(dataset_name) File "/home/admin/anaconda3/envs/moe-infinity/lib/python3.9/site-packages/datasets/inspect.py", line 347, in get_dataset_config_names dataset_module = dataset_module_factory( File "/home/admin/anaconda3/envs/moe-infinity/lib/python3.9/site-packages/datasets/load.py", line 1915, in dataset_module_factory raise e1 from None File "/home/admin/anaconda3/envs/moe-infinity/lib/python3.9/site-packages/datasets/load.py", line 1867, in dataset_module_factory can_load_config_from_parquet_export = "DEFAULT_CONFIG_NAME" not in f.read() File "/home/admin/anaconda3/envs/moe-infinity/lib/python3.9/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

Aug 07 '24 07:08 Tingberer

when I run interface_example.py, I meet the errors below: Traceback (most recent call last): File "/home/admin/Documents/MoE-Infinity-main/examples/interface_example.py", line 32, in names = datasets.get_dataset_config_names(dataset_name) File "/home/admin/anaconda3/envs/moe-infinity/lib/python3.9/site-packages/datasets/inspect.py", line 347, in get_dataset_config_names dataset_module = dataset_module_factory( File "/home/admin/anaconda3/envs/moe-infinity/lib/python3.9/site-packages/datasets/load.py", line 1915, in dataset_module_factory raise e1 from None File "/home/admin/anaconda3/envs/moe-infinity/lib/python3.9/site-packages/datasets/load.py", line 1867, in dataset_module_factory can_load_config_from_parquet_export = "DEFAULT_CONFIG_NAME" not in f.read() File "/home/admin/anaconda3/envs/moe-infinity/lib/python3.9/codecs.py", line 322, in decode (result, consumed) = self._buffer_decode(data, self.errors, final) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

the dataset loading error is not related to any part of MoE-Infinity, would suggest delete local cache and retry

Aug 29 '24 13:08 drunkcoding