Incompability between Pandas <2 and >=2 saved models

[martinkim0]

Error occurs when a model is saved with Pandas <2 and is attempted to be loaded in with Pandas >=2

scvi/model/base/_base_model.py:691: in load
    ) = _load_saved_files(
scvi/model/base/_utils.py:68: in _load_saved_files
    model = torch.load(model_path, map_location=map_location)
../../.../python3.11/site-packages/torch/serialization.py:1014: in load
    return _load(opened_zipfile,
../../.../python3.11/site-packages/torch/serialization.py:1422: in _load
    result = unpickler.load()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <torch.serialization._load.<locals>.UnpicklerWrapper object at 0x7f25fc273b10>
mod_name = 'pandas.core.indexes.numeric', name = 'Int64Index'

    def find_class(self, mod_name, name):
        if type(name) is str and 'Storage' in name:
            try:
                return StorageType(name)
            except KeyError:
                pass
        mod_name = load_module_mapping.get(mod_name, mod_name)
>       return super().find_class(mod_name, name)
E       ModuleNotFoundError: No module named 'pandas.core.indexes.numeric'

../../.../python3.11/site-packages/torch/serialization.py:1415: ModuleNotFoundError

[bio-la]

hey! still have this issue, is there a workaround to update older models? thanks!

[martinkim0]

Hey! Unfortunately, right now, you'd have to load in older models with pandas<2.0 for it to work. I haven't gotten around to patching this yet, sorry!

[martinkim0]

Following up on this, this issue is on the PyTorch side since they're not using the newer Pandas unpickler for loading files.