PyTorch 2.6.0: Weights only load failed
When upgrade from PyTorch 2.5.1 to 2.6.0, for code like
from ogb.graphproppred import PygGraphPropPredDataset
dataset = PygGraphPropPredDataset(name = 'ogbg-molhiv')
I got error:
Traceback (most recent call last):
File "/home/runner/work/hongbomiao.com/hongbomiao.com/machine-learning/graph-neural-network/src/main.py", line 103, in main
dataset, split_idx = fetch_dataset(config)
^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/hongbomiao.com/hongbomiao.com/machine-learning/graph-neural-network/src/model/data_loader.py", line 13, in fetch_dataset
dataset = PygGraphPropPredDataset(name=config.dataset)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/hongbomiao.com/hongbomiao.com/machine-learning/graph-neural-network/.venv/lib/python3.12/site-packages/ogb/graphproppred/dataset_pyg.py", line 68, in __init__
self.data, self.slices = torch.load(self.processed_paths[0])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/runner/work/hongbomiao.com/hongbomiao.com/machine-learning/graph-neural-network/.venv/lib/python3.12/site-packages/torch/serialization.py", line 1470, in load
raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
_pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint.
(1) In PyTorch 2.6, we changed the default value of the `weights_only` argument in `torch.load` from `False` to `True`. Re-running `torch.load` with `weights_only` set to `False` will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
(2) Alternatively, to load with `weights_only=True` please check the recommended steps in the following error message.
WeightsUnpickler error: Unsupported global: GLOBAL torch_geometric.data.data.DataEdgeAttr was not an allowed global by default. Please use `torch.serialization.add_safe_globals([DataEdgeAttr])` or the `torch.serialization.safe_globals([DataEdgeAttr])` context manager to allowlist this global if you trust this class/function.
I fixed by https://github.com/hongbo-miao/hongbomiao.com/pull/23546/files
from torch_geometric.data.data import DataEdgeAttr, DataTensorAttr
from torch_geometric.data.storage import GlobalStorage
torch.serialization.add_safe_globals([DataEdgeAttr, DataTensorAttr, GlobalStorage])
I am wondering if the library should handle this issue in a future version.
What would be the recommended approach? Thank you! ☺️
I'm having this problem too.
Having the same issue. This needs to be fixed, though a workaround for now could be the following:
import torch
from torch_geometric.data.storage import GlobalStorage
from torch_geometric.data.data import DataEdgeAttr, DataTensorAttr
torch.serialization.add_safe_globals([GlobalStorage, DataEdgeAttr, DataTensorAttr])
# the rest of the code
- It made it work for me.
Having the same issue. This needs to be fixed, though a workaround for now could be the following:
import torch from torch_geometric.data.storage import GlobalStorage from torch_geometric.data.data import DataEdgeAttr, DataTensorAttr
torch.serialization.add_safe_globals([GlobalStorage, DataEdgeAttr, DataTensorAttr])
the rest of the code
- It made it work for me.
I ran into the same issue with the version >=1.3.6 and can confirm that this resolved the issue for me.
Having the same issue, but your fix doesn't work for me. I have been adding this to my code:
from torch_geometric.data.storage import GlobalStorage
from torch_geometric.data.data import DataEdgeAttr, DataTensorAttr
from numpy.core.multiarray import _reconstruct
torch.serialization.add_safe_globals([_reconstruct, DataEdgeAttr, DataTensorAttr, GlobalStorage])
but, even if it's work on torch_geometric objects, it's stuck in failure for _reconstruct of numpy:
File "/pbs/home/v/vcuzinrambau/GCL-Link-Prediction/New/src/datasets.py", line 109, in loaddataset split_edge = dataset.get_edge_split() ^^^^^^^^^^^^^^^^^^^^^^^^ File "/pbs/home/v/vcuzinrambau/GCL-Link-Prediction/.venv/lib/python3.12/site-packages/ogb/linkproppred/dataset_pyg.py", line 77, in get_edge_split train = replace_numpy_with_torchtensor(torch.load(osp.join(path, 'train.pt'))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/pbs/home/v/vcuzinrambau/GCL-Link-Prediction/.venv/lib/python3.12/site-packages/torch/serialization.py", line 1494, in load raise pickle.UnpicklingError(_get_wo_message(str(e))) from None _pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint. (1) In PyTorch 2.6, we changed the default value of the
weights_onlyargument intorch.loadfromFalsetoTrue. Re-runningtorch.loadwithweights_onlyset toFalsewill likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source. (2) Alternatively, to load withweights_only=Trueplease check the recommended steps in the following error message. WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray._reconstruct was not an allowed global by default. Please usetorch.serialization.add_safe_globals([_reconstruct])or thetorch.serialization.safe_globals([_reconstruct])context manager to allowlist this global if you trust this class/function.
So my problem happened when I try to load collab dataset and get the edge splitting.
Did anyone, can help me, please ?
Thank you a lot !
Having the same issue, but your fix doesn't work for me. I have been adding this to my code:
from torch_geometric.data.storage import GlobalStorage from torch_geometric.data.data import DataEdgeAttr, DataTensorAttr from numpy.core.multiarray import _reconstruct
torch.serialization.add_safe_globals([_reconstruct, DataEdgeAttr, DataTensorAttr, GlobalStorage])
but, even if it's work on torch_geometric objects, it's stuck in failure for _reconstruct of numpy:
File "/pbs/home/v/vcuzinrambau/GCL-Link-Prediction/New/src/datasets.py", line 109, in loaddataset split_edge = dataset.get_edge_split() ^^^^^^^^^^^^^^^^^^^^^^^^ File "/pbs/home/v/vcuzinrambau/GCL-Link-Prediction/.venv/lib/python3.12/site-packages/ogb/linkproppred/dataset_pyg.py", line 77, in get_edge_split train = replace_numpy_with_torchtensor(torch.load(osp.join(path, 'train.pt'))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/pbs/home/v/vcuzinrambau/GCL-Link-Prediction/.venv/lib/python3.12/site-packages/torch/serialization.py", line 1494, in load raise pickle.UnpicklingError(_get_wo_message(str(e))) from None _pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint. (1) In PyTorch 2.6, we changed the default value of the
weights_onlyargument intorch.loadfromFalsetoTrue. Re-runningtorch.loadwithweights_onlyset toFalsewill likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source. (2) Alternatively, to load withweights_only=Trueplease check the recommended steps in the following error message. WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray._reconstruct was not an allowed global by default. Please usetorch.serialization.add_safe_globals([_reconstruct])or thetorch.serialization.safe_globals([_reconstruct])context manager to allowlist this global if you trust this class/function.So my problem happened when I try to load collab dataset and get the edge splitting.
Did anyone, can help me, please ?
Thank you a lot !
I'm also having this issue. Any idea on how to fix?
Having the same issue, but your fix doesn't work for me. I have been adding this to my code: from torch_geometric.data.storage import GlobalStorage from torch_geometric.data.data import DataEdgeAttr, DataTensorAttr from numpy.core.multiarray import _reconstruct torch.serialization.add_safe_globals([_reconstruct, DataEdgeAttr, DataTensorAttr, GlobalStorage]) but, even if it's work on torch_geometric objects, it's stuck in failure for _reconstruct of numpy:
File "/pbs/home/v/vcuzinrambau/GCL-Link-Prediction/New/src/datasets.py", line 109, in loaddataset split_edge = dataset.get_edge_split() ^^^^^^^^^^^^^^^^^^^^^^^^ File "/pbs/home/v/vcuzinrambau/GCL-Link-Prediction/.venv/lib/python3.12/site-packages/ogb/linkproppred/dataset_pyg.py", line 77, in get_edge_split train = replace_numpy_with_torchtensor(torch.load(osp.join(path, 'train.pt'))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/pbs/home/v/vcuzinrambau/GCL-Link-Prediction/.venv/lib/python3.12/site-packages/torch/serialization.py", line 1494, in load raise pickle.UnpicklingError(_get_wo_message(str(e))) from None _pickle.UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint. (1) In PyTorch 2.6, we changed the default value of the
weights_onlyargument intorch.loadfromFalsetoTrue. Re-runningtorch.loadwithweights_onlyset toFalsewill likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source. (2) Alternatively, to load withweights_only=Trueplease check the recommended steps in the following error message. WeightsUnpickler error: Unsupported global: GLOBAL numpy.core.multiarray._reconstruct was not an allowed global by default. Please usetorch.serialization.add_safe_globals([_reconstruct])or thetorch.serialization.safe_globals([_reconstruct])context manager to allowlist this global if you trust this class/function.So my problem happened when I try to load collab dataset and get the edge splitting. Did anyone, can help me, please ? Thank you a lot !
I'm also having this issue. Any idea on how to fix?
Has this fix been addressed yet, I also have the same issue and the proposed fixed do not solve this
It's been more than 5 months, and I'm not sure I remember correctly, but I think you have to modify the source code of snap-stanford directly, as it said in my error:
(1) In PyTorch 2.6, we changed the default value of the weights_only argument in torch.load from False to True. Re-running torch.load with weights_only set to False will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
So, following the error message, I have turned the parameter weights_only to its old setting:
train = replace_numpy_with_torchtensor(torch.load(osp.join(path, 'train.pt'), weights_only=False))
Try this, but I'm not sure. Hope that will work !