kedro-plugins
kedro-plugins copied to clipboard
fix(datasets): Use `put()` and `get()` instead of `copy` in `TensorFlowModelDataset`'s `_save` and `_load` methods.
Description
This PR resolves the issue in #839, where saving TensorFlowModelDataset
to Azure Blob Storage fails due to incorrect use of .copy()
from the fsspec.filesystem
interface.
Development notes
The issue occurs in the _save
and _load
methods on lines 147 and 172, where .copy()
is used. According to the fsspec documentation, .copy()
is intended for remote-to-remote transfers. Since this involves copying from a local filesystem to remote storage, .put()
should be used for saving and .get()
for loading.
The code has been updated to use .put()
and .get()
accordingly, replacing the use of .copy()
.
Both methods work for local-to-local and local-to-remote(& vice versa) transfers based on testing.
Checklist
- [ ] Opened this PR as a 'Draft Pull Request' if it is work-in-progress
- [ ] Updated the documentation to reflect the code changes
- [ ] Added a description of this change in the relevant
RELEASE.md
file - [ ] Added tests to cover my changes