kedro-plugins icon indicating copy to clipboard operation
kedro-plugins copied to clipboard

Published 20 hours ago • verified publisher icon kedro-org

Replace eager loading with lazy loading in `IncrementalDataSet`

Open jmholzer opened this issue 1 year ago • 3 comments

Description

In tech design 21/07/23 we decided to implement lazy loading in IncrementalDataSet. See the comment under https://github.com/kedro-org/kedro/issues/2456. A prerequisite for this is that it is first migrated to kedro-datasets (https://github.com/kedro-org/kedro-plugins/issues/249).

jmholzer avatar Jun 21 '23 15:06 jmholzer

I originally designed the pull request for the main repository, so I would start working on these two issues (#249 and #250) here now if that is ok for you?

PtrBld avatar Jun 26 '23 12:06 PtrBld

I originally designed the pull request for the main repository, so I would start working on these two issues (#249 and #250) here now if that is ok for you?

That's great, thanks @PtrBld !

jmholzer avatar Jun 26 '23 12:06 jmholzer

This is ready (finally!) to be done here on kedro-datasets, feel free to cherry-pick https://github.com/kedro-org/kedro/pull/2497

astrojuanlu avatar Oct 23 '23 13:10 astrojuanlu