NimbusML
NimbusML copied to clipboard
Expose IDataView in python
Currently when data is transformed as part of pipeline:
pipeline.transform(data)
the output is pandas dataframe.
It would be good to return a pointer to a IDataView object and realize it into a data later.
idataview = pipeline.transform(data)
...
while(idataview.move_next())
data = idataview.read_next()
or read in batch:
while(idataview.move_next())
data = idataview.read_next(batch_size)
this potentially would allow us to plugin our pipeline into other frameworks that operate on batches of data like pytorch.