modAL icon indicating copy to clipboard operation
modAL copied to clipboard

using fit_generator() with modAL

Open Tinzer opened this issue 6 years ago • 1 comments

hello, I'm using too much memory when I'm reading images, so I want to use fit_generator() in keras to training. Is it possible to use modAL with fit_generator() method? Or use another way to yield batches for training? Thank you!

Tinzer avatar Jun 19 '19 07:06 Tinzer

Currently, the .fit_generator() method of Keras is not supported by the scikit-learn wrapper also provided by Keras. Without this, there is no way to access this method at the moment, since modAL requires a scikit-learn interface from its estimators.

However, there is a workaround which might help. Before passing your model to the ActiveLearner, you can train it by iterating through the batches in the generator and training manually by calling fit. This would also work with PyTorch models. (And for PyTorch, this is the proper way to train from generator anyway.)

As a sidenote, lately I have been thinking that it would be a good idea to abandon the scikit-learn wrappers and write modAL in a way such that Keras was in the beginning: handling each backend specifically but hiding it from the user, providing a unified API. This of course would be a lot of work, but it would be a logical step in the evolution of modAL.

cosmic-cortex avatar Jun 19 '19 19:06 cosmic-cortex