Implementing Incremental Fit for LMNN

[sidak]

Currently, each time the fit method is called, it starts from a new identity matrix. But in many scenarios, it can be useful to incrementally fit (i.e. start from the last computed L).

I think that I can utilize the last L is stored in fit function. Then we can use a flag (let's say incremental) which can be passed in the fit and the _process_inputs functions. But I am still how thinking on how to handle it for base_LMNN class.

Also, does the LMNN version in modshogun allow for incremental fit?

Thank you so much! :)

[perimosocordiae]

I like the idea of allowing an incremental fit, but we should be careful about how the API should look. Perhaps we can follow the scikit-learn convention and add a partial_fit method which doesn't reset the L matrix.

The shogun version allows a custom L matrix to be used as the starting point of the optimization, so can work for both implementations.

[javad314]

Do we have something like partial_fit in LMNN in this package? Unfortunately, this is not possible to use it for big data. I would be very grateful if you can give me any solution for this matter. thank you very much

[bellet]

Hi @javad314, thanks for your interest. We currently do not have this implemented. More generally we know that we should implement solvers that scale better with large datasets, eg based on SGD.

There is currently a PR implementing a new algorithm which learns on triplets like LMNN along with a stochastic optimization solver: #278

We also welcome contributions that would improve scalability of existing algorithms