scikit-learn-intelex icon indicating copy to clipboard operation
scikit-learn-intelex copied to clipboard

memory leak using SVR

Open montagne5641 opened this issue 1 year ago • 7 comments

When using SVR with patch, a memory leak occurs every time an assignment is made repeatedly with for. This does not occur if patch is not used. Similar issues have been reported previously with SVC, and daal4py was suggested as an alternative at the time. However, SVR does not support daal4py. Is it possible to address the memory leak?

#----------------------------------------------------- use_intelex =1

if use_intelex==1: from sklearnex import patch_sklearn patch_sklearn()

from sklearn.svm import SVR import numpy as np

X = np.random.randint(10, size=(1000,500)) y = np.random.randint(2, size=(1000))

for i in range(10000): results = SVR().fit(X,y) #-----------------------------------------------------

montagne5641 avatar Jun 15 '24 13:06 montagne5641

This issue was not resolved by using del and go.collect().... I think this problem becomes even more pronounced when used in combination with gridsearchcv etc.


import gc

use_intelex =1

if use_intelex==1: from sklearnex import patch_sklearn patch_sklearn()

import os from sklearn.svm import SVR import numpy as np

X = np.random.randint(10, size=(1000,500)) y = np.random.randint(2, size=(1000))

for i in range(10000): results = SVR().fit(X,y) del results gc.collect()

montagne5641 avatar Jun 19 '24 13:06 montagne5641

Thank you @montagne5641 for the report! In order to reproduce the issues, please share the version of daal4py and scikit-learn-intelex and more details about your env

samir-nasibli avatar Jun 20 '24 12:06 samir-nasibli

dear samir-nasibli Thank you. This is my environment. If you need any other information, I'll be happy to help.

daal4py==2024.4.0 scikit-learn-intelex==2024.4.0 scikit-learn==1.5.0 Windows 10 Pro

Note: The same issue occurred even when I lowered the version as shown below. daal4py-2023.1.1 scikit-learn-intelex-2023.1.1

montagne5641 avatar Jun 20 '24 21:06 montagne5641

Any update on this?

montagne5641 avatar Jul 06 '24 07:07 montagne5641

Same here, any updates?

jamescporter avatar Sep 30 '24 14:09 jamescporter

This issue was not resolved by using del and go.collect().... I think this problem becomes even more pronounced when used in combination with gridsearchcv etc.

import gc

use_intelex =1

if use_intelex==1: from sklearnex import patch_sklearn patch_sklearn()

import os from sklearn.svm import SVR import numpy as np

X = np.random.randint(10, size=(1000,500)) y = np.random.randint(2, size=(1000))

for i in range(10000):

results = SVR().fit(X,y) del results gc.collect()

did you find that you would always run out of memory doing a grid search using this?

jamescporter avatar Sep 30 '24 14:09 jamescporter

The issue is confirmed to be on side of sklearnex data management. Investigation is still in progress.

Alexsandruss avatar Jan 29 '25 09:01 Alexsandruss

This is now solved in the latest release (2025.7).

david-cortes-intel avatar Jul 10 '25 06:07 david-cortes-intel