thundersvm icon indicating copy to clipboard operation
thundersvm copied to clipboard

thundersvm is very slow for large dataset on preparation before train with GPU

Open liningbo opened this issue 1 year ago • 0 comments

environment: windows 10 python 3.8 cuda 11.6 thundersvm commit :

smalll dataset train log: svm fit begin: (404, 13) (404,) 2024-04-06 20:30:50,603 INFO [default] #instances = 404, #features = 13 2024-04-06 20:30:50,825 INFO [default] total memory size is 0.000144489 max mem size is 8 2024-04-06 20:30:50,825 INFO [default] free mem is 7.99986 2024-04-06 20:30:50,825 INFO [default] working set size = 512 2024-04-06 20:30:50,825 INFO [default] training start 2024-04-06 20:30:51,208 INFO [default] global iter = 0, total local iter = 130, diff = 44.8 2024-04-06 20:30:51,219 INFO [default] global iter = 3, total local iter = 207, diff = 0.0999313 2024-04-06 20:30:51,219 INFO [default] training finished 2024-04-06 20:30:51,219 INFO [default] obj = -1203.39 2024-04-06 20:30:51,219 INFO [default] rho = -20.8858 2024-04-06 20:30:51,222 INFO [default] #sv = 395 2024-04-06 20:30:51,223 INFO [default] training finished train time elapse(s):0.6843371391296387

large dataset train log: svm fit begin: (1842924, 137) (1842924,) 2024-04-06 19:24:33,588 INFO [default] #instances = 1842924, #features = 137 2024-04-06 20:21:35,682 INFO [default] total memory size is 5.76696 max mem size is 8 2024-04-06 20:21:35,682 INFO [default] free mem is 2.23304 2024-04-06 20:21:35,682 INFO [default] working set size = 128 2024-04-06 20:21:35,935 INFO [default] training start 2024-04-06 20:21:36,376 INFO [default] global iter = 0, total local iter = 64, diff = 22.3704 2024-04-06 20:21:37,309 INFO [default] global iter = 5, total local iter = 224, diff = 18.7142 2024-04-06 20:21:37,309 INFO [default] training finished 2024-04-06 20:21:37,323 INFO [default] obj = -1547.53 2024-04-06 20:21:37,339 INFO [default] rho = 2.02697 2024-04-06 20:21:37,343 INFO [default] #sv = 448 2024-04-06 20:21:45,370 INFO [default] training finished train time elapse(s):3582.7167501449585

max_mem_size is set to -1, the time between the first log and the second log cost nearly 1 hour before training. all parameter: self.params = { 'kernel': 'rbf', 'degree':6, 'gamma':'auto', 'coef0':0.0, 'C':1.0, 'epsilon':0.1, 'tol':0.1, 'probability':False, 'shrinking':False, 'cache_size':None, 'verbose':True, 'max_iter':5, 'n_jobs':-1, 'max_mem_size':-1, 'gpu_id':self.config.gpu_id, }

liningbo avatar Apr 06 '24 12:04 liningbo