Cache size > 2000 not recognised

[JonathanTay]

Noticed this while using libSVM in sklearn - training an SVM when cache_size > 2000 or so on large problems does not seem to lead to any benefit/speed up. Looking at RAM usage, it shows that usage is still about 200MB (which is roughly the original dataset size, rather than the Kernel matrix size). Looks like the issue is in svm.cpp, where the cache size is set to (long int) cache_size*(1<<20). I suspect this overflows for cases where for example, cache_size=4000.

Testing done using Anaconda 2.4.1 on Windows 8.1, x64 processor.

[amueller]

Btw, this is compiler specific. It overflows when long int is 32 bit and works if its 64 bit.

[JaKasb]

The data type for number-of-bytes variables should be size_t.

Quote www.cplusplus.com/reference/cstddef/size_t/

It is a type able to represent the size of any object in bytes: size_t is the type returned by the sizeof operator and is widely used in the standard library to represent sizes and counts.

For further references: http://www.embedded.com/electronics-blogs/programming-pointers/4026076/Why-size-t-matters http://en.cppreference.com/w/cpp/types/size_t

[EkaterinaAbramova]

Hi sorry I am confused, this is not working for me on MacBook Pro M2 with python 3.11.8. Nomatter what cache I try the time it takes to train SVM is the same. How can cache_size actually work please? I tried changing it locally in _classes.py of the SVM library but it still doesn't help. I tried with cache 500 and also with less than 2000 and still no change.

[EkaterinaAbramova]

Ive also tried using these two compilers in my Spyder instead: env['CC'] = '/usr/local/bin/gcc-12' env['CXX'] = '/usr/local/bin/g++-12' and it didn't make any improvement on the speed using cache of 5000.

[cjlin1]

Can you show us details of your experiments?

On 2024-06-27 04:17, Ekaterina Abramova wrote:

Ive also tried using these two compilers in my Spyder instead: env['CC'] = '/usr/local/bin/gcc-12' env['CXX'] = '/usr/local/bin/g++-12' and it didn't make any improvement on the speed using cache of 5000.

-- Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. You are receiving this because you are subscribed to this thread.Message ID: @.> [ { @.": "http://schema.org", @.": "EmailMessage", "potentialAction": { @.": "ViewAction", "target": "https://github.com/cjlin1/libsvm/issues/80#issuecomment-2192552014", "url": "https://github.com/cjlin1/libsvm/issues/80#issuecomment-2192552014", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { @.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

Links:

[1] https://github.com/cjlin1/libsvm/issues/80#issuecomment-2192552014 [2] https://github.com/notifications/unsubscribe-auth/ABI3BHQFL6F37KHD3CYO7J3ZJMOWXAVCNFSM6AAAAABJ6NDBTWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJSGU2TEMBRGQ

[EkaterinaAbramova]

It would take me a while to write up the code such that it can be a simple one. Are you saying you are not aware of the problem? This has been an ongoing problem for a year at least. I am on MacBook Pro M2 chip python 3.11.8 with 96 GB of ram. So setting cache_size to 5000 should work well. Is there any specific information you need form me please, as I am sure this can be easily reproduced with any example. The cache_size simply doesn't do anything for me.

[cjlin1]

Please provide an example with exp log. Thanks

On 2024-06-27 07:36, Ekaterina Abramova wrote:

It would take me a while to write up the code such that it can be a simple one. Are you saying you are not aware of the problem? This has been an ongoing problem for a year at least. I am on MacBook Pro M2 chip python 3.11.8 with 96 GB of ram. So setting cache_size to 5000 should work well. Is there any specific information you need form me please, as I am sure this can be easily reproduced with any example. The cache_size simply doesn't do anything for me.

-- Reply to this email directly, view it on GitHub [1], or unsubscribe [2]. You are receiving this because you commented.Message ID: @.> [ { @.": "http://schema.org", @.": "EmailMessage", "potentialAction": { @.": "ViewAction", "target": "https://github.com/cjlin1/libsvm/issues/80#issuecomment-2192790439", "url": "https://github.com/cjlin1/libsvm/issues/80#issuecomment-2192790439", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { @.***": "Organization", "name": "GitHub", "url": "https://github.com" } } ]

Links:

[1] https://github.com/cjlin1/libsvm/issues/80#issuecomment-2192790439 [2] https://github.com/notifications/unsubscribe-auth/ABI3BHWHP3T5YWVXSI2CUTLZJNF7HAVCNFSM6AAAAABJ6NDBTWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJSG44TANBTHE