annoy-java
annoy-java copied to clipboard
Only supports files up to 2GB
Because the API for MappedByteBuffer, or whatever it's called, uses Java int for offsets, you cannot address past 2GB. To work around this, annoy-java would have to open multiple windows into the file as separate buffers 2gb apart.
If there is a need for this, please comment here.
This would be nice to have.
+1 that this would be nice to have
Hi! I am using this library now. I am curious whether files larger than 2GB can be supported now since my index file is around 100GB. Thanks.
Dear @Yuanrui1994, I've just submitted a PR with a solution to this restriction. It ran successfully in my index (3GB). It would be nice if you also execute some tests in your index of 100GB.