big-ann-benchmarks add Puck algorithm for track 1.

add Puck algorithm for track 1.

Open nk2014yj opened this issue 2 years ago • 15 comments

Oct 21 '21 07:10 nk2014yj

@nk2014yj Is the PR ready for review and running?

Oct 21 '21 10:10 harsha-simhadri

@nk2014yj Is the PR ready for review and running?

yes

Oct 22 '21 09:10 nk2014yj

@nk2014yj I see the following error when trying to run your algorithm on the bigann-1B dataset. IT looks like the algo needs more disk space. I have 100GB+ left on the drive to which big-ann-benchmarks/data is mapped to. Do you need more space than that? Or is the code trying to write to a different location?

$ python3 run.py --algorithm puck-t1 --dataset bigann-1B file data/bigann/query.public.10K.u8bin already exists file data/bigann/GT.public.1B.ibin already exists 2021-10-28 10:54:17,193 - annb - INFO - running only puck-t1 2021-10-28 10:54:17,217 - annb - INFO - Order: [Definition(algorithm='puck-t1', constructor='Puck', module='benchmark.algorithms.puck_t1', docker_tag='billion-scale-benchmark-puck', arguments=['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}], query_argument_groups=[['10,30,11000,100'], ['10,30,12000,100'], ['10,20,13000,200'], ['10,30,13000,200'], ['10,30,14000,200'], ['10,30,15000,100'], ['10,30,19000,200'], ['10,30,20000,300'], ['10,90,200000,1400'], ['10,100,230000,1300']], disabled=False)] 2021-10-28 10:54:17,677 - annb.5ceb78c5df - INFO - Created container 5ceb78c5df: CPU limit 0-31, mem limit 61131397888, timeout 43200, command ['--dataset', 'bigann-1B', '--algorithm', 'puck-t1', '--module', 'benchmark.algorithms.puck_t1', '--constructor', 'Puck', '--runs', '5', '--count', '10', '["euclidean", {"indexkey": "C10000_F5000_FN16_N32", "url": "https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar"}]', '["10,30,11000,100"]', '["10,30,12000,100"]', '["10,20,13000,200"]', '["10,30,13000,200"]', '["10,30,14000,200"]', '["10,30,15000,100"]', '["10,30,19000,200"]', '["10,30,20000,300"]', '["10,90,200000,1400"]', '["10,100,230000,1300"]'] 2021-10-28 10:54:17,960 - annb.5ceb78c5df - INFO - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] 2021-10-28 10:54:17,961 - annb.5ceb78c5df - INFO - Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Running puck-t1 on bigann-1B 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Got 10000 queries 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Loading index 2021-10-28 10:54:17,976 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: Exiting with failure status due to previous errors 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index 2021-10-28 10:54:32,253 - annb.5ceb78c5df - INFO - ls: cannot access 'puck_index/': Not a directory 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 256 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 1 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Loaded existing index 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Index memory footprint: 21492.0 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Starting query --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mStarting query\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - Running query argument group 1 of 10... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRunning query argument group 1 of 10...\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - <class 'numpy.ndarray'> 10000 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: "\x1b[34m<class 'numpy.ndarray'> 10000\x1b[0m" Arguments: () 2021-10-28 10:54:32,268 - annb.5ceb78c5df - INFO - Run 1/5... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRun 1/5...\x1b[0m' Arguments: () 2021-10-28 10:54:33,206 - annb.5ceb78c5df - ERROR - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) Running puck-t1 on bigann-1B Got 10000 queries Loading index ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device tar: Exiting with failure status due to previous errors ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index ls: cannot access 'puck_index/': Not a directory 256 1 Loaded existing index Index memory footprint: 21492.0 Starting query Running query argument group 1 of 10... <class 'numpy.ndarray'> 10000 Run 1/5...

--- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 301, in run_docker logger.error(colors.color(container.logs().decode(), fg='red')) Message: "\x1b[31m['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]\nTrying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}])\nRunning puck-t1 on bigann-1B\nGot 10000 queries\nLoading index\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device\ntar: Exiting with failure status due to previous errors\n ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index\nls: cannot access 'puck_index/': Not a directory\n256\n1\nLoaded existing index\nIndex memory footprint: 21492.0\nStarting query\nRunning query argument group 1 of 10...\n<class 'numpy.ndarray'> 10000\nRun 1/5...\n\x1b[0m" Arguments: () 2021-10-28 10:54:33,207 - annb.5ceb78c5df - ERROR - Child process for container 5ceb78c5df raised exception 139 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 302, in run_docker logger.error('Child process for container %s raised exception %d' % (container.short_id, exit_code)) Message: 'Child process for container 5ceb78c5df raised exception 139' Arguments: ()

Oct 28 '21 10:10 harsha-simhadri

@nk2014yj I see the following error when trying to run your algorithm on the bigann-1B dataset. IT looks like the algo needs more disk space. I have 100GB+ left on the drive to which big-ann-benchmarks/data is mapped to. Do you need more space than that? Or is the code trying to write to a different location?

$ python3 run.py --algorithm puck-t1 --dataset bigann-1B file data/bigann/query.public.10K.u8bin already exists file data/bigann/GT.public.1B.ibin already exists 2021-10-28 10:54:17,193 - annb - INFO - running only puck-t1 2021-10-28 10:54:17,217 - annb - INFO - Order: [Definition(algorithm='puck-t1', constructor='Puck', module='benchmark.algorithms.puck_t1', docker_tag='billion-scale-benchmark-puck', arguments=['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}], query_argument_groups=[['10,30,11000,100'], ['10,30,12000,100'], ['10,20,13000,200'], ['10,30,13000,200'], ['10,30,14000,200'], ['10,30,15000,100'], ['10,30,19000,200'], ['10,30,20000,300'], ['10,90,200000,1400'], ['10,100,230000,1300']], disabled=False)] 2021-10-28 10:54:17,677 - annb.5ceb78c5df - INFO - Created container 5ceb78c5df: CPU limit 0-31, mem limit 61131397888, timeout 43200, command ['--dataset', 'bigann-1B', '--algorithm', 'puck-t1', '--module', 'benchmark.algorithms.puck_t1', '--constructor', 'Puck', '--runs', '5', '--count', '10', '["euclidean", {"indexkey": "C10000_F5000_FN16_N32", "url": "https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar"}]', '["10,30,11000,100"]', '["10,30,12000,100"]', '["10,20,13000,200"]', '["10,30,13000,200"]', '["10,30,14000,200"]', '["10,30,15000,100"]', '["10,30,19000,200"]', '["10,30,20000,300"]', '["10,90,200000,1400"]', '["10,100,230000,1300"]'] 2021-10-28 10:54:17,960 - annb.5ceb78c5df - INFO - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] 2021-10-28 10:54:17,961 - annb.5ceb78c5df - INFO - Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Running puck-t1 on bigann-1B 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Got 10000 queries 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Loading index 2021-10-28 10:54:17,976 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: Exiting with failure status due to previous errors 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index 2021-10-28 10:54:32,253 - annb.5ceb78c5df - INFO - ls: cannot access 'puck_index/': Not a directory 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 256 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 1 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Loaded existing index 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Index memory footprint: 21492.0 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Starting query --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mStarting query\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - Running query argument group 1 of 10... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRunning query argument group 1 of 10...\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - <class 'numpy.ndarray'> 10000 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: "\x1b[34m<class 'numpy.ndarray'> 10000\x1b[0m" Arguments: () 2021-10-28 10:54:32,268 - annb.5ceb78c5df - INFO - Run 1/5... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRun 1/5...\x1b[0m' Arguments: () 2021-10-28 10:54:33,206 - annb.5ceb78c5df - ERROR - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) Running puck-t1 on bigann-1B Got 10000 queries Loading index ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device tar: Exiting with failure status due to previous errors ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index ls: cannot access 'puck_index/': Not a directory 256 1 Loaded existing index Index memory footprint: 21492.0 Starting query Running query argument group 1 of 10... <class 'numpy.ndarray'> 10000 Run 1/5...

--- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 301, in run_docker logger.error(colors.color(container.logs().decode(), fg='red')) Message: "\x1b[31m['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]\nTrying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}])\nRunning puck-t1 on bigann-1B\nGot 10000 queries\nLoading index\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device\ntar: Exiting with failure status due to previous errors\n ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index\nls: cannot access 'puck_index/': Not a directory\n256\n1\nLoaded existing index\nIndex memory footprint: 21492.0\nStarting query\nRunning query argument group 1 of 10...\n<class 'numpy.ndarray'> 10000\nRun 1/5...\n\x1b[0m" Arguments: () 2021-10-28 10:54:33,207 - annb.5ceb78c5df - ERROR - Child process for container 5ceb78c5df raised exception 139 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 302, in run_docker logger.error('Child process for container %s raised exception %d' % (container.short_id, exit_code)) Message: 'Child process for container 5ceb78c5df raised exception 139' Arguments: ()

hi, harsha.Because our index has more than 1 files, so the download file is in tar format, the tar file need be decompression before search. for BIGANN-1B, the download file is 56G, so it need space on the drive maybe 130G. Is that is ok?

Nov 03 '21 06:11 nk2014yj

@nk2014yj I see the following error when trying to run your algorithm on the bigann-1B dataset. IT looks like the algo needs more disk space. I have 100GB+ left on the drive to which big-ann-benchmarks/data is mapped to. Do you need more space than that? Or is the code trying to write to a different location? $ python3 run.py --algorithm puck-t1 --dataset bigann-1B file data/bigann/query.public.10K.u8bin already exists file data/bigann/GT.public.1B.ibin already exists 2021-10-28 10:54:17,193 - annb - INFO - running only puck-t1 2021-10-28 10:54:17,217 - annb - INFO - Order: [Definition(algorithm='puck-t1', constructor='Puck', module='benchmark.algorithms.puck_t1', docker_tag='billion-scale-benchmark-puck', arguments=['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}], query_argument_groups=[['10,30,11000,100'], ['10,30,12000,100'], ['10,20,13000,200'], ['10,30,13000,200'], ['10,30,14000,200'], ['10,30,15000,100'], ['10,30,19000,200'], ['10,30,20000,300'], ['10,90,200000,1400'], ['10,100,230000,1300']], disabled=False)] 2021-10-28 10:54:17,677 - annb.5ceb78c5df - INFO - Created container 5ceb78c5df: CPU limit 0-31, mem limit 61131397888, timeout 43200, command ['--dataset', 'bigann-1B', '--algorithm', 'puck-t1', '--module', 'benchmark.algorithms.puck_t1', '--constructor', 'Puck', '--runs', '5', '--count', '10', '["euclidean", {"indexkey": "C10000_F5000_FN16_N32", "url": "https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar"}]', '["10,30,11000,100"]', '["10,30,12000,100"]', '["10,20,13000,200"]', '["10,30,13000,200"]', '["10,30,14000,200"]', '["10,30,15000,100"]', '["10,30,19000,200"]', '["10,30,20000,300"]', '["10,90,200000,1400"]', '["10,100,230000,1300"]'] 2021-10-28 10:54:17,960 - annb.5ceb78c5df - INFO - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] 2021-10-28 10:54:17,961 - annb.5ceb78c5df - INFO - Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Running puck-t1 on bigann-1B 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Got 10000 queries 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Loading index 2021-10-28 10:54:17,976 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: Exiting with failure status due to previous errors 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index 2021-10-28 10:54:32,253 - annb.5ceb78c5df - INFO - ls: cannot access 'puck_index/': Not a directory 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 256 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 1 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Loaded existing index 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Index memory footprint: 21492.0 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Starting query --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mStarting query\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - Running query argument group 1 of 10... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRunning query argument group 1 of 10...\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - <class 'numpy.ndarray'> 10000 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: "\x1b[34m<class 'numpy.ndarray'> 10000\x1b[0m" Arguments: () 2021-10-28 10:54:32,268 - annb.5ceb78c5df - INFO - Run 1/5... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRun 1/5...\x1b[0m' Arguments: () 2021-10-28 10:54:33,206 - annb.5ceb78c5df - ERROR - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) Running puck-t1 on bigann-1B Got 10000 queries Loading index ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device tar: Exiting with failure status due to previous errors ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index ls: cannot access 'puck_index/': Not a directory 256 1 Loaded existing index Index memory footprint: 21492.0 Starting query Running query argument group 1 of 10... <class 'numpy.ndarray'> 10000 Run 1/5... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 301, in run_docker logger.error(colors.color(container.logs().decode(), fg='red')) Message: "\x1b[31m['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]\nTrying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}])\nRunning puck-t1 on bigann-1B\nGot 10000 queries\nLoading index\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device\ntar: Exiting with failure status due to previous errors\n ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index\nls: cannot access 'puck_index/': Not a directory\n256\n1\nLoaded existing index\nIndex memory footprint: 21492.0\nStarting query\nRunning query argument group 1 of 10...\n<class 'numpy.ndarray'> 10000\nRun 1/5...\n\x1b[0m" Arguments: () 2021-10-28 10:54:33,207 - annb.5ceb78c5df - ERROR - Child process for container 5ceb78c5df raised exception 139 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 302, in run_docker logger.error('Child process for container %s raised exception %d' % (container.short_id, exit_code)) Message: 'Child process for container 5ceb78c5df raised exception 139' Arguments: ()

hi, harsha.Because our index has more than 1 files, so the download file is in tar format, the tar file need be decompression before search. for BIGANN-1B, the download file is 56G, so it need space on the drive maybe 130G. Is that is ok?

@nk2014yj Where is the puck_index directory located within the docker and where is it mapped to on the local hardware. Please keep in mind that the drive with the OS image and user accounts is 30GB with <10GB free space. We have to map large folders to other mounted drives.

Nov 05 '21 06:11 harsha-simhadri

@nk2014yj I see the following error when trying to run your algorithm on the bigann-1B dataset. IT looks like the algo needs more disk space. I have 100GB+ left on the drive to which big-ann-benchmarks/data is mapped to. Do you need more space than that? Or is the code trying to write to a different location? $ python3 run.py --algorithm puck-t1 --dataset bigann-1B file data/bigann/query.public.10K.u8bin already exists file data/bigann/GT.public.1B.ibin already exists 2021-10-28 10:54:17,193 - annb - INFO - running only puck-t1 2021-10-28 10:54:17,217 - annb - INFO - Order: [Definition(algorithm='puck-t1', constructor='Puck', module='benchmark.algorithms.puck_t1', docker_tag='billion-scale-benchmark-puck', arguments=['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}], query_argument_groups=[['10,30,11000,100'], ['10,30,12000,100'], ['10,20,13000,200'], ['10,30,13000,200'], ['10,30,14000,200'], ['10,30,15000,100'], ['10,30,19000,200'], ['10,30,20000,300'], ['10,90,200000,1400'], ['10,100,230000,1300']], disabled=False)] 2021-10-28 10:54:17,677 - annb.5ceb78c5df - INFO - Created container 5ceb78c5df: CPU limit 0-31, mem limit 61131397888, timeout 43200, command ['--dataset', 'bigann-1B', '--algorithm', 'puck-t1', '--module', 'benchmark.algorithms.puck_t1', '--constructor', 'Puck', '--runs', '5', '--count', '10', '["euclidean", {"indexkey": "C10000_F5000_FN16_N32", "url": "https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar"}]', '["10,30,11000,100"]', '["10,30,12000,100"]', '["10,20,13000,200"]', '["10,30,13000,200"]', '["10,30,14000,200"]', '["10,30,15000,100"]', '["10,30,19000,200"]', '["10,30,20000,300"]', '["10,90,200000,1400"]', '["10,100,230000,1300"]'] 2021-10-28 10:54:17,960 - annb.5ceb78c5df - INFO - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] 2021-10-28 10:54:17,961 - annb.5ceb78c5df - INFO - Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Running puck-t1 on bigann-1B 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Got 10000 queries 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Loading index 2021-10-28 10:54:17,976 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: Exiting with failure status due to previous errors 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index 2021-10-28 10:54:32,253 - annb.5ceb78c5df - INFO - ls: cannot access 'puck_index/': Not a directory 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 256 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 1 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Loaded existing index 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Index memory footprint: 21492.0 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Starting query --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mStarting query\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - Running query argument group 1 of 10... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRunning query argument group 1 of 10...\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - <class 'numpy.ndarray'> 10000 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: "\x1b[34m<class 'numpy.ndarray'> 10000\x1b[0m" Arguments: () 2021-10-28 10:54:32,268 - annb.5ceb78c5df - INFO - Run 1/5... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRun 1/5...\x1b[0m' Arguments: () 2021-10-28 10:54:33,206 - annb.5ceb78c5df - ERROR - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) Running puck-t1 on bigann-1B Got 10000 queries Loading index ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device tar: Exiting with failure status due to previous errors ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index ls: cannot access 'puck_index/': Not a directory 256 1 Loaded existing index Index memory footprint: 21492.0 Starting query Running query argument group 1 of 10... <class 'numpy.ndarray'> 10000 Run 1/5... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 301, in run_docker logger.error(colors.color(container.logs().decode(), fg='red')) Message: "\x1b[31m['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]\nTrying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}])\nRunning puck-t1 on bigann-1B\nGot 10000 queries\nLoading index\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device\ntar: Exiting with failure status due to previous errors\n ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index\nls: cannot access 'puck_index/': Not a directory\n256\n1\nLoaded existing index\nIndex memory footprint: 21492.0\nStarting query\nRunning query argument group 1 of 10...\n<class 'numpy.ndarray'> 10000\nRun 1/5...\n\x1b[0m" Arguments: () 2021-10-28 10:54:33,207 - annb.5ceb78c5df - ERROR - Child process for container 5ceb78c5df raised exception 139 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 302, in run_docker logger.error('Child process for container %s raised exception %d' % (container.short_id, exit_code)) Message: 'Child process for container 5ceb78c5df raised exception 139' Arguments: ()

hi, harsha.Because our index has more than 1 files, so the download file is in tar format, the tar file need be decompression before search. for BIGANN-1B, the download file is 56G, so it need space on the drive maybe 130G. Is that is ok?

@nk2014yj Where is the puck_index directory located within the docker and where is it mapped to on the local hardware. Please keep in mind that the drive with the OS image and user accounts is 30GB with <10GB free space. We have to map large folders to other mounted drives.

The tar file will be decompressed in the root directory of this app in docker container. For deep-1B, index files are in /home/app/deep-1B.C10000_F5000_FN16_N32.puckindex， and the script will create a soft link using cmd "ln -s ./bigann-1B.C10000_F5000_FN16_N32.puckindex/ ./puck_index"

Nov 05 '21 08:11 nk2014yj

@nk2014yj I see the following error when trying to run your algorithm on the bigann-1B dataset. IT looks like the algo needs more disk space. I have 100GB+ left on the drive to which big-ann-benchmarks/data is mapped to. Do you need more space than that? Or is the code trying to write to a different location? $ python3 run.py --algorithm puck-t1 --dataset bigann-1B file data/bigann/query.public.10K.u8bin already exists file data/bigann/GT.public.1B.ibin already exists 2021-10-28 10:54:17,193 - annb - INFO - running only puck-t1 2021-10-28 10:54:17,217 - annb - INFO - Order: [Definition(algorithm='puck-t1', constructor='Puck', module='benchmark.algorithms.puck_t1', docker_tag='billion-scale-benchmark-puck', arguments=['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}], query_argument_groups=[['10,30,11000,100'], ['10,30,12000,100'], ['10,20,13000,200'], ['10,30,13000,200'], ['10,30,14000,200'], ['10,30,15000,100'], ['10,30,19000,200'], ['10,30,20000,300'], ['10,90,200000,1400'], ['10,100,230000,1300']], disabled=False)] 2021-10-28 10:54:17,677 - annb.5ceb78c5df - INFO - Created container 5ceb78c5df: CPU limit 0-31, mem limit 61131397888, timeout 43200, command ['--dataset', 'bigann-1B', '--algorithm', 'puck-t1', '--module', 'benchmark.algorithms.puck_t1', '--constructor', 'Puck', '--runs', '5', '--count', '10', '["euclidean", {"indexkey": "C10000_F5000_FN16_N32", "url": "https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar"}]', '["10,30,11000,100"]', '["10,30,12000,100"]', '["10,20,13000,200"]', '["10,30,13000,200"]', '["10,30,14000,200"]', '["10,30,15000,100"]', '["10,30,19000,200"]', '["10,30,20000,300"]', '["10,90,200000,1400"]', '["10,100,230000,1300"]'] 2021-10-28 10:54:17,960 - annb.5ceb78c5df - INFO - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] 2021-10-28 10:54:17,961 - annb.5ceb78c5df - INFO - Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Running puck-t1 on bigann-1B 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Got 10000 queries 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Loading index 2021-10-28 10:54:17,976 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: Exiting with failure status due to previous errors 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index 2021-10-28 10:54:32,253 - annb.5ceb78c5df - INFO - ls: cannot access 'puck_index/': Not a directory 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 256 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 1 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Loaded existing index 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Index memory footprint: 21492.0 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Starting query --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mStarting query\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - Running query argument group 1 of 10... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRunning query argument group 1 of 10...\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - <class 'numpy.ndarray'> 10000 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: "\x1b[34m<class 'numpy.ndarray'> 10000\x1b[0m" Arguments: () 2021-10-28 10:54:32,268 - annb.5ceb78c5df - INFO - Run 1/5... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRun 1/5...\x1b[0m' Arguments: () 2021-10-28 10:54:33,206 - annb.5ceb78c5df - ERROR - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) Running puck-t1 on bigann-1B Got 10000 queries Loading index ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device tar: Exiting with failure status due to previous errors ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index ls: cannot access 'puck_index/': Not a directory 256 1 Loaded existing index Index memory footprint: 21492.0 Starting query Running query argument group 1 of 10... <class 'numpy.ndarray'> 10000 Run 1/5... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 301, in run_docker logger.error(colors.color(container.logs().decode(), fg='red')) Message: "\x1b[31m['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]\nTrying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}])\nRunning puck-t1 on bigann-1B\nGot 10000 queries\nLoading index\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device\ntar: Exiting with failure status due to previous errors\n ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index\nls: cannot access 'puck_index/': Not a directory\n256\n1\nLoaded existing index\nIndex memory footprint: 21492.0\nStarting query\nRunning query argument group 1 of 10...\n<class 'numpy.ndarray'> 10000\nRun 1/5...\n\x1b[0m" Arguments: () 2021-10-28 10:54:33,207 - annb.5ceb78c5df - ERROR - Child process for container 5ceb78c5df raised exception 139 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 302, in run_docker logger.error('Child process for container %s raised exception %d' % (container.short_id, exit_code)) Message: 'Child process for container 5ceb78c5df raised exception 139' Arguments: ()

hi, harsha.Because our index has more than 1 files, so the download file is in tar format, the tar file need be decompression before search. for BIGANN-1B, the download file is 56G, so it need space on the drive maybe 130G. Is that is ok?

@nk2014yj Where is the puck_index directory located within the docker and where is it mapped to on the local hardware. Please keep in mind that the drive with the OS image and user accounts is 30GB with <10GB free space. We have to map large folders to other mounted drives.

If I update the script and the tar file is decompressed in ./data/ path, like in /home/app/data/deep-1B.C10000_F5000_FN16_N32.puckindex in docker container, will this be ok?

Nov 05 '21 08:11 nk2014yj

@nk2014yj Can we just avoid the tar download and unzip? You can download multiple files into the index location. See diskann-t2 example. See: https://github.com/harsha-simhadri/big-ann-benchmarks/blob/c13dbd9ab4c14b76e726df55635e7fbd11626d5c/benchmark/algorithms/diskann-t2.py#L117

Nov 07 '21 05:11 harsha-simhadri

@nk2014yj I see the following error when trying to run your algorithm on the bigann-1B dataset. IT looks like the algo needs more disk space. I have 100GB+ left on the drive to which big-ann-benchmarks/data is mapped to. Do you need more space than that? Or is the code trying to write to a different location? $ python3 run.py --algorithm puck-t1 --dataset bigann-1B file data/bigann/query.public.10K.u8bin already exists file data/bigann/GT.public.1B.ibin already exists 2021-10-28 10:54:17,193 - annb - INFO - running only puck-t1 2021-10-28 10:54:17,217 - annb - INFO - Order: [Definition(algorithm='puck-t1', constructor='Puck', module='benchmark.algorithms.puck_t1', docker_tag='billion-scale-benchmark-puck', arguments=['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}], query_argument_groups=[['10,30,11000,100'], ['10,30,12000,100'], ['10,20,13000,200'], ['10,30,13000,200'], ['10,30,14000,200'], ['10,30,15000,100'], ['10,30,19000,200'], ['10,30,20000,300'], ['10,90,200000,1400'], ['10,100,230000,1300']], disabled=False)] 2021-10-28 10:54:17,677 - annb.5ceb78c5df - INFO - Created container 5ceb78c5df: CPU limit 0-31, mem limit 61131397888, timeout 43200, command ['--dataset', 'bigann-1B', '--algorithm', 'puck-t1', '--module', 'benchmark.algorithms.puck_t1', '--constructor', 'Puck', '--runs', '5', '--count', '10', '["euclidean", {"indexkey": "C10000_F5000_FN16_N32", "url": "https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar"}]', '["10,30,11000,100"]', '["10,30,12000,100"]', '["10,20,13000,200"]', '["10,30,13000,200"]', '["10,30,14000,200"]', '["10,30,15000,100"]', '["10,30,19000,200"]', '["10,30,20000,300"]', '["10,90,200000,1400"]', '["10,100,230000,1300"]'] 2021-10-28 10:54:17,960 - annb.5ceb78c5df - INFO - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] 2021-10-28 10:54:17,961 - annb.5ceb78c5df - INFO - Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Running puck-t1 on bigann-1B 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Got 10000 queries 2021-10-28 10:54:17,971 - annb.5ceb78c5df - INFO - Loading index 2021-10-28 10:54:17,976 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat 2021-10-28 10:54:32,244 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - tar: Exiting with failure status due to previous errors 2021-10-28 10:54:32,245 - annb.5ceb78c5df - INFO - ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index 2021-10-28 10:54:32,253 - annb.5ceb78c5df - INFO - ls: cannot access 'puck_index/': Not a directory 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 256 2021-10-28 10:54:32,265 - annb.5ceb78c5df - INFO - 1 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Loaded existing index 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Index memory footprint: 21492.0 2021-10-28 10:54:32,266 - annb.5ceb78c5df - INFO - Starting query --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mStarting query\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - Running query argument group 1 of 10... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRunning query argument group 1 of 10...\x1b[0m' Arguments: () 2021-10-28 10:54:32,267 - annb.5ceb78c5df - INFO - <class 'numpy.ndarray'> 10000 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: "\x1b[34m<class 'numpy.ndarray'> 10000\x1b[0m" Arguments: () 2021-10-28 10:54:32,268 - annb.5ceb78c5df - INFO - Run 1/5... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "/usr/lib/python3.6/threading.py", line 884, in _bootstrap self._bootstrap_inner() File "/usr/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/usr/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 291, in stream_logs logger.info(colors.color(line.decode().rstrip(), fg='blue')) Message: '\x1b[34mRun 1/5...\x1b[0m' Arguments: () 2021-10-28 10:54:33,206 - annb.5ceb78c5df - ERROR - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}] Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]) Running puck-t1 on bigann-1B Got 10000 queries Loading index ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat tar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device tar: Exiting with failure status due to previous errors ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index ls: cannot access 'puck_index/': Not a directory 256 1 Loaded existing index Index memory footprint: 21492.0 Starting query Running query argument group 1 of 10... <class 'numpy.ndarray'> 10000 Run 1/5... --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 301, in run_docker logger.error(colors.color(container.logs().decode(), fg='red')) Message: "\x1b[31m['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}]\nTrying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex.tar'}])\nRunning puck-t1 on bigann-1B\nGot 10000 queries\nLoading index\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/filer_data.dat: Wrote only 6656 of 10240 bytes\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_coarse.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/GNOIMI_fine.dat: Cannot write: No space left on device\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/index.dat: Cannot write: No space left on device\n./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat\ntar: ./bigann-1B.C10000_F5000_FN16_N32.puckindex/learn_assign.dat: Cannot write: No space left on device\ntar: Exiting with failure status due to previous errors\n ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex.tar ./puck_index\nls: cannot access 'puck_index/': Not a directory\n256\n1\nLoaded existing index\nIndex memory footprint: 21492.0\nStarting query\nRunning query argument group 1 of 10...\n<class 'numpy.ndarray'> 10000\nRun 1/5...\n\x1b[0m" Arguments: () 2021-10-28 10:54:33,207 - annb.5ceb78c5df - ERROR - Child process for container 5ceb78c5df raised exception 139 --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 998, in emit self.flush() File "/usr/lib/python3.6/logging/init.py", line 978, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start self._popen = self._Popen(self) File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 73, in _launch code = process_obj._bootstrap() File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap self.run() File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 56, in run_worker args.blob_prefix, args.sas_string) File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 302, in run_docker logger.error('Child process for container %s raised exception %d' % (container.short_id, exit_code)) Message: 'Child process for container 5ceb78c5df raised exception 139' Arguments: ()

hi, harsha.Because our index has more than 1 files, so the download file is in tar format, the tar file need be decompression before search. for BIGANN-1B, the download file is 56G, so it need space on the drive maybe 130G. Is that is ok?

@nk2014yj Where is the puck_index directory located within the docker and where is it mapped to on the local hardware. Please keep in mind that the drive with the OS image and user accounts is 30GB with <10GB free space. We have to map large folders to other mounted drives.

If I update the script and the tar file is decompressed in ./data/ path, like in /home/app/data/deep-1B.C10000_F5000_FN16_N32.puckindex in docker container, will this be ok?

Nov 08 '21 05:11 nk2014yj

@nk2014yj Can we just avoid the tar download and unzip? You can download multiple files into the index location. See diskann-t2 example. See:

https://github.com/harsha-simhadri/big-ann-benchmarks/blob/c13dbd9ab4c14b76e726df55635e7fbd11626d5c/benchmark/algorithms/diskann-t2.py#L117

done.

Nov 08 '21 05:11 nk2014yj

sorry，forget to update the download url of dataset random-xs. updated.

Nov 12 '21 04:11 nk2014yj

@nk2014yj Can we just avoid the tar download and unzip? You can download multiple files into the index location. See diskann-t2 example. See: https://github.com/harsha-simhadri/big-ann-benchmarks/blob/c13dbd9ab4c14b76e726df55635e7fbd11626d5c/benchmark/algorithms/diskann-t2.py#L117

done.

@nk2014yj still unable to run. I see the following error when running bigann-1B dataset

2021-11-13 12:08:22,383 - annb.c8d8ed4675 - ERROR - Child process for container c8d8ed4675 raised exception 137 --- Logging error --- Traceback (most recent call last): File "/home/harshasi/anaconda3/lib/python3.8/logging/init.py", line 1085, in emit self.flush() File "/home/harshasi/anaconda3/lib/python3.8/logging/init.py", line 1065, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch code = process_obj._bootstrap(parent_sentinel=child_r) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 52, in run_worker run_docker(definition, args.dataset, args.count, File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 302, in run_docker logger.error('Child process for container %s raised exception %d' % (container.short_id, exit_code)) Message: 'Child process for container c8d8ed4675 raised exception 137' Arguments: () (benchmark) harshasi@f32node1:~/big-ann-benchmarks$ df -h Filesystem Size Used Avail Use% Mounted on udev 32G 0 32G 0% /dev tmpfs 6.3G 888K 6.3G 1% /run /dev/sdb1 29G 22G 7.9G 73% / tmpfs 32G 0 32G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 32G 0 32G 0% /sys/fs/cgroup /dev/sdb15 105M 6.6M 98M 7% /boot/efi /dev/sda1 251G 161G 78G 68% /mnt /dev/sdc 2.0T 166G 1.8T 9% /ssd tmpfs 6.3G 0 6.3G 0% /run/user/1000 tmpfs 6.3G 0 6.3G 0% /run/user/998 (benchmark) harshasi@f32node1:~/big-ann-benchmarks$ ls -l data lrwxrwxrwx 1 harshasi harshasi 10 Oct 28 15:44 data -> /ssd/data/

Nov 13 '21 12:11 harsha-simhadri

@nk2014yj Can we just avoid the tar download and unzip? You can download multiple files into the index location. See diskann-t2 example. See: https://github.com/harsha-simhadri/big-ann-benchmarks/blob/c13dbd9ab4c14b76e726df55635e7fbd11626d5c/benchmark/algorithms/diskann-t2.py#L117

done.

@nk2014yj still unable to run. I see the following error when running bigann-1B dataset

2021-11-13 12:08:22,383 - annb.c8d8ed4675 - ERROR - Child process for container c8d8ed4675 raised exception 137 --- Logging error --- Traceback (most recent call last): File "/home/harshasi/anaconda3/lib/python3.8/logging/init.py", line 1085, in emit self.flush() File "/home/harshasi/anaconda3/lib/python3.8/logging/init.py", line 1065, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch code = process_obj._bootstrap(parent_sentinel=child_r) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 52, in run_worker run_docker(definition, args.dataset, args.count, File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 302, in run_docker logger.error('Child process for container %s raised exception %d' % (container.short_id, exit_code)) Message: 'Child process for container c8d8ed4675 raised exception 137' Arguments: () (benchmark) harshasi@f32node1:~/big-ann-benchmarks$ df -h Filesystem Size Used Avail Use% Mounted on udev 32G 0 32G 0% /dev tmpfs 6.3G 888K 6.3G 1% /run /dev/sdb1 29G 22G 7.9G 73% / tmpfs 32G 0 32G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 32G 0 32G 0% /sys/fs/cgroup /dev/sdb15 105M 6.6M 98M 7% /boot/efi /dev/sda1 251G 161G 78G 68% /mnt /dev/sdc 2.0T 166G 1.8T 9% /ssd tmpfs 6.3G 0 6.3G 0% /run/user/1000 tmpfs 6.3G 0 6.3G 0% /run/user/998 (benchmark) harshasi@f32node1:~/big-ann-benchmarks$ ls -l data lrwxrwxrwx 1 harshasi harshasi 10 Oct 28 15:44 data -> /ssd/data/

the total memory of index files is 57G, I think this maybe ok. I think this error may caused by "tmpfs 32G 0 32G 0% /dev/shm", it looks like the same problem as this url: https://stackoverflow.com/questions/44664900/oserror-errno-28-no-space-left-on-device-docker-but-i-have-space

Two solutions are mentioned: A. B.

could you try this method to see if it can solve the upper problem?

Nov 16 '21 10:11 nk2014yj

@nk2014yj Can we just avoid the tar download and unzip? You can download multiple files into the index location. See diskann-t2 example. See: https://github.com/harsha-simhadri/big-ann-benchmarks/blob/c13dbd9ab4c14b76e726df55635e7fbd11626d5c/benchmark/algorithms/diskann-t2.py#L117

done.

@nk2014yj still unable to run. I see the following error when running bigann-1B dataset 2021-11-13 12:08:22,383 - annb.c8d8ed4675 - ERROR - Child process for container c8d8ed4675 raised exception 137 --- Logging error --- Traceback (most recent call last): File "/home/harshasi/anaconda3/lib/python3.8/logging/init.py", line 1085, in emit self.flush() File "/home/harshasi/anaconda3/lib/python3.8/logging/init.py", line 1065, in flush self.stream.flush() OSError: [Errno 28] No space left on device Call stack: File "run.py", line 6, in main() File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in main [worker.start() for worker in workers] File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 241, in [worker.start() for worker in workers] File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/context.py", line 277, in _Popen return Popen(process_obj) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/popen_fork.py", line 75, in _launch code = process_obj._bootstrap(parent_sentinel=child_r) File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap self.run() File "/home/harshasi/anaconda3/lib/python3.8/multiprocessing/process.py", line 108, in run self._target(*self._args, **self._kwargs) File "/home/harshasi/big-ann-benchmarks/benchmark/main.py", line 52, in run_worker run_docker(definition, args.dataset, args.count, File "/home/harshasi/big-ann-benchmarks/benchmark/runner.py", line 302, in run_docker logger.error('Child process for container %s raised exception %d' % (container.short_id, exit_code)) Message: 'Child process for container c8d8ed4675 raised exception 137' Arguments: () (benchmark) harshasi@f32node1:~/big-ann-benchmarks$ df -h Filesystem Size Used Avail Use% Mounted on udev 32G 0 32G 0% /dev tmpfs 6.3G 888K 6.3G 1% /run /dev/sdb1 29G 22G 7.9G 73% / tmpfs 32G 0 32G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 32G 0 32G 0% /sys/fs/cgroup /dev/sdb15 105M 6.6M 98M 7% /boot/efi /dev/sda1 251G 161G 78G 68% /mnt /dev/sdc 2.0T 166G 1.8T 9% /ssd tmpfs 6.3G 0 6.3G 0% /run/user/1000 tmpfs 6.3G 0 6.3G 0% /run/user/998 (benchmark) harshasi@f32node1:~/big-ann-benchmarks$ ls -l data lrwxrwxrwx 1 harshasi harshasi 10 Oct 28 15:44 data -> /ssd/data/

the total memory of index files is 57G, I think this maybe ok. I think this error may caused by "tmpfs 32G 0 32G 0% /dev/shm", it looks like the same problem as this url: https://stackoverflow.com/questions/44664900/oserror-errno-28-no-space-left-on-device-docker-but-i-have-space

Two solutions are mentioned: A. B.

could you try this method to see if it can solve the upper problem?

I have tried as you said -- wipe up docker images and containers -- and install and run puck from scratch. I still see the following exception 137 (out of memory?). I also don't understand the links you are creating for the index in the logs below. Please email me at [email protected], let's find a time to get on a call and debug?

(benchmark) harshasi@f32node1:~/big-ann-benchmarks$ python3 run.py --algorithm puck-t1 --dataset bigann-1B file data/bigann/query.public.10K.u8bin already exists file data/bigann/GT.public.1B.ibin already exists 2021-11-18 14:37:34,385 - annb - INFO - running only puck-t1 2021-11-18 14:37:34,398 - annb - INFO - Order: [Definition(algorithm='puck-t1', constructor='Puck', module='benchmark.algorithms.puck_t1', docker_tag='billion-scale-benchmark-puck', arguments=['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex'}], query_argument_groups=[['10,30,11000,100'], ['10,30,12000,100'], ['10,20,13000,200'], ['10,30,13000,200'], ['10,30,14000,200'], ['10,30,15000,100'], ['10,30,19000,200'], ['10,30,20000,300'], ['10,90,200000,1400'], ['10,100,230000,1300']], disabled=False)] 2021-11-18 14:37:35,408 - annb.2e0b49e8eb - INFO - Created container 2e0b49e8eb: CPU limit 0-31, mem limit 60814609152, timeout 43200, command ['--dataset', 'bigann-1B', '--algorithm', 'puck-t1', '--module', 'benchmark.algorithms.puck_t1', '--constructor', 'Puck', '--runs', '5', '--count', '10', '["euclidean", {"indexkey": "C10000_F5000_FN16_N32", "url": "https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex"}]', '["10,30,11000,100"]', '["10,30,12000,100"]', '["10,20,13000,200"]', '["10,30,13000,200"]', '["10,30,14000,200"]', '["10,30,15000,100"]', '["10,30,19000,200"]', '["10,30,20000,300"]', '["10,90,200000,1400"]', '["10,100,230000,1300"]'] 2021-11-18 14:37:35,796 - annb.2e0b49e8eb - INFO - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex'}] 2021-11-18 14:37:35,797 - annb.2e0b49e8eb - INFO - Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex'}]) 2021-11-18 14:37:35,836 - annb.2e0b49e8eb - INFO - Running puck-t1 on bigann-1B 2021-11-18 14:37:35,836 - annb.2e0b49e8eb - INFO - Got 10000 queries 2021-11-18 14:37:35,836 - annb.2e0b49e8eb - INFO - data/bigann-1B.C10000_F5000_FN16_N32.puckindex 2021-11-18 14:37:35,836 - annb.2e0b49e8eb - INFO - Loading index 2021-11-18 14:37:35,836 - annb.2e0b49e8eb - INFO - ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex ./puck_index 2021-11-18 14:37:35,850 - annb.2e0b49e8eb - INFO - total 58992308 2021-11-18 14:37:35,850 - annb.2e0b49e8eb - INFO - drwxr-xr-x 2 root root 4096 Nov 13 11:52 . 2021-11-18 14:37:35,850 - annb.2e0b49e8eb - INFO - drwxrwxrwx 8 root root 4096 Nov 13 11:22 .. 2021-11-18 14:37:35,850 - annb.2e0b49e8eb - INFO - -rw-r--r-- 1 root root 5160000 Nov 13 11:28 GNOIMI_coarse.dat 2021-11-18 14:37:35,850 - annb.2e0b49e8eb - INFO - -rw-r--r-- 1 root root 2580000 Nov 13 11:28 GNOIMI_fine.dat 2021-11-18 14:37:35,850 - annb.2e0b49e8eb - INFO - -rw-r--r-- 1 root root 20000147456 Nov 13 11:27 filer_data.dat 2021-11-18 14:37:35,850 - annb.2e0b49e8eb - INFO - -rw-r--r-- 1 root root 4400040087 Nov 13 11:29 index.dat 2021-11-18 14:37:35,850 - annb.2e0b49e8eb - INFO - -rw-r--r-- 1 root root 36000163840 Nov 13 11:52 learn_assign.dat

2021-11-18 14:41:44,990 - annb.2e0b49e8eb - ERROR - ['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex'}] Trying to instantiate benchmark.algorithms.puck_t1.Puck(['euclidean', {'indexkey': 'C10000_F5000_FN16_N32', 'url': 'https://gips-test-bucket-0-gz.gz.bcebos.com/similar/bigann-1B.puckindex'}]) Running puck-t1 on bigann-1B Got 10000 queries data/bigann-1B.C10000_F5000_FN16_N32.puckindex Loading index ln -s data/bigann-1B.C10000_F5000_FN16_N32.puckindex ./puck_index total 58992308 drwxr-xr-x 2 root root 4096 Nov 13 11:52 . drwxrwxrwx 8 root root 4096 Nov 13 11:22 .. -rw-r--r-- 1 root root 5160000 Nov 13 11:28 GNOIMI_coarse.dat -rw-r--r-- 1 root root 2580000 Nov 13 11:28 GNOIMI_fine.dat -rw-r--r-- 1 root root 20000147456 Nov 13 11:27 filer_data.dat -rw-r--r-- 1 root root 4400040087 Nov 13 11:29 index.dat -rw-r--r-- 1 root root 36000163840 Nov 13 11:52 learn_assign.dat

2021-11-18 14:41:44,991 - annb.2e0b49e8eb - ERROR - Child process for container 2e0b49e8eb raised exception 137

Nov 18 '21 15:11 harsha-simhadri

I see the following results

puck-t1,"Puck(10,20,20000,2000)",text2image-1B,10,8421.55159395627,0.0,1000000.0,58235268.0,6915.028347245721,0,0,0.18284499999999998 puck-t1,"Puck(10,20,19000,700)",text2image-1B,10,9992.153845145944,0.0,1000000.0,58235268.0,5828.0996172101495,0,0,0.169676 puck-t1,"Puck(10,20,10000,1000)",text2image-1B,10,11486.521144580363,0.0,1000000.0,58235268.0,5069.878622691336,0,0,0.156922 puck-t1,"Puck(10,10,8000,500)",text2image-1B,10,13356.761913581453,0.0,1000000.0,58235268.0,4359.98398240408,0,0,0.139568 puck-t1,"Puck(10,20,20000,1000)",text2image-1B,10,9476.185872600267,0.0,1000000.0,58235268.0,6145.433276945658,0,0,0.175994 puck-t1,"Puck(10,30,12000,800)",text2image-1B,10,11071.05747014423,0.0,1000000.0,58235268.0,5260.136003904362,0,0,0.160653 puck-t1,"Puck(10,20,9000,700)",text2image-1B,10,12278.106292357348,0.0,1000000.0,58235268.0,4743.017091833553,0,0,0.149852 puck-t1,"Puck(10,10,6000,500)",text2image-1B,10,13968.31473840465,0.0,1000000.0,58235268.0,4169.097639236841,0,0,0.130515 puck-t1,"Puck(10,30,11000,1000)",text2image-1B,10,11007.853983343128,0.0,1000000.0,58235268.0,5290.3379794209195,0,0,0.16063 puck-t1,"Puck(10,30,14000,600)",text2image-1B,10,10895.878411849368,0.0,1000000.0,58235268.0,5344.7061171927735,0,0,0.160987 puck-t1,"Puck(10,90,100000,900)",msturing-1B,10,4001.8958871850186,0.0,1000000.0,58110520.0,14520.747575188827,0,0,0.8428610000000001 puck-t1,"Puck(10,30,20000,200)",msturing-1B,10,11864.653408637023,0.0,1000000.0,58110520.0,4897.784873993683,0,0,0.773385 puck-t1,"Puck(10,50,20000,300)",msturing-1B,10,11351.811248596823,0.0,1000000.0,58110520.0,5119.052698060227,0,0,0.7820280000000001 puck-t1,"Puck(10,60,23000,200)",msturing-1B,10,10793.241367891951,0.0,1000000.0,58110520.0,5383.972990066623,0,0,0.787577 puck-t1,"Puck(10,70,24000,300)",msturing-1B,10,10335.074102827077,0.0,1000000.0,58110520.0,5622.651509010887,0,0,0.79199 puck-t1,"Puck(10,20,22000,300)",msturing-1B,10,11461.173484450277,0.0,1000000.0,58110520.0,5070.206822961044,0,0,0.767586 puck-t1,"Puck(10,70,25000,300)",msturing-1B,10,10139.28810483003,0.0,1000000.0,58110520.0,5731.22288263197,0,0,0.793812 puck-t1,"Puck(10,40,19000,200)",msturing-1B,10,11683.661252533644,0.0,1000000.0,58110520.0,4973.656694077683,0,0,0.7760849999999999 puck-t1,"Puck(10,80,19000,600)",msturing-1B,10,10814.728634059795,0.0,1000000.0,58110520.0,5373.27583208952,0,0,0.7826660000000001 puck-t1,"Puck(10,60,19000,200)",msturing-1B,10,11713.697869548969,0.0,1000000.0,58110520.0,4960.903093724537,0,0,0.779256 puck-t1,"Puck(10,20,2000,200)",random-xs,10,177049.55677501054,0.0,1000000.0,2492.0,0.014075155258178712,0,0,0.9885999999999999

Jan 04 '22 20:01 harsha-simhadri

big-ann-benchmarks big-ann-benchmarks copied to clipboard

add Puck algorithm for track 1.

big-ann-benchmarks
big-ann-benchmarks copied to clipboard