opendylan icon indicating copy to clipboard operation
opendylan copied to clipboard

Desired size 13039606 exceeds maximum table vector size

Open cgay opened this issue 4 years ago • 2 comments

It's 2022, we should be able to put 10_000_000 elements in a hash table.

define function main
    (name :: <string>, arguments :: <vector>)
  let n = string-to-integer(arguments[0]);
  let t = make(<table>);
  for (k from 1 to n)
    t[k] := k;
  end;
end function main;

main(application-name(), application-arguments());
$ _build/bin/table-size 10000000
GC Warning: Repeated allocation of very large block (appr. size 7061504):
        May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 69545984):
        May lead to memory leak and poor performance
Desired size 13039606 exceeds maximum table vector size.
Backtrace:
  0
  default-handler:dylan:dylan##1 + 0x12
  0x79643a6e616c7964
  default-last-handler:common-dylan-internals:common-dylan##0 + 0x2f6
  error:dylan:dylan##0 + 0x23
  error:dylan:dylan##1 + 0x76
  compute-entry-count:internal:dylan + 0x110
  make-table-vector:internal:dylan + 0x56
  rehash-table:dylan-extensions:dylan + 0x21a
  puthash:internal:dylan + 0x5b
  main:table-size:table-size + 0x8e

Looking at dylan/table.dylan briefly, it seems like https://github.com/dylan-lang/opendylan/blob/89e84131706393e255be12a388853191c12b9d46/sources/dylan/table.dylan#L37 could easily be the problem. Probably designed with 32-bit limits in mind.

(The GC warning for a 7MB allocation seems kind of quaint also at this point, but that's obviously a separate issue.)

cgay avatar Dec 25 '21 04:12 cgay

We can generate a new sequence of primes, but what do you think would be a reasonable limit? 5.0e8?

housel avatar Aug 03 '22 19:08 housel

"5.0e8 should be enough for anyone." --me, 2022

As usual I wish I had put more context when I filed this bug. I suspect it was something someone else was doing in a different language and I tried it in Dylan.

cgay avatar Aug 03 '22 21:08 cgay