Eliminating bignums by leveraging space unsafety

[sorear]

COUNT_LIST is a fairly hot function in the current compiler (a different issue, but let's pretend it's desired). It is compiled with bignum support, but a human can easily see that bignums aren't actually reachable there: the numbers represent elements in a list that exists, and memory will always be exhausted before the largest representable number is reached. Can we automate this reasoning in any way? Does any existing ML implementation do such reasoning?

Any such optimization would need to be fairly late since it needs to assume no deforestation-type optimization will run subsequently.

[tanyongkiam]

My understanding is that the bignum library already tries pretty hard to use small num operations where possible. So the overhead that would be eliminated here would be the checks that args/results fit inside small nums.

If I remember correctly, some ML compilers do not have this distinction (i.e., just run bignum computation regardless of size).

[sorear]

Yes, this is about the bignum checks. I believe you are correct that the bignum code is never invoked unless one argument or the result does not fit in a fixnum.