Avian can't use more than ~2GB of heap space

[dicej]

We're using 32-bit integers (signed in some cases) all over heap.cpp to represent sizes, so we overflow when we get close to 2^31. We should replace these fields and variables with 64-bit integers.

[joshuawarner32]

I would say it's actually more appropriate to replace it with something like intptr_t, uintptr_t, or size_t. That way, we don't waste space or cycles on 32-bit platforms.

[dicej]

On Wed, 6 Mar 2013, Joshua Warner wrote:

I would say it's actually more appropriate to replace it with something like intptr_t, uintptr_t, or size_t. That way, we don't waste space or cycles on 32-bit platforms.

Agreed.

[Techcable]

This makes avian pretty much unusable for me.

[joshuawarner32]

@Techcable - could you describe your use-case a little more?

Keep in mind that even if we bump the integer sizes, it's unlikely the GC algorithm will scale to large heap sizes anywhere near as well as e.g. hotspot's will.

[Techcable]

Couldn't you use reference counting and garbage collection?

[joshuawarner32]

Avian does use garbage collection. Only toy VMs can really get away without it (and PHP, apparently, but that's another story). Essentially, the problem is that the Avian's GC algorithm, which is a fairly basic generational copying collector, hasn't had anywhere near the amount of tuning that many other VM's garbage collectors have had.

Reference counting, in it's most basic form, will likely make things worse in all but the largest heaps, far beyond the capacity of even HotSpot's GC - we're talking hundreds of GB here. There are some variants of reference counting that start to be competitive on smaller heap sizes (what Avian is designed for anyway), but those get dramatically more complicated - so we come back to the matter of how much effort has been poured into development. It hasn't been a priority, and even then, it's not clear it would be a net win.

So, I ask again - could you describe your use-case a little more?