Make multithreading scalable

[LADSoft]

space related to threads is allocated on the local heap by the C runtime. The default heap size is 1MB. That means with enough threads, the heap is going to overflow. it will be worse if there is a lot of thread local data in addition to the runtime's allocation of thread local data.

Default operation when that happens is for abort() to be called. Omake exhibits this problem when the -j switch is used.

Long term we need to get these allocations off the local heap to make threading scalable. Short term, we are just going to increase the heap size on omake.

[chuggafan]

I think a raw -j should basically be a "last ditch you have an infinite number of computers at your beck and call and something like distcc" move, when I -j on GNU Make it similarly will crash-out the system.

Would it be possible instead of abort to just allow whatever Out Of Memory killer to kill the memory access? That's how the Linux GNU Make normally kills him.

[LADSoft]

so im thinking, if we don't use LocalAlloc and instead just use malloc or new then if we could have lots more threads before we run out of memory, and if we really run out of memory we've got other problems.... The only reason for using LocalAlloc was to use LMEM_FIXED but I don't suppose our main memory store is going to go moving around lol...