malloc icon indicating copy to clipboard operation
malloc copied to clipboard

Support SharedArrayBuffer please :)

Open benjamingr opened this issue 6 years ago • 10 comments

benjamingr avatar Jun 13 '18 14:06 benjamingr

it was always the plan but I ran out of steam on the project I was using this for. I'd still like to add it but with SAB being spectred out of browsers the only target would be node :(

phpnode avatar Jun 13 '18 15:06 phpnode

and can we even rely on SharedArrayBuffer sticking around in V8 if chrome doesn't use it any more?

phpnode avatar Jun 13 '18 15:06 phpnode

@phpnode hey, nice to see you here!

I specifically want to use it or Node.js since workers landed.

I want to use this package to build thread-safe data structures for Node.js using the experimental --worker flag since I needed an allocator :)

benjamingr avatar Jun 13 '18 15:06 benjamingr

and can we even rely on SharedArrayBuffer sticking around in V8 if chrome doesn't use it any more?

@bmeurer is it safe to say "yes"?

Also, as far as I know, SharedArrayBuffer isn't going anywhere and there are plans to re-enable it in the future in Chrome.

benjamingr avatar Jun 13 '18 15:06 benjamingr

Hey @benjamingr! it would be possible to make the existing implementation work with SAB simply by changing every store and load to an Atomics.store() and Atomics.load() but I suspect it would be horribly slow.

This paper describes a supposedly practical lock-free skiplist which might be worth looking at: http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-579.pdf

phpnode avatar Jun 13 '18 16:06 phpnode

This paper describes a supposedly practical lock-free skiplist which might be worth looking at: http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-579.pdf

That's cool, I think initially just putting a hook around malloc/free would be enough for starters :)

benjamingr avatar Jun 13 '18 17:06 benjamingr

Making the alloc() and free() calls thread safe is one thing but there's also the question of how to safely write and read within those allocated chunks. Maybe that's a concern to be left to other libraries though?

phpnode avatar Jun 13 '18 17:06 phpnode

btw this was the project that built on top of this library, sounds broadly similar to what you need (not threadsafe though sadly) https://github.com/codemix/reign

phpnode avatar Jun 13 '18 17:06 phpnode

@benjamingr "yes"!

bmeurer avatar Jun 13 '18 17:06 bmeurer

Maybe that's a concern to be left to other libraries though?

Yes, I think it is - libraries can use Atomics themselves (at least I was anticipating so) for the read/write. The important part is to have the SharedArrayBuffer shared between the workers - it's fine (for me) if only one worker can allocate/free memory and then pass that to the other workers.

benjamingr avatar Jun 13 '18 18:06 benjamingr