Timeline for adding manually allocated hashtables?

[cyertai]

Hello,

Very nice project - I've been looking for the right way to use the convenience of golang along with manually managed memory. In my view, now that there are shared_ptr implementations that are extremely performant (e.g. Anderson's) golang should switch to being ref counted.

Since that will never happen, your project could be the next best thing. I see there is an open branch implementing hashmaps; I'm curious what you are thinking on timeline? I would offer to help, but am unfortunately not an expert on golang internals.

My use case is the same as many - server with a large hash table and I want to avoid gc pauses.

Kind Regards

[joetifa2003]

First thank you so much!

I was busy with college and work and couldn't contribute more to the project, but going to comeback to this.

The issue with the current implementation was it wasn't stable with purego (c ffi without cgo) and I stopped working on it at the time. will experiment again during next week and hopefully get this working

[joetifa2003]

I will mention here if I have it working and you can contribute by just testing it! or adding more unit tests.

[joetifa2003]

Working on it here #6! Will need to add some tests and settle on the API then it's done!

[joe-at-startupmedia]

@joetifa2003 this looks like a super awesome project. I'm curious if you're aware that anything else besides this project (which seems to have stagnated) is being worked on? I don't know but I think giving developers the option to manage their own memory in Golang would quell a lot of performance concerns solely due to the garbage collection feature. On the other hand maybe the power of Golang comes from its training wheels that force terrible devs (like myself) to relinquish control to the garbage collector. It seems that this library is data-structure centric with regard to how memory can be manually managed and to date there are only a few data structures implemented, is that a safe assessment?

[joetifa2003]

@joetifa2003 this looks like a super awesome project. I'm curious if you're aware that anything else besides this project (which seems to have stagnated) is being worked on? I don't know but I think giving developers the option to manage their own memory in Golang would quell a lot of performance concerns solely due to the garbage collection feature. On the other hand maybe the power of Golang comes from its training wheels that force terrible devs (like myself) to relinquish control to the garbage collector. It seems that this library is data-structure centric with regard to how memory can be manually managed and to date there are only a few data structures implemented, is that a safe assessment?

@joe-at-startupmedia Hi, glad that you found this interesting!

I'm planning to continue working on this in the future, lots of data structures can be added (most importantly maps) which I need to figure out how to implement efficiently.

I have a pull request already with a hashmap implementation, I need to test it more and make sure it's performant enough.

Also with the release of go 1.23, I can add iterators to all the data structures.

I just have been busy with life :) work and college, and any contributions are welcome!

[joetifa2003]

BIG UPDATES! #6 changes:

• A new allocator interface which is being passed to every data structure to allow for more flexibility and you can compose multiple allocator together, and using this implemented a batchallocator (better name needed) to reduce CGO calls, it takes another allocator (currently we only have C allocator) and allocates big chunks and slices the memory up, so we reduce calls to CGO and the OS.
• Added hashmaps, which works the same as the go hashmap, you don't need to define any hashing function, it just works!
• Added a new data structure MinHeap, which I use to implement batchallocator
• Update to go 1.23 and adding iterators for datastructures

Things needed:

• MORE MORE MORE tests
• Update the readme (lots of changes happened)
• Include more examples in the docs to make things clear

Benchmarks results for hashmaps, manual hashmaps uses 0 allocations

HashmapGo-4           169.0µ ± ∞ ¹
HashmapCAlloc-4       413.5µ ± ∞ ¹
HashmapBatchAlloc-4   235.4µ ± ∞ ¹

[joetifa2003]

@cyertai @joe-at-startupmedia

Hashmaps is merged now and available!