go-hbase
go-hbase copied to clipboard
General usage questions
I started with go-hbase from lazyshot, ran into problems. He eventually referred me to your version. I got yours working, loading a file with 129k rows, 16 columns into HBase. I expected somewhat slower performance running in HBase standalone mode and testing on a Linux VM on VMWare, but was still surprised at how much slower it was -- 250 updates/second, With an eventual target of at least 500k/second in fully distributed mode with proper configuration, I was looking for at least indicative standalone rates say in the 10k range or at the very least 1k-10k.
-
Would you rate this package as "ready for primetime," meaning I could use it with sufficient testing in a live mission critical application?
-
the lazyshot version had methods for "Gets" and "Puts" which I understood were to support batching of updates and requests, your version does not have these. Is there a reason they were removed other than just the scope of your effort and available time? Is there any reason I couldn't work to add them back in if I needed them for the performance boost?
Thanks.
Thank you for your feedback!
-
I use this package for https://github.com/pingcap/go-themis which is a ACID transaction library for HBase, I benched this library, performance is OK, so may I take a look at your test code and test environment, so that I can find out the cause. BTW since go-themis have been discard, so I'm afraid it's not been tested in production environment.
-
According to the use case in go-themis, I don't need to port lazyshot's Gets function, and I think his implementation of Gets/Puts is buggy, I create my implementation in go-themis using coprocessor.
I remember I did a simple benchmark, something like https://github.com/pingcap/go-themis/blob/master/benchmark/main.go ,but with raw go-hbase API, and I got ~30k qps in read. How many goroutine did you use?