jsonparser
jsonparser copied to clipboard
`stackbuf` in `ObjectEach` is actually allocated on the heap
I found this when profiling some code that uses jsonparser, and 15% of the total execution time was spent in allocating memory for this buffer. The Go compiler's analysis looks correct, since it can't know what the callback does with key
so it isn't safe to allocate stackbuf
on the stack.
My solution was to just move the declaration of stackbuf
inside the if keyEscaped
clause, so that it's only allocated when needed. This would probably hurt performance if more than one key per object needed escaping, but I wasn't worried about that for my particular case since I have full control over the keys and know they will never need escaping.
You can see my commit at https://github.com/Carevoyance/jsonparser/commit/224355b6dbc786c3e479107fa2dd52cfd39bd3c8 but I didn't submit this as a PR in case you prefer a different or more elegant solution. I'm happy to submit it as a PR if you'd like.
This issue aside, I obtained huge speedups in my JSON parsing code when switching to jsonparser, so thank you very much for your work!
I ran into this same "problem". In my case, it was with EachKey
. This is the diff that reduced the allocs from 2 to 1, since, indeed, this allocation is only rarely actually used (in my case):
diff --git a/vendor/github.com/buger/jsonparser/parser.go b/vendor/github.com/buger/jsonparser/parser.go
index 9e8b788..cfdd31b 100644
--- a/vendor/github.com/buger/jsonparser/parser.go
+++ b/vendor/github.com/buger/jsonparser/parser.go
@@ -355,7 +355,6 @@ func EachKey(data []byte, cb func(int, []byte, ValueType, error), paths ...[]str
}
}
- var stackbuf [unescapeStackBufSize]byte // stack-allocated array for allocation-free unescaping of small strings
pathsBuf := make([]string, maxPath)
for i < ln {
@@ -389,9 +388,13 @@ func EachKey(data []byte, cb func(int, []byte, ValueType, error), paths ...[]str
var keyUnesc []byte
if !keyEscaped {
keyUnesc = key
- } else if ku, err := Unescape(key, stackbuf[:]); err != nil {
- return -1
} else {
+ var stackbuf [unescapeStackBufSize]byte // stack-allocated array for allocation-free unescaping of small strings
+ ku, err := Unescape(key, stackbuf[:])
+ if err != nil {
+ return -1
+ }
+
keyUnesc = ku
}