jsonparser icon indicating copy to clipboard operation
jsonparser copied to clipboard

`stackbuf` in `ObjectEach` is actually allocated on the heap

Open dimfeld opened this issue 7 years ago • 1 comments

I found this when profiling some code that uses jsonparser, and 15% of the total execution time was spent in allocating memory for this buffer. The Go compiler's analysis looks correct, since it can't know what the callback does with key so it isn't safe to allocate stackbuf on the stack.

My solution was to just move the declaration of stackbuf inside the if keyEscaped clause, so that it's only allocated when needed. This would probably hurt performance if more than one key per object needed escaping, but I wasn't worried about that for my particular case since I have full control over the keys and know they will never need escaping.

You can see my commit at https://github.com/Carevoyance/jsonparser/commit/224355b6dbc786c3e479107fa2dd52cfd39bd3c8 but I didn't submit this as a PR in case you prefer a different or more elegant solution. I'm happy to submit it as a PR if you'd like.

This issue aside, I obtained huge speedups in my JSON parsing code when switching to jsonparser, so thank you very much for your work!

dimfeld avatar Jan 08 '18 07:01 dimfeld

I ran into this same "problem". In my case, it was with EachKey. This is the diff that reduced the allocs from 2 to 1, since, indeed, this allocation is only rarely actually used (in my case):

diff --git a/vendor/github.com/buger/jsonparser/parser.go b/vendor/github.com/buger/jsonparser/parser.go
index 9e8b788..cfdd31b 100644
--- a/vendor/github.com/buger/jsonparser/parser.go
+++ b/vendor/github.com/buger/jsonparser/parser.go
@@ -355,7 +355,6 @@ func EachKey(data []byte, cb func(int, []byte, ValueType, error), paths ...[]str
                }
        }
 
-       var stackbuf [unescapeStackBufSize]byte // stack-allocated array for allocation-free unescaping of small strings
        pathsBuf := make([]string, maxPath)
 
        for i < ln {
@@ -389,9 +388,13 @@ func EachKey(data []byte, cb func(int, []byte, ValueType, error), paths ...[]str
                                var keyUnesc []byte
                                if !keyEscaped {
                                        keyUnesc = key
-                               } else if ku, err := Unescape(key, stackbuf[:]); err != nil {
-                                       return -1
                                } else {
+                                       var stackbuf [unescapeStackBufSize]byte // stack-allocated array for allocation-free unescaping of small strings
+                                       ku, err := Unescape(key, stackbuf[:])
+                                       if err != nil {
+                                               return -1
+                                       }
+
                                        keyUnesc = ku
                                }

JeanMertz avatar Jun 17 '18 22:06 JeanMertz