easyjson icon indicating copy to clipboard operation
easyjson copied to clipboard

Optimization allocations during Unmarshal

Open prateek opened this issue 5 years ago • 1 comments

Hey folks, thanks for a pretty bad ass library! I'm seeing a 3-3.3x gain when decoding fairly large JSON payloads (40-50MB) as compared stdlib/json-iterator.

Few things in the generated code that struck me as odd are the allocation sizes for slices/maps:

  • Slice is set to size 1 initially, values are appended using append
  • Map does not have a default size specified

Instead of this, I configured a static value for initial sizes for both slice/map, and changed the slice growth to always 2x the slice when len == cap (go's append will only 2x until 1024 and then it grows linearly. Each of the two sped things up by about 7% in my benchmarking.

I suggest the following changes to the library:

  1. Allow users to specify initial size for containers per key. Something like –
type Foo struct{
  bar []int `json:"bar","easyjson_initial_size:10"`
}
  1. Similarly allow users to either optionally specify (or always), do the 2x slice growth in the generated code.

prateek avatar Apr 27 '20 07:04 prateek

Hello. I am very sorry that our library caused you such a reaction. Your suggestions are very interesting and we will try to implement them. Thank.

GoWebProd avatar May 10 '20 09:05 GoWebProd