pysimdjson icon indicating copy to clipboard operation
pysimdjson copied to clipboard

as_buffer() support for Object values.

Open TkTech opened this issue 4 years ago • 3 comments

We now support as_buffer() #59 which drastically improves performance when loading arrays from JSON into numpy.

We should also support as_buffer() for Objects, which would retrieve the values as an array. This is to support calling it like this:

https://github.com/riddell-stan/pystan-next/blob/1e027a8bded88d51c6b957a841b859938c0ac86d/stan/fit.py#L93

For @riddell-stan's use case, this should use less memory and be roughly 4x faster (since we also void the iterator creation for values())

TkTech avatar Nov 22 '20 14:11 TkTech

Additionally, we should make sure this works for round-tripping into a Panda's dataframe.

TkTech avatar Mar 20 '21 02:03 TkTech

Fixing this would be wonderful. Glad it's on your radar.

FYI: we renamed the repository. Here's an updated link: https://github.com/stan-dev/pystan/blob/1ffd41a5827c4d54c31205e350f0e698bcb2d87f/stan/fit.py#L93

riddell-stan avatar Sep 29 '21 11:09 riddell-stan