Consume less memory

[Miha-x64]

Reduced memory consumption: — removed unused BitInteger constants in IterImpl; — changed IterImplNumber.intDigits, IterImplNumber.floatDigits type from int[] to byte[]; — changed IterImplString.hexDigits type from int[] to byte[] and shifted it by 48 ('0'); — using byte constants instead of ValueType in JsonIterator, using own byte[] dictionary instead of ValueType[].

[codecov-io]

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@72231fa). Click here to learn what that means. The diff coverage is 67.94%.

@@           Coverage Diff            @@
##             master    #159   +/-   ##
========================================
  Coverage          ?   68.5%           
========================================
  Files             ?     107           
  Lines             ?    7354           
  Branches          ?    1388           
========================================
  Hits              ?    5038           
  Misses            ?    1869           
  Partials          ?     447

Impacted Files	Coverage Δ
...ain/java/com/jsoniter/ReflectionObjectDecoder.java	50.18% <ø> (ø)
...rc/main/java/com/jsoniter/extra/Base64Support.java	78.57% <ø> (ø)
.../main/java/com/jsoniter/output/CodegenImplMap.java	95.08% <ø> (ø)
src/main/java/com/jsoniter/IterImpl.java	65.65% <ø> (ø)
src/main/java/com/jsoniter/IterImplString.java	87.5% <0%> (ø)
src/main/java/com/jsoniter/IterImplNumber.java	67.44% <0%> (ø)
src/main/java/com/jsoniter/any/ArrayLazyAny.java	78.04% <100%> (ø)
src/main/java/com/jsoniter/JsonIteratorPool.java	72.72% <100%> (ø)
src/main/java/com/jsoniter/any/LazyAny.java	45.16% <100%> (ø)
.../main/java/com/jsoniter/output/MapKeyEncoders.java	84.21% <50%> (ø)
... and 4 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 72231fa...f8e8764. Read the comment docs.

[taowen]

what is your usage scenario to concern the memory usage so much?

[Miha-x64]

I can't see any reason to bloat memory. When we can use 256 bytes instead of 1K, why don't do so?

[taowen]

Do you have memory benchmark to prove this change worthwhile? It looks non substantial to me.

[Miha-x64]

public static void main(String[] args) throws Exception {
    JsonIterator ji = new JsonIterator();
    ji.reset("[1, \"str\"]".getBytes());

    IterImpl.nextToken(ji);
    System.out.println(IterImplNumber.readInt(ji));
    IterImpl.nextToken(ji);
    System.out.println(IterImplString.readString(ji));

    System.gc();
    System.gc();
    System.gc();

    System.out.println(Runtime.getRuntime().freeMemory());
    /* -Xmx4M, running from IDEA
     *
     * Java version | 1.8.0_131 | 9         |
     * -------------|-----------|-----------|
     * Original     | 3 230 040 | 2 956 408 |
     * Optimized    | 3 254 040 | 2 979 504 |
     *              |      Free memory      |
     * _____________|_______________________|
     */
}

Output differs a bit between invocations. In general, I've saved about 20 KB. It's not a thing, but i don't see any reason to allocate more, e. g. by using int[] in places where byte[] is appropriate.

[taowen]

the saving is not proportional to the length of input. 20kb constant memory saving is not worth it.

[Miha-x64]

I's a bit philosophical decision: when I can free some memory without affecting performance, I do it. https://www.youtube.com/watch?v=b6zKBZcg5fk

[miere]

@Miha-x64 Totally agreed. Specially if we use jsoniter where we have strict hardware constraints, like AWS Lambda functions.

[plokhotnyuk]

I would say that this PR proposes a trade off which save memory by spending more CPU cycles in runtime.

Also, worst cases in case of malformed input are getting more worse here. As example rethowing of exception can take the same time as a small message parsing. See more info here: https://shipilev.net/blog/2014/exceptional-performance/

[miere]

It seems like a good point. I'd rather consume a bit more memory if we have any sort of memory penalty.

On Thu, Apr 18, 2019 at 5:01 PM Andriy Plokhotnyuk [email protected] wrote:

I would say that this PR proposes a trade off which save memory by spending more CPU cycles in runtime.

Also, worst cases in case of malformed input are getting more worse here. As example rethowing of exception can take the same time as a small message parsing. See more info here: https://shipilev.net/blog/2014/exceptional-performance/

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/json-iterator/java/pull/159#issuecomment-484380931, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD7NUCBI4UNZJVHUCE3KULPRAMDLANCNFSM4ETDY65A .

-- /* Miere L. Teixeira */

[miere]

ops: CPU performance penalty.

On Thu, Apr 18, 2019 at 5:11 PM Miere Teixeira [email protected] wrote:

It seems like a good point. I'd rather consume a bit more memory if we have any sort of memory penalty.

On Thu, Apr 18, 2019 at 5:01 PM Andriy Plokhotnyuk < [email protected]> wrote:

I would say that this PR proposes a trade off which save memory by spending more CPU cycles in runtime.

Also, worst cases in case of malformed input are getting more worse here. As example rethowing of exception can take the same time as a small message parsing. See more info here: https://shipilev.net/blog/2014/exceptional-performance/

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/json-iterator/java/pull/159#issuecomment-484380931, or mute the thread https://github.com/notifications/unsubscribe-auth/AAD7NUCBI4UNZJVHUCE3KULPRAMDLANCNFSM4ETDY65A .

-- /* Miere L. Teixeira */

-- /* Miere L. Teixeira */