java

java copied to clipboard

I'm writing an application where JSON parsing needs to be really fast. Here's the piece of code I've written :

        String appId = null;
        String itemId = null;
        String weapon = null;
        String price = null;

        long start = System.nanoTime();
        
        JsonIterator iter = JsonIterator.parse("{\"app_id\":\"730\",\"context_id\":\"2\",\"item_id\":\"13960643001\",\"class_id\":\"472839281\",\"instance_id\":\"188530139\",\"image\":\"https://steamcommunity-a.akamaihd.net/economy/image/-9a81dlWLwJ2UUGcVs_nsVtzdOEdtWwKGZZLQHTxDZ7I56KU0Zwwo4NUX4oFJZEHLbXH5ApeO4YmlhxYQknCRvCo04DEVlxkKgpovbSsLQJfx_LLZTRB7dCJlZG0kfjmML7VqWZU7Mxkh6eW94mt31bjqkVsZDulcdTAIQM_Z1HSrFi4wOq7gpK-u86dzCRis3Mg-z-DyJKr1fVx/257fx257f\",\"market_hash_name\":\"★ Huntsman Knife | Stained (Field-Tested)\",\"price\":\"60.56\",\"discount\":\"23\",\"event_type\":\"listed\",\"broadcasted_at\":1519649919.58952}");

        for (String field = iter.readObject(); field != null; field = iter.readObject()) {
            switch (field) {
                case "app_id":
                    appId = iter.readString();
                    continue;
                case "item_id":
                    itemId = iter.readString();
                    continue;
                case "market_hash_name":
                    weapon = iter.readString();
                    continue;
                case "price":
                    price = iter.readString();
                    continue;
                default:
                    iter.skip();
            }
        }

I've done the same thing with a library as org.json which is supposed to be much slower but actually executes faster in this case. What am I doing wrong ?

if you want performance, use object binding api with dynamic codegen or static codegen. Iterator is only useful when dealing with large streaming input.

If you want to use streaming 'by hand':

1. Reuse JsonIterator instance: take it by calling JsonIteratorPool.borrowJsonIterator(), then put back with JsonIteratorPool.returnJsonIterator, set JSON by calling reset.
2. Don't allocate object keys, use IterImpl.readObjectFieldAsSlice(iter) instead. If your keys' hashes do not collide, you can switch on key's hashCode() then:

Slice key = iter.readObjectFieldAsSlice();
switch (key.hashCode) {
    case HASH_APP_ID:
        // some code here
}

where HASH_APP_ID is a compile-time constant equal to "app_id".hashCode(). Note: Slice returned by this method will be changed in next iterations, don't save it anywhere. 3) When parsing objects where many strings are equal, you may want to create your own string pool, e. g. HashMap<Slice, String>, and allocate string only if there's no such string in cache. Don't forget to copy slices acquired by readStringAsSlice before putting into map!

@taowen Is there any example out there ? Could you provide more details, please ? I don't know where to start from, especially for the codegen part. Thank you

@Miha-x64 I'm stuck at this instructions ( Slice key = iter.readObjectFieldAsSlice(); ) . I've borrowed the iterator and set the json string as you suggested but I can't get the key as in the example above. The method can't be found

just use the bind api

JsonIterator.deserialize("[1,2,3]", int[].class); 

That's what I've done, however it takes between 30 and 50 ms to read those 4 fields from the above json. That's waaay too much. Am I missing something ?

how did you do the benchmark? Do you use JMH?

Well I've just used System.currentTimeMillis(), I know it's not the best way to measure things like this but it's slower than org.json anyway, which executes in less than 1 ms.

use JMH and SetDecodeMode, then try again. If it is still slower. Give me your benchmark code. I will fix it.