JSMN Iterator

[jdomeij]

Hello, after having re-written JSMN state parser multiple times in C code I got frustrated and started to look at something that could simplify the state tracking needed for iterating over JSMN tokens. This resulted in that I implemented iterator functionality, the iterator will keep track of all states needed and the user gets pointers to JSMN tokens representing identifier (if object) and value.

The code can iterate JSON with nested Arrays/Objects in single pass if hint value from inner iterator telling where next item begin is used.

I have added sample application that can pretty print JSON object (two different pretty print functions) and some simple unit tests.

Example

void test(const char *psz, jsmntok_t *jsmn_tokens, unsigned int jsmn_len) {
  jsmn_iterator_t   iterator;
  jsmntok_t        *jsmn_identifier;
  jsmntok_t        *jsmn_value;
  unsigned int      iterator_hint = 0;

  if (jsmn_iterator_init(&iterator, jsmn_tokens, jsmn_len, 0) < 0)
    return;

  while(jsmn_iterator_next(&iterator, &jsmn_identifier, &jsmn_value, iterator_hint) > 0) {
    /* Get values from jsmn_identifier (if Object) and jsmn_value */
  }
}

[aakropotkin]

Could you clarify exactly how next_value_index is used, and does it relate to the return value of jsmn_iterator_next?

I've loved using this extension, but I'm still not 100% clear on what that parameter is used for. The spot where I'm hitting issues is when I am trying to create a new iterator on a sub-object/array. I can't determine if the index relates to the collection of "all tokens", or if it is indexing withing the target structure.

[jdomeij]

Hmmmm this was a while ago, glad you liked this extension. After you enter sub-object/array and iterate over it and jsmn_iterator_next return 0 (iterated over all items) you can take the value from jsmn_iterator_position and pass it to the parent iterator. This will allow the parent iterator to skip over the sub-object/array without needing to linear search for the next object

Take a look at https://github.com/jdomeij/jsmn_iterator/blob/master/example/jsonprint.c#L176 This traverses an JSON hierarchy printing information using an stack of iterators, row 194 we take the current iterator position pop the stack and pass it as an hint when calling jsmn_iterator_next using the parent iterator

[aakropotkin]

Hmmmm this was a while ago, glad you liked this extension. After you enter sub-object/array and iterate over it and jsmn_iterator_next return 0 (iterated over all items) you can take the value from jsmn_iterator_position and pass it to the parent iterator. This will allow the parent iterator to skip over the sub-object/array without needing to linear search for the next object

Take a look at https://github.com/jdomeij/jsmn_iterator/blob/master/example/jsonprint.c#L176 This traverses an JSON hierarchy printing information using an stack of iterators, row 194 we take the current iterator position pop the stack and pass it as an hint when calling jsmn_iterator_next using the parent iterator

Thank you so much, this was a huge help.