Json length problem

[kwis2]

I test Basic_WebSkeletonApp example and there is problem with JsonObjectStream. With about 400 bytes of data there is stream cut - it looks like key/value were NULL/NULL on the end of stream and js script hangs.

[mikee47]

Can you provide a little more detail so we can reproduce the issue?

[kwis2]

I have modified onAjaxGetState with

 for(uint16_t i=0;i<counter;i++)
 	{
 		json["counter"+String(i)] = i;
 	}

and it worked fine until 37 and next loop it was null null terminated and js hangs and with opera preview there was state:

 counter0: 0
 counter1: 1
 ......
 counter34: 34
 counter35: 35
 counter36: 36
 counter37: 37
 counter38: 38
 null: null

[mikee47]

JsonObjectStream uses an internal DynamicJsonDocument with a default capacity of 1024 bytes. The problem you describe occurs when the document is full and cannot allocate space for the final key/value pair.

You can increase the document size like this:

JsonObjectStream* stream = new JsonObjectStream(2048);

But if your requirements are too large then you'll need a more involved approach to seralising the data. There's an example of how to do this in the RingTone library, in RtttlJsonListStream.cpp.

[kwis2]

I think I have to clear where the problem is. Finite capacity of JsonObjectStream is not a problem. Problem is that unexpectedly there is null: null which kill js and how could we avoid such situations. On the other hand it could be good idea to know if data we add to stream reach the www.... I mean with modified Basic_WebSkeletonApp example we expect for instance counter40: 40 and there is only counter38: 38 availiable. JsonObjectStream(2048) changed that only counter78: 78 is the last

[mikee47]

@kwis2 You may find this helpful https://arduinojson.org/v6/api/jsonobject/subscript/

[kmihaylov]

I had similar problem (in my case the values were "0" (integer).

Long story short, I had to increase the size of the object defined by DynamicJsonDocument despite my string's length was less than the jsonDocument object.

Just increased it twice or so and the "0" values became what was expected.

Now I see that I even use DeserializationError without checking the result. Probably there is similar type for serialization.

[kmihaylov]

@kwis2, may you provide example of the problem? I've prepared some simple skeleton to use.

https://github.com/kmihaylov/Sming_ArduinoJson6_DebugSkeleton

[kwis2]

@kmihaylov Sorry I missed your post. in your github I commit example. (I am not sure if it was succesful). Json failed after about 430 bytes

#include <SmingCore.h>
#include <Debug.h>
#include "ArduinoJson.h"

#define MQTT_CONF_FILENAME "test"

const uint16_t ConfigJsonBufferSize = 2048;

void init()
{
	Serial.begin(SERIAL_BAUD_RATE, SERIAL_8N1,
				 SERIAL_FULL);		// 115200 by default, GPIO1,GPIO3, see Serial.swap(), HardwareSerial
	Serial.systemDebugOutput(true); // Debug output to serial

	spiffs_mount_manual(0x0c0000, SPIFF_SIZE); //ESP8266-01 module
	int check = SPIFFS_check(&_filesystemStorageHandle);
	debugf("FS check res: %d", check);

	Serial.commandProcessing(true); //also executes commandHandler.registerSystemCommands(); ?

	Debug.setDebug(Serial);
	Debug.initCommand();
	Debug.start();
	Debug.printf("This is the debug output\r\n");

	StaticJsonDocument<ConfigJsonBufferSize> root;

	root["User"] = "abcdefghijklmnopqrtsuvwxyz";
	root["Password"] = "abcdefghijklmnopqrtsuvwxyz";
	root["DeviceTopic"] = "abcdefghijklmnopqrtsuvwxyz";

	DeserializationError error;

	int element_cntr = 0;

	do {
		element_cntr++;
		fileDelete(MQTT_CONF_FILENAME);
		for(int i = 0; i < element_cntr; i++) {
			root[ String(i)] = String(i);
		}
		String jsonString;
		serializeJson(root, jsonString);

		int file_w_bytes = fileSetContent(MQTT_CONF_FILENAME, jsonString);

		debugf("bytes written %d ", file_w_bytes);

		int file_size = fileGetSize(MQTT_CONF_FILENAME);

		debugf("file size %d", file_size);

		String jsonString2 = fileGetContent(MQTT_CONF_FILENAME);

		debugf("string size %d", jsonString2.length());

		StaticJsonDocument<ConfigJsonBufferSize> root2;
		error = deserializeJson(root2, jsonString2);
		debugf("JSON deserial err: %s", error.c_str());

		Serial.printf("%s\r\n", (const char*)root2["User"]);
		Serial.printf("%s\r\n", (const char*)root2["Password"]);
		Serial.printf("%s\r\n", (const char*)root2["DeviceTopic"]);

		for(int i = 0; i < element_cntr; i++) {
			Serial.printf("%d -> %s\r\n", i, (const char*)root2[String(i)]);
		}

	} while(error == 0);
}