rapidjson icon indicating copy to clipboard operation
rapidjson copied to clipboard

ParseStream "eating" bytes after json body

Open diehard2 opened this issue 1 year ago • 0 comments

I recently updated from an old version of RapidJSON and I'm seeing some unexpected behavior. I've attached the minimal reproducer below. ParseStream is consuming some bytes of the stream that I don't want it to consume. Since this is a non-seekable stream, I can't move the stream back a byte or two.

Minimal reproducer

For this payload

[{"nodes":[{"node-path":"foo","content":1}]}]
--test

and this code, which reads a json header while ideally not advancing the buffer

#include <rapidjson/document.h>
#include <rapidjson/istreamwrapper.h>
#include <rapidjson/reader.h>
#include <rapidjson/stringbuffer.h>
#include <rapidjson/writer.h>

int main()
{
  std::ifstream file("mini.txt");
  rapidjson::IStreamWrapper isw(file);
  rapidjson::Document doc;
  doc.ParseStream<rapidjson::kParseStopWhenDoneFlag>(isw);

  // Create a JSON string buffer
  rapidjson::StringBuffer jsonBuffer;
  rapidjson::Writer<rapidjson::StringBuffer> writer(jsonBuffer);
  doc.Accept(writer);

  // Print the JSON string
  std::cout << jsonBuffer.GetString() << std::endl;

  print_istream(file);
  std::cout << "\n";

  return 0;
}

I see different behavior after June 2020

Before June 2020 (expected)

[{"nodes":[{"node-path":"foobar","content":1}]}]

--test

After June 2020

[{"nodes":[{"node-path":"foobar","content":1}]}]
test

interestingly, the number consumed appears dependent on the size of the json

pre 2020

[{"nodes":[{"node-path":"foo","content":1}]}]

--test

post 2020

[{"nodes":[{"node-path":"foo","content":1}]}]
-test

diehard2 avatar Jul 26 '24 14:07 diehard2