Sequential message processing

[senier]

Applications that receive messages from a stream have to be able to determine whether a message is complete or if more input bytes are required to decide on its validity. With the Incomplete and Incomplete_Message functions RecordFlux already provides a way to detect those situations.

There exists a limitation that prevents efficient stream-like processing, though. When input data has been read into a buffer, the pointer to this buffer needs to be moved into the message context using the Initialize procedure. Only then the message can be validated using the Verify procedures.

If the message (or a particular field) is found to be incomplete during parsing, new data needs to be added to the buffer. To do this, however, the buffer needs to be moved out of the context using the Take_Buffer procedure. While data can then be appended to the buffer, the previous message validation is lost and must be redone when initializing the context again using the updated buffer pointer. While stream-like processing can be emulated this way, it may be prohibitively inefficient.

We would like to enable updates to the input buffer while parsing (without invalidating the parser state). The idea is to generate a generic procedure, e.g. Append_Data which is instantiated with a procedure reading from an input stream:

generic
   with procedure Read (Data : out Types.Bytes; Last : out Types.Index)
procedure Append_Data (Ctx : in out Context);

Whenever an Incomplete condition is encountered, the user calls Append_Data on the current context. Internally, it will call the provided Read procedure, potentially append more bytes to the buffer. The new end of the buffer is signaled by the Last out parameter.

Its important that bytes that have already been validated cannot be overwritten. Generated code ensures this by only passing the slice of the message buffer to the user-provided Read procedure which is beyond the previous end of the input data. Effectively, data can only be appended without invalidating the context.