fprime
fprime copied to clipboard
BufferLogger CRC calculations are inefficient
F´ Version | v3.1.1 |
Affected Component | BufferLogger,Os/ValidateFileCommon.cpp |
Feature Description
Update BufferLogger to calculate CRC32 as the file is being written to disk.
Rationale
Currently, when BufferLogger closes a file it goes on to create a validation file. On Linux, this re-opens the file and reads it 256 bytes at a time. For larger files, say 10 MB, this results in around 40,000 syscalls when closing a file. This can impact the performance of other threads running on the same core, regardless of the priority.
This 256 byte size is hidden in a header file and is not configurable without forking fprime. https://github.com/nasa/fprime/blob/91c578ee37c01ba92e556dbca0287d82f9f18b95/Os/ValidateFile.hpp#L16
Increasing this value would partially alleviate the issue but I want to propose another solution. Calculate the CRC at the same time as the buffers are being written to disk. This would not require any read syscalls or any additional processing when closing a file. However, it would mean that the ValidateFile
abstraction could not be used within BufferLogger