ecal icon indicating copy to clipboard operation
ecal copied to clipboard

Extend imeasurement.h API

Open FlorianReimold opened this issue 3 years ago • 1 comments

>> This issue was requested internally and copied to github.com<<

Is your feature request related to a problem? Please describe. Yes. We have a tool to export measurement data from an input eCAL recording. Currently we are iterating over all channel measurements. In our case, reading each measurement can take a while due to the large data in it.

Since we occasionally know our desired eCAL timestamps beforehand, it would be very fast to access this data directly.

Describe the solution you'd like It would be great if you could provide getter functions for the EntryInfoSet in imeasurement.h: For the BinaryChannel class:

eh5::EntryInfoSet getEntryInfoSet() { return entry_infos; }

For the IChannel class:

eh5::EntryInfoSet getEntryInfoSet() { return binary_channel.getEntryInfoSet(); }

We use this in our export tool like this

        /*!< special mode if list of timestamps is specified AND the eCAL timestamp is used --> load only the desired data instead of iterating over whole recording */
        if (opModeExportSingleData){
            // locate relevant meas entries and store their SEntryInfo struct.
            std::vector<eCAL::eh5::SEntryInfo> relevantMeasEntries;
            for (const auto& entryInfo : channel.getEntryInfoSet()) {
                if (std::find(std::begin(timestampSelection), std::end(timestampSelection), entryInfo.SndTimestamp) != std::end(timestampSelection)) {
                    relevantMeasEntries.push_back(entryInfo);
                }
            }
            std::cout << "exporting only the " << relevantMeasEntries.size() << " relevant entries for " << channel.name() << "... " << std::endl;
            for (const auto& entryInfo : relevantMeasEntries) {
                const auto& meas = channel[entryInfo];
                processCore(meas.message, meas.send_timestamp);
            }
        }

Instead of the slow version

        // -----------------------------------------
        else {  // iterate over all messages
            std::cout << "Iterating over all messages..." << std::endl;
            for (const auto& raw_image : channel){...}
...

FlorianReimold avatar Feb 15 '22 08:02 FlorianReimold

The measurement API is a high level API that abstracts over the eCAL hdf5 API and therefore hides internal implementation detail. Therefore I don't feel very good about exposing eCAL::eh5::SEntryInfo to the high level API, as it requires knowledge about the internal structure (mainly the ID).

I haven't thought this trough, but maybe it makes sense to be able to create iterators from timestamp, und this create views / filters on the actual channel, or even the whole measurement.

channel;
auto first_part_of_measurement = ChannelTimeFilter(channel, Channel::Start, Channel::End - Channel::Start / 2)

for (const auto& raw_image : first_part_of_measurement)
{
  ...
} 

I know that C++17 or 20 makes use of these kind of patterns more and more, e.g. see string_view. I would have something similar in mind.

Kerstin-Keller avatar Feb 15 '22 09:02 Kerstin-Keller