Recording and using advanced mode at the same time

[oscarthorn]

---

Required Info
Camera Model	D455
Platform	Jetson
SDK Version	2.50.0
Language	C++

Issue Description

I have the same issue as #10184 and #6739.

Running: dev.as<rs400::advanced_mode>(); returns a null pointer if you have enabled recording with cfg_.enable_record_to_file(record_bag_path.string());

Is there a fix for this? We have a lot of d455's and need to be able to record bag files and we need to be able to be able to use our custom settings.

[MartyG-RealSense]

Hi @oscarthorn If one of the existing Visual Presets (jsons) provided by the RealSense SDK is close to your custom json then you could try applying that preset instead of loading your custom json as a string by using the C++ code at the link below.

https://www.intel.com/content/www/us/en/support/articles/000028416/emerging-technologies/intel-realsense-technology.html

Here is a list of the parameter names of the presets so that you can amend the code in the above link to use the appropriate preset.

RS2_RS400_VISUAL_PRESET_CUSTOM RS2_RS400_VISUAL_PRESET_DEFAULT RS2_RS400_VISUAL_PRESET_HAND RS2_RS400_VISUAL_PRESET_HIGH_ACCURACY RS2_RS400_VISUAL_PRESET_HIGH_DENSITY RS2_RS400_VISUAL_PRESET_MEDIUM_DENSITY RS2_RS400_VISUAL_PRESET_REMOVE_IR_PATTERN RS2_RS400_VISUAL_PRESET_COUNT

[oscarthorn]

Hi @MartyG-RealSense, no none of the presets are suitable

[MartyG-RealSense]

In addition to the C++ json cases that you linked to at the beginning, there is also another example of using dev.asrs400::advanced_mode(); at https://github.com/IntelRealSense/librealsense/issues/3277#issuecomment-464750895

A less efficient alternative to json may be to investigate whether the settings that you are using in the json can be applied with C++ script code instead.

If your preference is to use the custom json file then you could investigate whether using threads and putting the record function on a separate thread from the json loading code might make a difference. There is a C++ threads example at https://github.com/IntelRealSense/librealsense/issues/6865 though I would personally recommend against using threading unless there is no other option as it makes programs more complex.

[oscarthorn]

The problem is not with the json loading the problem is that dev.as<rs400::advanced_mode>(); returns a nullptr. Could you clarify if #3277 is relevant to that?

I have no problem setting them manually, are the settings available without accessing advanced mode with dev.as<rs400::advanced_mode>();?

There is no recording function to put on a separate thread, please clarify if there is something I could change. This problem occurs regardless of if dev.as<rs400::advanced_mode>(); is called before or after pipe.start().

[MartyG-RealSense]

It would depend on which settings you want to change. If they are Advanced Mode settings then there is little available documentation for controlling those options, unfortunately. If it is non-Advanced Mode options such as exposure then it is straightforward to change those with C++ code.

There is not enough information available about this particular nullptr issue (using json + record in the same script) for me to offer advice about the relevancy of https://github.com/IntelRealSense/librealsense/issues/3277 unfortunately. I do apologize.

https://github.com/IntelRealSense/librealsense/issues/3802 is a C++ discussion about implementing threading for recording a bag with cfg.enable_record_to_file()

[oscarthorn]

Is there a way to get a hold of the developer team for librealsense to get the bug resolved? So far I have traced it to the serializable_device constructor setting the device to null.

[MartyG-RealSense]

I will discuss your issue with my Intel RealSense colleagues. Thanks very much for your patience.

[oscarthorn]

My findings so far dev.as<rs400::advanced_mode>(); calls as() in rs_device.hpp which tries to construct the correct type advanced_mode. The advanced_mode constructor calls the serializable_device constructor which in turn sets _dev (the member variable that holds the actual rs2_device struct) to nullptr since rs2_is_device_extendable_to(_dev.get(), RS2_EXTENSION_SERIALIZABLE, &e) returns 0.

I will try to determine why that is, my assumption is that there is some interface/inheritance missing somewhere which causes the VALIDATE_INTERFACE_NO_THROW, macro to fail but it would be good to get input from someone involved in this code.

[oscarthorn]

Okey, so the main issue seems to be that in VALIDATE_INTERFACE_NO_THROW_ADV the statement ext->extend_to(TypeToExtension<T>::value, (void**)&p) evaluates to false since extend_to in record_device does not implement RS2_EXTENSION_SERIALIZABLE as an extension it can extend to. Not sure if this is an oversight or on purpose.

[oscarthorn]

Just adding the serializable extension and making it extend to advanced seems to resolve the issue. Though I'm not sure if this is a valid solution. I tried extending it to serializable but that interface does not have create_snapshot which is expected by write_device_extension_changes. So I would love clarification from a developer if this is a valid solution that you can implement (or I can open a PR).

case RS2_EXTENSION_SERIALIZABLE:
case RS2_EXTENSION_ADVANCED_MODE   : return extend_to_aux<RS2_EXTENSION_ADVANCED_MODE  >(m_device, ext);

[oscarthorn]

Don't have time to investigate that also but dev.first<rs2::color_sensor>().as<rs2::roi_sensor>() also seems to return nullptr if you have enabled recording.

Used to set region of interest for exposure with .set_region_of_interest(roi);

[MartyG-RealSense]

Hi @oscarthorn My colleagues were able to replicate the issue and created an internal Intel report so that it can be investigated as a possible bug. They also said that you are more than welcome to submit a PR.

I have added an Enhancement label to this case to signify that it should be kept open.

[oscarthorn]

Hi @MartyG-RealSense Thanks, sounds good!

For now I found a workaround after realizing that realsense-viewer does not seem to have the same problem. After looking at the code the it seems to use the recorder class directly and not setting the record setting on the pipeline. Just instantiating with recorder = std::make_shared<recorder>(path, dev); seems to do the trick without polluting the device interface. You can then use the usual functions and if you want to stop recording you can just run recorder.reset(); and the restart again (possibly to a different file) with recorder.reset(new rs2::recorder(path, dev)); Perhaps this method of recording could be added to the examples or documentation?

Thanks for the help!

[MartyG-RealSense]

You are very welcome. Thanks so much for sharing the details of your workaround!