many warning when change the timestamp_resolution and --auto_timestamp_resolution set to true

[gwgknudayanga]

Hi,

For the given sample video (tennis.mov) I have use the following command to generate events. Here i set the timestamp_resolution to 5ms instead and set the auto_timestamp_resolution to true. (in the example command given in the read me time_stamp_resolution was 3ms and auto_timestamp_resolution was set to False. ). In that case there are many warning as shown. So, for our own video how do we decide the exact timestamp resolution to generate events without these kind of warnings ?(because 3ms timestamp these warning did not come). Thank you. Command: python v2e.py -i inputs/tennis.mov --overwrite --slomo_model inputs/SuperSloMo39.ckpt --timestamp_resolution=.005 --auto_timestamp_resolution=True --dvs_exposure duration 0.005 --output_folder=output/tennis --overwrite --pos_thres=.15 --neg_thres=.15 --sigma_thres=0.03 --dvs_aedat2 tennis.aedat --output_width=346 --output_height=260 --stop_time=5 --cutoff_hz=15

Warnings:

WARNING:v2ecore.emulator:no events generated for frame, generating any sampled temporal noise for this frame with 1 iteration WARNING:v2ecore.emulator_utils:IIR lowpass filter update has large maximum update eps=0.39 from delta_time/tau=0.00417/0.0106 WARNING:v2ecore.emulator_utils:IIR lowpass filter update has large maximum update eps=0.39 from delta_time/tau=0.00417/0.0106

[tobidelbruck]

It means your timestamp resolution is too coarse (your sample rate is too low) to support the IIR lowpass filter cutoff you chose. You need to either decrease the cutoff frequency or increase the upsampling resolution (i.e., decrease the timestamp resolution).

This restriction is actually quite severe. For instance, if you want a reasonable (but still quite slow) 100Hz photoreceptor cutoff frequency, then you will need to use a timestamp resolution that allows accurate IIR lowpass updates. So if

f3dB=100Hz

then the tau would be

tau=1/(f3dB2Pi)=1/(1002Pi)=1.6ms.

Therefore the sample interval should be at least 3 times higher for accurate IIR lowpass filter updates, which use eps=dt/tau. We need to make sure eps is much smaller than 1. Therefore, in this case, we could use for eps=0.3 a value of dt=1.6ms/3=500us.

Unfortunately, unless we are missing something about how IIR filters work, this expensive upsampling is not avoidable for accurate photoreceptor filtering modeling.

PS if you don't care about modeling DVS photoreceptor lowpass filtering, then just disable it. But then you may get unrealistically-fast DVS pixel responses that do not model the motion blur that DVS pixels really do have (despite the many false claims in the computer vision literature).

We recommend reading at least the pixel behavior section of the v2e paper to better understand these real effects.

Hope this helps.

Tobi