Unable to reproduce the metrics in the paper

[MrFisher97]

Since there is no code to evaluate the effect of reconstruction in this project, we refer to the implementation of the evaluation function of WarranWeng/ET-Net. However, we find that the metrics (the model is downloaded from there and the setting complies with the value given in the project) are different from the metrics published in the paper. For example, on the HQF data-set, we test E2VID model, the MSE value is only 0.0930 and the LPIPs is 0.5390, which are quite different from the 0.0611 and 0.4740 in the paper.

So, would you provide the test code or give a more detailed test configuration?

[midofalasol]

Hi, @wanzengy !

Do you use a fixed number of events or take the number of events per frame interval when generating the voxel grid?

[MrFisher97]

Hi, @wanzengy !

Do you use a fixed number of events or take the number of events per frame interval when generating the voxel grid?

I fix the number of events when training, while testing, I take the events between frame interval to generate the voxel grid.

[ercanburak]

For anyone seeking quantitative evaluation, you can utilize EVREAL, our library designed to evaluate and analyze PyTorch-based event-based video reconstruction methods (including SSL-E2VID).