(mx>kx) failed for hidden mx: regrid_smth:mx=0

[sakuracq]

I find a error when i used render_view_3g.py dfitpack.error: (mx>kx) failed for hidden mx: regrid_smth:mx=0 i find in it happened in render.py line 1461,when _x <win_size，the cropping of rgb_value gets result like (0,171,3).

[freerafiki]

Ok, checking the code I see what you mean. This happens because the window size is greater that the coordinates of the sampling point.

Conceptual explanation: The rendering process works by selecting a patch of points in each micro-image (there is some literature you can check on the creation of images from microlenses, a good one is reducing plenoptic artifact from Georgiev (2010 I believe), where he explain the importance of the selection of the patch size. In this case, (in our script) we select a patch based on the disparity value (if the object in the image is close or far from the camera, it results in different resolution, so the patch size must change. We use a sampling pattern which means we use a grid of points where we select/extract/interpolate the value to allow a finer choice (if we select only pixel, we are constrained to using only integer, which results in artifacts)

If you remember, there are different configurations (galilean/keplerian focused plenoptic cameras, check Christoper Hahne's work for example for deeper explanation) so it may be that your setup is slightly different than mine and you get the patch size wrong.

Checking what happens in the code Let me explain to clarify what we are doing there:

• we take pixel coordinates of the microlens
• we get the integer values of x and y to extract the intensity and disparity
• then we create a sampling pattern based on the sample per lens which is based on the disparity value in that microlens and we set the window size based on this sampling pattern.
• we interpolate the values for each channel if we did not have them yet.

Now that the we know conceptually and in the code what we are doing, my guess would be:

the disparity estimation is in a different scale (this would be easy to fix, as you can tune the parameter changing the proportion from the disparity value to the sampling pattern. If you change the 2.5 (or dmax) and you set it relative to the diameter of your image (or to whatever number you empirically find correct), this may solve your problem. If this (patch_size_for_sampling) change, the sampling pattern changes and therefore the win_size changes (if win_size is smaller than _x and _y, the error will not be raised)

I am sorry about this (a bit clunky), I think this was adapted for the R29 camera (as you see it's the default) and worked also for the R42, but I did not use too much other models, what model are you using? If you manage to find the correct parameter (or a better way to estimate/find this), I would be super happy to merge the solution in the toolbox!