Add Custom Image Plane

[david-cattermole]

Feature

The image plane (Maya) with node-based compositing system and multi-layered lens (un)distortion system.

For many aspects of the VFX process, viewing image sequences in 3D is an absolute must. There are limitations of the currently existing tools in Maya, and we need a modern, fast, and capable featured Image Plane.

Feature goals:

• Good playback speeds, with RAM and disk-based caching for pre-computed images.

• Real-time image adjustments (such as exposure, colour, soft-clipping, etc)

• Support for EXR, JPEG, and PNG formats at least is mandatory.

• 8-bit, 16-bit and 32-bit image display and manipulation.

• On-the-fly down-scaling of images (for better playback and resource usage).

• Support for "Data Window" and "Display Window" - so lens distortion can go outside the camera frustum and not be clipped off.

• Node-based system, the ability to create Maya nodes and link them together to perform image operations (such as resizing, cropping, colour changes, transformations and lens distortion).

Technical goals:

• The image operations node-based system should be completely separate from Maya (possibly a different project)

• The image plane must be independent from mmSolver, with the ability to load without mmSolver installed or running.

• The image plane should...

  • be a "viewer" of the final images computed by the nodes.

  • be able to concatenate transform, deformation and colour operations to avoid costly computations.

• The image plane and node-based system should be able to read and write files independent of Maya, performing the exact same operations with the same results.

• Image operations should be multi-threaded, as best as possible.

Software Versions

• mmSolver version: v0.4.0
• Maya version: 2018+
• Operating System (OS): Windows, Linux and MacOS.

[david-cattermole]

This feature will likely be split into a different project, so that users can install it without needing to install all of mmSolver.

Also, the underlying library for the image plane will be "OpenCompGraph": https://github.com/david-cattermole/OpenCompGraph

[david-cattermole]

Update, the actual image plane is being added to https://github.com/david-cattermole/OpenCompGraphMaya and is in active development.

[david-cattermole]

The issue #106 and branch solver_issue106 contains a lens distortion deformer that could be used with OpenCompGraphMaya data structures to provide undistortion of meshes to help visualise lens distortion in Maya.

[Nik1506]

Hi Daivd, I really like the work done for the MM solver plugin. I would like to learn from you and help if possible to develop this tool further more. Since we have moved from python 2 to python 3 can we include opencv package to maya and use its features for camera tracking?

[david-cattermole]

Hello @Nik1506,

I really like the work done for the MM solver plugin. I would like to learn from you and help if possible to develop this tool further more.

Great! I would recommend you look at the existing issues with the tag "help wanted", and comment on an issue if you feel you have time to implement. If you have an idea for a new feature, create a new issue on GitHub with details for the new feature, I would recommend starting small at first.

Since we have moved from python 2 to python 3 can we include opencv package to maya and use its features for camera tracking?

This should be discussed in a new issue. It is out of scope and off topic for this issue ("Add Custom Image Plane"). I'm not quite sure what you mean by "camera tracking"? Do you mean 2D image tracking, or 3D camera solving?

Thanks, David

[Nik1506]

Hi David, Sorry for not being clear about my Idea. I wanted to ask you have programmed the mm solver using the bundle adjustment. can we use structure from motion to calculate 3d point position to help in camera solving and common geo construction for multiple shots.? Carl Olsson (lth.se) https://www.maths.lth.se/matematiklth/personal/calle/dataset/dataset.html I really appreciate your work and want to learn from you and contribute to your program. can you please guide me how can I be of any help?

On Sun, Apr 3, 2022 at 4:43 AM David Cattermole @.***> wrote:

Hello @Nik1506 https://github.com/Nik1506,

I really like the work done for the MM solver plugin. I would like to learn from you and help if possible to develop this tool further more.

Great! I would recommend you look at the existing issues with the tag "help wanted", and comment on an issue if you feel you have time to implement. If you have an idea for a new feature, create a new issue on GitHub with details for the new feature, I would recommend starting small at first.

Since we have moved from python 2 to python 3 can we include opencv package to maya and use its features for camera tracking?

This should be discussed in a new issue. It is out of scope and off topic for this issue ("Add Custom Image Plane"). I'm not quite sure what you mean by "camera tracking"? Do you mean 2D image tracking, or 3D camera solving?

Thanks, David

— Reply to this email directly, view it on GitHub https://github.com/david-cattermole/mayaMatchMoveSolver/issues/173#issuecomment-1086740020, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARDDVBBGURI7KINVCSRPLRTVDDA6TANCNFSM4TEQA6FA . You are receiving this because you were mentioned.Message ID: @.***>

[david-cattermole]

Hello @Nik1506,

If you want to talk about a different topic than "Add Custom Image Plane" please use the "New Issue" button to create a new "Issue" and add your question there, or sign up to the mailing list (https://groups.google.com/forum/#!forum/maya-matchmove-solver).

Photogrammetry is currently out of scope for this project and I do not plan to add such features. I recommend you use Meshroom: https://alicevision.org/#meshroom

If you would like to help the project, you could try downloading and testing the latest alpha releases and submitting bug reports. You can download the latest alpha release here: https://github.com/david-cattermole/mayaMatchMoveSolver/releases/tag/v0.4.0.alpha3

[david-cattermole]

Closing. This has been released in v0.4.0.