alicevision
[bug] Rigs, 360 photos, and SfM
Describe the problem
Basic attempts at photogrammetry with equirectangular images, split via the Split360Images node, keep failing at the PrepareDenseScene node with the error:
[16:14:26.759253][warning] The number of rigs is incoherent:
[16:14:26.759253][warning] 0 rigs declared and 1 rigs used.
[16:14:26.759253][error] The input SfMData file 'C:/Users/ICER/Documents/MeshroomCache/StructureFromMotion/26bb648e21fdad50c6d1b0d719df73d12b7a58a0/sfm.abc' cannot be read.
I've tried using many images, varying how many splits are done on the 360 original, and ensuring the photos are of good quality (broad daylight, 4K resolution, clear details)...but I keep getting this error.
I eventually narrowed my tests down to just a single 360 image, split into 36 segments, just to see if I could get anything working properly. The cameras.sfm file generated during the StructureFromMotion node correctly identifies the rig setup:
"rigs": [
{
"rigId": "3860714464",
"subPoses": [
{
"status": "uninitialized",
"pose": {
"rotation": [
"1",
"0",
"0",
"0",
"1",
"0",
"0",
"0",
"1"
],
"center": [
"-0",
"-0",
"-0"
]
}
},
...
And each of the views listed in the same file reference the correct rig:
{
"viewId": "755731533",
"poseId": "755731533",
"rigId": "3860714464",
"subPoseId": "22",
"frameId": "372",
"intrinsicId": "511350622",
"resectionId": "4",
...
But when I feed the generated sfm.abc file into PrepareDenseScene, I always seem to get this error.
I have no clue what could be causing the issue at this point. I don't know how to crack open the abc file data so I can inspect it myself. Importing into blender goes just fine though, so I know its not corrupted.
I've tried fiddling with settings (using ply instead) and rebooting everything, but nothing seems to change.
Is there something obvious I'm missing here?
Desktop:
- OS: Windows 11
- Python: 3.11.8
- Qt/PySide: 6.8.3
- Meshroom version: 2025.1.0
Did you group them into the folder structure that is required when using fixed rigs? https://github.com/alicevision/Meshroom/wiki/Multi-Camera-Rig
Yes. I tried following the guide exactly (dragging in the subfolders) and even tried dragging in the main folder too, just in case I was misunderstanding.
The Split360Images node seems to auto-generate the correct file structure for rigs, so I don't think the issue lies there.
The issue seems to be with the generated sfm.abc file...I'm going to try looking into that and seeing what I dig up.
I've cracked open the sfm.abc file and confirmed the rig information is correctly attached to the cameras:
I went through the source code of relevant files (AlembicImporter) and couldn't see any reasons why I would be losing these declared rigs on import....Guess I'll try debugging my own build next.
I might have found the first hint....I noticed that the rig information is being pulled from the parent Xform of the Camera...but when I try to query it for UserProperties, I keep getting segfaulted:
camera.parent.schema.getUserProperties().getDataType()
Segmentation fault (core dumped)
This happens for any command I run on it...leading me to believe something went wrong with the AlembicExporter and the addSfMCameraRig function in particular. So I dug deeper and realized that there should be a parent object for these camxform objects that I should actually be querying instead.
I would guess that the issue lies somewhere in how this section of code is generating the schema:
std::stringstream ssLabel;
ssLabel << "rigxform_" << std::setfill('0') << std::setw(5) << rigId << "_" << rigPoseId;
std::map<bool, Alembic::AbcGeom::OXform> rigObj;
for (const IndexT viewId : viewIds)
{
const sfmData::View& view = *(sfmData.getViews().at(viewId));
const sfmData::RigSubPose& rigSubPose = rig.getSubPose(view.getSubPoseId());
const bool isReconstructed = (rigSubPose.status != sfmData::ERigSubPoseStatus::UNINITIALIZED);
const std::string name = fs::path(view.getImage().getImagePath()).stem().string();
const std::shared_ptr<camera::IntrinsicBase> intrinsic =
(flagsPart & ESfMData::INTRINSICS) ? sfmData.getIntrinsicSharedPtr(view.getIntrinsicId()) : nullptr;
std::unique_ptr<sfmData::CameraPose> subPosePtr;
if (isReconstructed && (flagsPart & ESfMData::EXTRINSICS))
{
subPosePtr = std::unique_ptr<sfmData::CameraPose>(new sfmData::CameraPose(rigSubPose.pose));
}
Alembic::Abc::OObject& parent = isReconstructed ? _dataImpl->_mvgCameras : _dataImpl->_mvgCamerasUndefined;
if (rigObj.find(isReconstructed) == rigObj.end())
{
// The first time we declare a view, we have to create a RIG entry.
// The RIG entry will be different if the view is reconstructed or not.
rigObj[isReconstructed] = Alembic::AbcGeom::OXform(parent, ssLabel.str());
auto schema = rigObj.at(isReconstructed).getSchema();
schema.set(xformsample);
{
auto userProps = schema.getUserProperties();
OUInt32Property(userProps, "mvg_rigId").set(rigId);
OUInt32Property(userProps, "mvg_poseId").set(rigPoseId);
OUInt16Property(userProps, "mvg_nbSubPoses").set(nbSubPoses);
OBoolProperty(userProps, "mvg_rigPoseLocked").set(rigPoseLocked);
}
}
_dataImpl->addCamera(name, view, subPosePtr.get(), intrinsic, nullptr, &(rigObj.at(isReconstructed)));
I seem to be lacking these rigxform objects in my alembic file—which is what I should be querying for rig data. There must be something going wrong here, even though the source sfm file seems to store the rig data in the expected format.
I forcibly generated a .sfm output file instead of alembic, which actually worked! The rig information was correctly generated, allowing me to move onto the PrepareDenseScene node.
This didn't help for long though...my mesh seems screwed up and the Texture node won't work at all. I'm guessing we can't customize the file output type on the SfM node from .abc for that reason.
Regardless, I'm thinking something is wrong with the AlembicExporter since I was able to maintain rig information with a sfm file instead. I'm curious if anyone else has thoughts about this?