having trouble understanding pose matrix....

[ofZach]

trafficstars

Hi ! I am trying to use the pose matrix to project points in space but having trouble understanding what's happening internally, I am basically trying to project points on three axis to make the same orientation as drawDebugPose. I saw there is a pose example, but that is "drawing" I kind of need project points myself (I need to calculate a 3d plane which is oriented to the face)

here's what I get when I try to draw it (next to drawDebugPose, which looks right):

I see that loadPoseMatrix() is also calling transformPosePosition().... I am a little lost about what's happening there.

if I wanted to rotate points by hand (rather than drawing) to make something that looks like drawDebugPose is there an easy way to do this?

ie:

if(tracker.size() > 0){ 
        ofMatrix4x4 pose = tracker.getInstances()[0].getPoseMatrix();
        ofMultMatrix(pose);
        ofPoint aa(0,0,0);
        ofPoint bb(100,0,0);
        ofPoint cc(0,100,0);
        ofPoint dd(0,0,100);
        ofLine(aa, bb);
        ofLine(aa, cc);
        ofLine(aa, dd);
    }

or

    if(tracker.size() > 0){
        ofMatrix4x4 pose = tracker.getInstances()[0].getPoseMatrix();
        ofPoint aa(0,0,0);
        ofPoint bb(100,0,0);
        ofPoint cc(0,100,0);
        ofPoint dd(0,0,100);
        aa = pose * aa;
        bb = pose * bb;
        cc = pose * cc;
        dd = pose * dd;
        ofLine(aa, aa + (bb-aa).getNormalized() * 100);
        ofLine(aa, aa + (cc-aa).getNormalized()* 100);
        ofLine(aa, aa + (dd-aa).getNormalized()* 100);
        
    }

[ofZach]

may be related:

https://github.com/HalfdanJ/ofxFaceTracker2/issues/12

may be useful:

https://forum.openframeworks.cc/t/ofxfactetracker2-how-to-manipulate-pose-matrix/25173

[ofZach]

I experimented with the code here:

https://forum.openframeworks.cc/t/ofxfactetracker2-how-to-manipulate-pose-matrix/25173

and found I can get something that's close-ish so what draw debug is giving -- mine is in white, the original draw debug is in r/g/b

my intuition is that the only difference is the projection matrix that happens in draw debug -- so maybe this is ok and it's just visually different since it's two different projections?

[HalfdanJ]

I have had a feeling that the debug drawer is wrong for some time, so maybe yours is actually more correct then the one I wrote.

On Sun, Dec 17, 2017, 15:51 ofZach [email protected] wrote:

I experimented with the code here:

https://forum.openframeworks.cc/t/ofxfactetracker2-how-to-manipulate-pose-matrix/25173

and found I can get something that's close-ish so what draw debug is giving -- mine is in white, the original draw debug is in r/g/b

[image: screen shot 2017-12-17 at 4 48 12 pm] https://user-images.githubusercontent.com/142897/34084169-1bab35c4-e34a-11e7-8d3c-e1accab37c62.png

[image: screen shot 2017-12-17 at 4 45 52 pm] https://user-images.githubusercontent.com/142897/34084171-1bc1e8a0-e34a-11e7-8e0f-4c2e3cccefc6.png

my intuition is that the only difference is the projection matrix that happens in draw debug -- so maybe this is ok and it's just visually different since it's two different projections?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/HalfdanJ/ofxFaceTracker2/issues/34#issuecomment-352288483, or mute the thread https://github.com/notifications/unsubscribe-auth/AAN4yWfIvaKnDqyG50yA7tWeLrbKY1ueks5tBYzZgaJpZM4REsm3 .

[ofZach]

it could be but debug draw looks better to my eye... I guess I am just confused about why Intrinsics::loadProjectionMatrix is called in debug draw -- is it important somehow?

I experimented with making an ofCamera and setting the FOV really low gives something visually similar to debug draw.

with a normal FOV

with a very low FOV -- matches closer...

[ofZach]

also, if it's helpful, setting up an ortho camera (ofCamera::enableOrtho) also produces something very visually similar to debug draw.....

[ofZach]

I don't have a PR yet, but just a quick note that I found this useful:

http://answers.opencv.org/question/23089/opencv-opengl-proper-camera-pose-using-solvepnp/

I am only interested in rotation, so something like this works for me after the solvepnp call to get a matrix which captures rotation in a way that's more OF compatible :

rotation = cv::Mat::zeros(4, 4, CV_64F);
    cv::Mat viewMatrix = cv::Mat::zeros(4, 4, CV_64F);
    cv::Rodrigues(poservec, rotation);
    
    for(unsigned int row=0; row<3; ++row)
    {
        for(unsigned int col=0; col<3; ++col)
        {
            viewMatrix.at<double>(row, col) = rotation.at<double>(row, col);
        }
        viewMatrix.at<double>(row, 3) = 0; //posetvec.at<double>(row, 0);
    }
    viewMatrix.at<double>(3, 3) = 1.0f;
    cv::Mat cvToGl = cv::Mat::zeros(4, 4, CV_64F);
    glViewMatrix = cv::Mat::zeros(4, 4, CV_64F);
    cv::transpose(viewMatrix , glViewMatrix);   // <--- glViewMatrix has rotation in a way easy to use...

dunno if it's helpful, just posting if folks are in similar boat to me trying to get useful info out...

[vanderlin]

I found my way to this thread via searching. I am too having trouble getting a directional vector from the pose.

I experimented with the code from this thread - https://forum.openframeworks.cc/t/ofxfactetracker2-how-to-manipulate-pose-matrix/25173/2?u=vanderlin

but looks like I'm way off. The pink line is me...

I was trying to convert pitch, roll, yaw to a unit vector.

x = cos(yaw)*cos(pitch)
y = sin(yaw)*cos(pitch)
z = sin(pitch)

I also tried to get a vector from the rotation quaternion.

 glm::vec3 vec = rotation * glm::vec3(0, 0, 1);

Am I totally off?