Capture Displays Nodes in Different Placement

[avierkant]

Hi,

Thank you for the framework! Super useful

I'm wondering if you (or anyone else) have gotten the following error:

When capturing photo or video with ARVideoKit, in some specific instances the photos/videos don't match the placement of nodes in the scene.

(see attached images--the one with a circular camera button at bottom of image is a screenshot of how ARSCNView is displaying the image on my device, and the other image is the image captured by ARVideoKit and saved to camera roll).

This is only happening for me when nodes are populated by fetching the user's current position from currentFrame with frame.camera.transform. When I populate nodes in the scene with positions relative to the rootNode, ARVideoKit works perfectly and captures look exactly like what is displayed on the device.

I'm wondering if this is something having to do with how ARVideoKit interacts with currentFrame, and if getting the location in this way is somehow adjusting the position that the nodes appear to be in the scene when it's passed to ARVideoKit to render and export? Any help would be very much appreciated!

Thanks

[AFathi]

@avierkant, can you please specify what device you're using?

It would also be great if you add snippet of the code in order for me to recreate the issue.

[avierkant]

@AFathi Running on an iPhone 8 Plus

Here's what I'm using to fetch user position, and the function that calls it:

func getUserVector() -> (SCNVector3, SCNVector3) { // (direction, position)

        if let frame = sceneView.session.currentFrame {
    
        let mat = SCNMatrix4(frame.camera.transform)
        let dir = SCNVector3(0, frame.camera.eulerAngles.y, 0)
        let pos = SCNVector3(mat.m41, mat.m42, mat.m43) 
        return (dir, pos)
    }
    return (SCNVector3(0, 0, 0), SCNVector3(0, 0, -0.2))
}

func addRandom() {
    
    let rand01 = SCNNode()
    rand01.geometry = SCNPlane(width: 2, height: 2)
    let randomSelection = arc4random_uniform(213)
    rand01.geometry?.firstMaterial?.diffuse.contents = UIImage(named: "\(randomSelection).png")
    rand01.geometry?.firstMaterial?.isDoubleSided = true
    let (direction, position) = getUserVector()
    rand01.position = position
    let rand01Rotation = direction
    rand01.eulerAngles = rand01Rotation

    self.sceneView.scene.rootNode.addChildNode(rand01)

    
}

[raulbude]

I am experiencing the same problem.

[bergenrothn97]

My videos are recorded differently than what they are on the screen. My text nodes are oriented differently in the recorded movie than they are in the ARKit SceneView. Can you help?

[bergenrothn97]

Nevermind, figured it out. Turns out I had a camera node in my scn file and ARVideoKit was taking video from that point.