SwiftUI Support

[usayuki]

Is your feature request related to a problem? Please describe. Is it possible to use it in SwiftUI?

Describe the solution you'd like I want to display VideoRenderView in Swift UI. It is possible to place it by using UIViewRepresentable, but I need to pass a VideoRenderView in bindVideoView, so I would like to know how to do it.

[zhinang-amazon]

Hi @usayuki You should be able to create a class that conforms to both UIViewRepresentable and VideoRenderView. Your makeUIView() method can return a UIImageView or any UIView that you want to render the video frames on. you renderFrame() method needs to update the UIView based on the CVPixelBuffer.

You might not be able to use DefaultVideoRenderView out of the box, but it's a good reference. Unfortunately, we have not written any demo with SwiftUI, but I hope this information can help you.

[usayuki]

@zhinang-amazon Thanks for the reply.

I think we need a CVPixelBuffer to render with the renderFrame() method. However, the ChimeSDK has no way for us to handle the received CVPixelBuffer. Therefore, even if we create a class that can be handled by SwiftUI, we can't render it. If there is a way to do that, please let me know.

[zhinang-amazon]

Your renderFrame() method will be invoked by VideoTileController::onReceiveFrame with the CVPixelBuffer You can look at DefaultVideoTileController. If you are using default implementation for all facade and controllers, you just need to implement renderFrame(), and it will be invoked automatically when there is a new video frame for any attendee.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[SantoshKhandare59]

Is it possible to share example of how to display VideoRenderView or VideoTileController in SWIFTUI?

[adesun2k]

Hi @usayuki You should be able to create a class that conforms to both UIViewRepresentable and VideoRenderView. Your makeUIView() method can return a UIImageView or any UIView that you want to render the video frames on. you renderFrame() method needs to update the UIView based on the CVPixelBuffer.

You might not be able to use DefaultVideoRenderView out of the box, but it's a good reference. Unfortunately, we have not written any demo with SwiftUI, but I hope this information can help you.

Thanks for this but can you help us with a Complete ChimeSDK SwiftUI example to make the stuff easy for us?

[adesun2k]

Is it possible to share example of how to display VideoRenderView or VideoTileController in SWIFTUI?

[adesun2k]

My project is completely built using SWIFTUI no storyboard. I was able to get this to work and the audio call works perfectly fine but I don't know how to render the video signal in SWIFTUI any help on this? I have tried meetingModel.videoModel but don't know where or how to render the Video signal in SWIFTUI. Any help or idea please?

[adesun2k]

Your renderFrame() method will be invoked by VideoTileController::onReceiveFrame with the CVPixelBuffer You can look at DefaultVideoTileController. If you are using default implementation for all facade and controllers, you just need to implement renderFrame(), and it will be invoked automatically when there is a new video frame for any attendee.

Hi zhinang Can you just show a little bit of how to implement this. I have spent 2 months trying to understand this but to no avail. Thanks

[adesun2k]

Hi @usayuki You should be able to create a class that conforms to both UIViewRepresentable and VideoRenderView. Your makeUIView() method can return a UIImageView or any UIView that you want to render the video frames on. you renderFrame() method needs to update the UIView based on the CVPixelBuffer.

You might not be able to use DefaultVideoRenderView out of the box, but it's a good reference. Unfortunately, we have not written any demo with SwiftUI, but I hope this information can help you.

I got this errors Non-class type 'Kall.ImageView' cannot conform to class protocol 'VideoSink' Non-class type 'Kall.ImageView' cannot conform to class protocol 'VideoRenderView'

Here is my ImageView code struct ImageView: UIViewRepresentable,VideoRenderView {

        var name: String
        
        fileprivate var imageView: UIImageView = UIImageView()
        fileprivate var originalImage: UIImage
        

        init(name: String) {
            self.name = name
            self.originalImage = UIImage(named: name)!
        }
        
        func makeUIView(context: Context) -> UIImageView {
            imageView.image = self.originalImage

            return imageView;
        }
        
        func updateUIView(_ uiView: UIImageView, context: Context) {
        }
        
        fileprivate func scaledImage(width: CGFloat, height: CGFloat) -> UIImage {
            let size = CGSize(width: width, height: height)
            if (self.originalImage.size == size) {
                return self.originalImage
            }
            
            UIGraphicsBeginImageContextWithOptions(size, false, 0.0)
            self.originalImage.draw(in: CGRect(x: 0, y: 0, width: width, height: height))
            let image = UIGraphicsGetImageFromCurrentImageContext()
            UIGraphicsEndImageContext()
            
            return image!;
        }
        
        func resize(width: CGFloat, height: CGFloat) -> some View {
            self.imageView.image = scaledImage(width: width, height: height)
            print(self.imageView.image!.size)

            return self.frame(width: width, height: height)
        }
    }

I will be glad if anyone can help. Thanks

[hokyungh]

Hi @adesun2k one workaround you could do is put wrapper around UIViewRepresentable

struct CustomVideoView: UIViewRepresentable {
    var defaultRenderView: DefaultVideoRenderView
    
    func updateUIView(_ uiView: UIView, context: Context) {
    }
    
    func makeUIView(context: Context) -> UIView {
        return defaultRenderView
    }
}

var remoteVideoView = CustomVideoView(defaultRenderView: DefaultVideoRenderView())

and when you bind audioVideo?.bindVideoView(videoView: remoteVideoView.defaultRenderView, tileId: tileState.tileId) just use defaultRenderView.

Let me know if this workaround works for you.

Thanks.

[zeeshanz]

@adesun2k were you able to make it work? I am also stuck trying to make Chime work in SwiftUI.

[adesun2k]

@adesun2k were you able to make it work? I am also stuck trying to make Chime work in SwiftUI. What worked for me was to wrap the ViewController that contains the Storyboard using UIViewControllerRepresentable. I removed every scene except the tab that contains the Video render.

[zeeshanz]

@adesun2k were you able to make it work? I am also stuck trying to make Chime work in SwiftUI. What worked for me was to wrap the ViewController that contains the Storyboard using UIViewControllerRepresentable. I removed every scene except the tab that contains the Video render.

@adesun2k Thanks, it is encouraging to know that there is a solution after all. I'll try that too and see how it goes. If you get a chance to post some code for reference, that will save my time. I am stuck on it for over a week and I tend to struggle a bit with UIViewRepresentable.

[jonah-katz]

I am successfully using SwiftUI without storyboard. @zeeshanz post your code?

[zeeshanz]

I am successfully using SwiftUI without storyboard. @zeeshanz post your code?

@jonah-katz I am using storyboard so I have nothing new to post. If you have managed to make it work without the storyboard, could you please share your code?

[jonah-katz]

Sure.

I basically have this class which houses the UIKit imageview:

final class VideoTileView: UIViewRepresentable, VideoRenderView {
    var imageView: UIImageView = UIImageView()
    func onVideoFrameReceived(frame: VideoFrame) {
        if Thread.isMainThread {
            renderFrame(frame: frame)
        } else {
            DispatchQueue.main.async {
                self.renderFrame(frame: frame)
            }
        }
    }
    
    func makeUIView(context: Context) -> UIImageView {
        return imageView
    }
    
    private func renderFrame(frame: VideoFrame) {
        guard let buffer = (frame.buffer as? VideoFramePixelBuffer)?.pixelBuffer else {
            return
        }

        var cgImage: CGImage?
        VTCreateCGImageFromCVPixelBuffer(buffer, options: nil, imageOut: &cgImage)
        guard let image = cgImage else {
            return
        }
        
        let i =  UIImage(cgImage: image)
        if i != nil {
            imageView.image = i
        }
    }
}

And then I have a manager class (similar to the controller class in the demo), which initiates and saves a a bunch of VideoTilesView's:

class MeetingManager: ObservableObject, VideoTileObserver {
    @Published var video_tile_views: Array<VideoTileView> = []
init() {...make the meeting session, set up observers, etc....}
 func videoTileDidAdd(tileState: VideoTileState) {
      video_tile_views.append(VideoTileView())
      activeMeetingSession!.audioVideo.bindVideoView(videoView: video_tile_views[video_tile_views.append.count - 1])
 }
}

Then finally in the View I can just grab the views and render them:

  VStack { 
       ForEach(meeting_manager.video_tile_views, id: \.self) { $0() }
  }
}

PS. wrote that all offhand. Im sure there's some bugs, but it captures the important pieces.

[notapplicableio]

How's everyone getting along with this? If it's not too late then @hokyungh solution worked for me.