Provide better Swift APIs?

[ztepsa]

Currently it's possible to use iOS Parsec SDK in Swift project, but the experience is not the best and it could be improved. Most of the issues seem to be from auto bridging code into Swift, here's a couple of issues I ran into while recreating SDK example provided in Objective-C.

1. Some of the defines are not available, PARSEC_VER is not availabe in Swift APIs, so we need to manually add one

let PARSEC_VER = ((UInt32(PARSEC_VER_MAJOR) << 16) | UInt32(PARSEC_VER_MINOR))
ParsecInit(PARSEC_VER, nil, nil, &parsec)

2. Parsec client define is missing, here's the image from Swift APIs, there's no Parsec struct.

Since client is just passed around we can bypass this by using OpaquePointer instead of Parsec type. Here's the init method signature for reference:

ParsecInit(ver: UInt32, cfg: UnsafeMutablePointer<ParsecConfig>!, reserved: UnsafeMutableRawPointer!,ps: UnsafeMutablePointer<OpaquePointer?>

Most of these issues and APIs could be upgraded to provide nice support when bridging into Swift, but for the first release it's quite nice! 🚀

[jdjingdian]

Hey, I am a newbie to Swift. I am trying to create a parsec client on iOS since there hasn't an official release for a long time. Thank you very much on the hint about the init method, that's very helpful!!!🌹

[ztepsa]

I'm glad it helped you! I did not play around with it for a while now, but I might find time to cleanup the app and opensource it. It's a SwiftUI app, API client, Parsec client and gamepad support. I used it to play Zelda on my iPad with PS4 joystick and it was really nice. Latest release of the SDK has some API changes so the app is not working (and that's good as there were issues with decoding audio).

[jdjingdian]

I'm glad it helped you! I did not play around with it for a while now, but I might find time to cleanup the app and opensource it. It's a SwiftUI app, API client, Parsec client and gamepad support. I used it to play Zelda on my iPad with PS4 joystick and it was really nice. Latest release of the SDK has some API changes so the app is not working (and that's good as there were issues with decoding audio).

Hey ztepsa! I Just figure out how to connect to the host. My code looks like this.

struct ContentView: View {
    var parsec = UnsafeMutablePointer<OpaquePointer?>.allocate(capacity: 1)
    @State var status: ParsecStatus = ParsecStatus.init(rawValue: -1)
    var body: some View {
        VStack(){
            Text("Hello, world!")
                .padding()
        }.onAppear {
            print("PasecState: \(status)")
            print(ParsecSDK.ParsecVersion())
            status = ParsecSDK.ParsecInit(ParsecSDK.ParsecVersion(), nil, nil,parsec)
            print("PasecState: \(status)")
            if status != PARSEC_OK {
                print("Parsec init failed: \(status)")
            }
            
            ParsecSDK.ParsecClientConnect(parsec.pointee, nil, SESSION_ID, PEER_ID)
            if status != PARSEC_OK {
                print("Parsec Client Connect failed: \(status)")
                
            }else{
                DispatchQueue.global(qos: .userInteractive).asyncAfter(deadline: .now()+5, execute: {
                    while true {
                        status = ParsecClientGLRenderFrame(parsec.pointee, UInt8(DEFAULT_STREAM), nil, nil, 10)
                        print("while Status:\(status)")
                    }
                })
            }
        }
        
    }
}

I try to play some video in fullscreen mode on the host. In the debug session I can see the CPU usage becomes higher, and my demo consume aboout 10Mbps of the network. So I guess my demo is working properly receiving the stream.

However, I am stuck on how to display the rendered screen. I have no idea where is the ouput frame of the ParsecClientGLRenderFrame, and I have no idea how I can show the screen on my screen.

It would be so kind if you could share your code with me. Even if it is not working properly right now, it would give me hints and inspiration.

Thanks in advance, here is my email: [email protected]

[ztepsa]

Parsec is using OpenGL to render the frames (not sure what's the status on Metal support in the SDK, it's implemented in their app, that would make things a lot nicer), so you need a GLKView and rest of the stuff to make it render. My suggestion is to make UIViewController which will handle everything and then you can just expose it to SwiftUI using UIViewControllerRepresentable.

Here's the example from my code:

import UIKit
import SwiftUI
import GLKit
import OpenGLES

extension ParsecViewController: UIViewControllerRepresentable {
  typealias UIViewControllerType = ParsecViewController
  func makeUIViewController(context: UIViewControllerRepresentableContext<ParsecViewController>) -> ParsecViewController {
    return ParsecViewController(peerId: peerId, sessionId: sessionId)
  }
  
  func updateUIViewController(_ uiViewController: ParsecViewController, context: UIViewControllerRepresentableContext<ParsecViewController>) {
    
  }
}

final class ParsecViewController: UIViewController {
  private let parsec: Parsec
  private let peerId: String
  private let sessionId: String
  
  private let glView: GLKView
  private let glContext: EAGLContext
  private let glController: GLKViewController
  
  init(parsec: Parsec = Parsec(), peerId: String, sessionId: String) {
    self.parsec = parsec
    self.peerId = peerId
    self.sessionId = sessionId
    let context = EAGLContext(api: .openGLES2)!
    let view = GLKView(frame: UIScreen.main.bounds, context: context)
    let controller = GLKViewController()
    controller.view = view
    controller.preferredFramesPerSecond = 60
    self.glContext = context
    self.glView = view
    self.glController = controller
    super.init(nibName: nil, bundle: nil)
    self.view.backgroundColor = .red
    view.delegate = self
    controller.delegate = self
    self.addChild(controller)
    self.view.addSubview(controller.view)
    controller.didMove(toParent: self)
    print("Parsec initialized.")
  }
  
  required init?(coder: NSCoder) {
    fatalError("init(coder:) has not been implemented")
  }
  
  override func viewDidAppear(_ animated: Bool) {
    super.viewDidAppear(animated)
    print("Parsec setup...")
    guard parsec.setup() else { fatalError("Could not perform setup") }
    print("Parsec connecting...")
    guard parsec.connect(to: peerId, sessionId: sessionId) else { fatalError("Could not connect") }
    print("Parsec connected.")
  }
  
  override func viewWillDisappear(_ animated: Bool) {
    super.viewWillDisappear(animated)
    parsec.destroy()
    print("Parsec destroyed.")
  }
  
  deinit {
    print("Parsec deinitialized.")
  }
}

extension ParsecViewController: GLKViewControllerDelegate {
  func glkViewControllerUpdate(_ controller: GLKViewController) {
//    print("glkViewControllerUpdate: \(controller)")
  }
}

extension ParsecViewController: GLKViewDelegate {
  func glkView(_ view: GLKView, drawIn rect: CGRect) {
    parsec.drawingHandler(view, drawIn: rect)
  }
}

Along with this one you need a Parsec client, my version needs updates to work with the latest SDK, but there's and example so it should be easy to fix. Anyhow here's the example:

import ParsecSDK
import GLKit

final class Parsec {
  public var version = ((UInt32(PARSEC_VER_MAJOR) << 16) | UInt32(PARSEC_VER_MINOR))
  private var parsec: OpaquePointer? = nil
  private var audio: UnsafeMutablePointer<audio>? = nil
  private var config: ParsecConfig
  
  public func drawingHandler(_ view: GLKView, drawIn rect: CGRect) {
    guard let client = parsec, let decoder = audio else { return }
    ParsecClientPollAudio(client, audio_cb, 0, decoder)
    ParsecClientSetDimensions(parsec, UInt32(view.frame.width), UInt32(view.frame.height), Float(UIScreen.main.scale))
    ParsecClientGLRenderFrame(parsec, nil, nil, nil, 8)
    glFinish() // May improve latency
  }
  
  private var logHandler: ParsecLogCallback = { (log, message, _) in
    if let msg = message, let payload = String(validatingUTF8: msg) {
      let prefix = log == LOG_DEBUG ? "[debug]" : "[info]"
      let output = [prefix, ": ", payload].joined()
      print(output)
    }
  }
  
  init(clientPort: Int32 = 8000, hostPort: Int32 = 9000, upnp: Bool = true) {
    self.config = ParsecConfig(upnp: upnp ? 1 : 0, clientPort: clientPort, hostPort: hostPort)
  }
  
  func setup() -> Bool {
    let ps = ParsecInit(self.version, &self.config, nil, &self.parsec)
    guard ps == PARSEC_OK else { return false }
    audio_init(&self.audio)
    guard self.audio != nil else { return false }
    ParsecSetLogCallback(self.logHandler, nil)
    return true
  }
  
  func destroy() {
    ParsecSetLogCallback(nil, nil)
    ParsecDestroy(parsec)
    audio_destroy(&audio)
  }
  
  deinit {
    guard parsec != nil else { return }
    destroy()
  }
}

extension Parsec {
  func connect(to peerId: String, sessionId: String) -> Bool {
    return peerId.withCString { peerIdPtr in
      return sessionId.withCString { sessionIdPtr in
        let ps = ParsecClientConnect(self.parsec,
                                     nil,
                                     UnsafeMutablePointer(mutating: sessionIdPtr),
                                     UnsafeMutablePointer(mutating: peerIdPtr))
        return ps == PARSEC_OK
      }
    }
  }
}

After all of this it can be used like this, as for peerId and sessionId you can create a login screen in the app and hit their API, or for testing just grab you credentials and hardcode them in the app.

struct ParsecView: View {
    
    let peerId = "some-peer-id"
    let sessionId = "some-sesson-id"
    
    var body: some View {
      ParsecViewController(
        peerId: peerId,
        sessionId: sessionId
      )
    }
}

Once I find time to play around with my project I'll ping you, this should give you enough to play around with 👍 Also my suggestion is to join their Discord server, there's Parsec and ParsecSDK, you can also find me there.

[jdjingdian]

Parsec is using OpenGL to render the frames (not sure what's the status on Metal support in the SDK, it's implemented in their app, that would make things a lot nicer), so you need a GLKView and rest of the stuff to make it render. My suggestion is to make UIViewController which will handle everything and then you can just expose it to SwiftUI using UIViewControllerRepresentable.

Here's the example from my code:

import UIKit
import SwiftUI
import GLKit
import OpenGLES

extension ParsecViewController: UIViewControllerRepresentable {
  typealias UIViewControllerType = ParsecViewController
  func makeUIViewController(context: UIViewControllerRepresentableContext<ParsecViewController>) -> ParsecViewController {
    return ParsecViewController(peerId: peerId, sessionId: sessionId)
  }
  
  func updateUIViewController(_ uiViewController: ParsecViewController, context: UIViewControllerRepresentableContext<ParsecViewController>) {
    
  }
}

final class ParsecViewController: UIViewController {
  private let parsec: Parsec
  private let peerId: String
  private let sessionId: String
  
  private let glView: GLKView
  private let glContext: EAGLContext
  private let glController: GLKViewController
  
  init(parsec: Parsec = Parsec(), peerId: String, sessionId: String) {
    self.parsec = parsec
    self.peerId = peerId
    self.sessionId = sessionId
    let context = EAGLContext(api: .openGLES2)!
    let view = GLKView(frame: UIScreen.main.bounds, context: context)
    let controller = GLKViewController()
    controller.view = view
    controller.preferredFramesPerSecond = 60
    self.glContext = context
    self.glView = view
    self.glController = controller
    super.init(nibName: nil, bundle: nil)
    self.view.backgroundColor = .red
    view.delegate = self
    controller.delegate = self
    self.addChild(controller)
    self.view.addSubview(controller.view)
    controller.didMove(toParent: self)
    print("Parsec initialized.")
  }
  
  required init?(coder: NSCoder) {
    fatalError("init(coder:) has not been implemented")
  }
  
  override func viewDidAppear(_ animated: Bool) {
    super.viewDidAppear(animated)
    print("Parsec setup...")
    guard parsec.setup() else { fatalError("Could not perform setup") }
    print("Parsec connecting...")
    guard parsec.connect(to: peerId, sessionId: sessionId) else { fatalError("Could not connect") }
    print("Parsec connected.")
  }
  
  override func viewWillDisappear(_ animated: Bool) {
    super.viewWillDisappear(animated)
    parsec.destroy()
    print("Parsec destroyed.")
  }
  
  deinit {
    print("Parsec deinitialized.")
  }
}

extension ParsecViewController: GLKViewControllerDelegate {
  func glkViewControllerUpdate(_ controller: GLKViewController) {
//    print("glkViewControllerUpdate: \(controller)")
  }
}

extension ParsecViewController: GLKViewDelegate {
  func glkView(_ view: GLKView, drawIn rect: CGRect) {
    parsec.drawingHandler(view, drawIn: rect)
  }
}

Along with this one you need a Parsec client, my version needs updates to work with the latest SDK, but there's and example so it should be easy to fix. Anyhow here's the example:

import ParsecSDK
import GLKit

final class Parsec {
  public var version = ((UInt32(PARSEC_VER_MAJOR) << 16) | UInt32(PARSEC_VER_MINOR))
  private var parsec: OpaquePointer? = nil
  private var audio: UnsafeMutablePointer<audio>? = nil
  private var config: ParsecConfig
  
  public func drawingHandler(_ view: GLKView, drawIn rect: CGRect) {
    guard let client = parsec, let decoder = audio else { return }
    ParsecClientPollAudio(client, audio_cb, 0, decoder)
    ParsecClientSetDimensions(parsec, UInt32(view.frame.width), UInt32(view.frame.height), Float(UIScreen.main.scale))
    ParsecClientGLRenderFrame(parsec, nil, nil, nil, 8)
    glFinish() // May improve latency
  }
  
  private var logHandler: ParsecLogCallback = { (log, message, _) in
    if let msg = message, let payload = String(validatingUTF8: msg) {
      let prefix = log == LOG_DEBUG ? "[debug]" : "[info]"
      let output = [prefix, ": ", payload].joined()
      print(output)
    }
  }
  
  init(clientPort: Int32 = 8000, hostPort: Int32 = 9000, upnp: Bool = true) {
    self.config = ParsecConfig(upnp: upnp ? 1 : 0, clientPort: clientPort, hostPort: hostPort)
  }
  
  func setup() -> Bool {
    let ps = ParsecInit(self.version, &self.config, nil, &self.parsec)
    guard ps == PARSEC_OK else { return false }
    audio_init(&self.audio)
    guard self.audio != nil else { return false }
    ParsecSetLogCallback(self.logHandler, nil)
    return true
  }
  
  func destroy() {
    ParsecSetLogCallback(nil, nil)
    ParsecDestroy(parsec)
    audio_destroy(&audio)
  }
  
  deinit {
    guard parsec != nil else { return }
    destroy()
  }
}

extension Parsec {
  func connect(to peerId: String, sessionId: String) -> Bool {
    return peerId.withCString { peerIdPtr in
      return sessionId.withCString { sessionIdPtr in
        let ps = ParsecClientConnect(self.parsec,
                                     nil,
                                     UnsafeMutablePointer(mutating: sessionIdPtr),
                                     UnsafeMutablePointer(mutating: peerIdPtr))
        return ps == PARSEC_OK
      }
    }
  }
}

After all of this it can be used like this, as for peerId and sessionId you can create a login screen in the app and hit their API, or for testing just grab you credentials and hardcode them in the app.

struct ParsecView: View {
    
    let peerId = "some-peer-id"
    let sessionId = "some-sesson-id"
    
    var body: some View {
      ParsecViewController(
        peerId: peerId,
        sessionId: sessionId
      )
    }
}

Once I find time to play around with my project I'll ping you, this should give you enough to play around with 👍 Also my suggestion is to join their Discord server, there's Parsec and ParsecSDK, you can also find me there.

Thank you very much! Your explanation is very detailed, and these codes are very helpful for me to understand the SDK!🌹🌹

[jdjingdian]

I focus on the video frame and ignored the transmission of audio. I modify your code to use the latest parsec api, and change the opengles2 to opengles3, then I finally got frames show on my screen, though the performance is poor, it still makes me very happy, thank you again for your help!🌹🌹🌹🌹

[sgade]

Still, improving the Swift support would be great. Currently, there is only an old Objective-C/C example for OpenGL. Now, there is a metal call available.