NextLevel
NextLevel copied to clipboard
Published
20 hours ago •
NextLevel
NextLevel
Frozen Previews
I came across an issue that persists with the previewView (referencing to demo) or perhaps the previewView's layer (previewView.layer.addSublayer(NextLevel.shared.previewLayer)) freezing.
The previewView is a generic view in the view controller with a tap method AND long press method attached to the same uibutton. The tap method calls the following method:
NextLevel.shared.capturePhotoFromVideo()
The view freezes when (1) a selfie is taken, (2) a photo with the back camera is taken, and then (3) a video recording is taken with the front camera. Each time these actions are executed, it pushes to a view controller, but whenever its popped back to the camera using this API, it freezes at (3). What could some of the causes for this be? (ie: flipping device's camera?)
hey @HackShitUp you just need to move your camera start to viewWillAppear or don't stop the camera.
hey @HackShitUp i don't have enough information to debug the issue in your app. i suggest not stopping the camera, as you mention you're doing here: https://github.com/NextLevel/NextLevel/issues/95#issuecomment-325444491
also, if you can provide an example or gist with the sample project that shows this issue and it being caused within NextLevel itself. or the lines you're using for setup specifically.
also ensure that the preview layer is being re-enabled after photo capture
/// Un-freezes the live camera preview layer.
public func unfreezePreview() {
if let previewConnection = self.previewLayer.connection {
previewConnection.isEnabled = true
}
}
@piemonte Thank you so much for your response. I haven't been able to debug the issue for some time. Sorry for this, but this is the entirety of the class using NextLevel. If you scroll down to the extension, you'll be able to see how I manage your framework's delegate methods. Perhaps there might be an issue there? Or could it be possible that I'm not utilizing the session's clips? :
//
// Camera.swift
// Created by Joshua Choi on 8/25/17.
// Copyright © 2017 Joshua Choi. All rights reserved.
import UIKit
import AVFoundation
import GPUImage
import NextLevel
class Camera: UIViewController, UIGestureRecognizerDelegate {
/// Initialized CGPoint to determine when the pan gesture began for zooming in.
var _panStartPoint: CGPoint = .zero
/// Initialized CGFloat to determine when the pan gesture began for zooming in.
var _panStartZoom: CGFloat = 0.0
/// Initialized UIView for front camera flash.
var flashView: UIView!
@IBOutlet var previewView: UIView!
@IBOutlet weak var captureButton: UIButton!
@IBOutlet weak var flashButton: UIButton!
@IBAction func toggleFlash(_ sender: Any) {
// MARK: - NextLevel
if NextLevel.shared.flashMode == .on {
// Set flashButton image
flashButton.setImage(UIImage(named: "FlashOff"), for: .normal)
// Set flashMode
NextLevel.shared.flashMode = .off
// Set torchMode
NextLevel.shared.torchMode = .off
}
if NextLevel.shared.flashMode == .off {
// Set flashButton image
flashButton.setImage(UIImage(named: "FlashOn"), for: .normal)
// Set flashMode
NextLevel.shared.flashMode = .on
// Set torchMode
NextLevel.shared.torchMode = .on
}
}
@IBOutlet weak var exitButton: UIButton!
@IBAction func exit(_ sender: Any) {
}
@IBOutlet weak var searchButton: UIButton!
@IBAction func search(_ sender: Any) {
let searchVC = self.storyboard?.instantiateViewController(withIdentifier: "searchVC") as! Search
self.navigationController?.pushViewController(searchVC, animated: true)
}
@IBOutlet weak var cameraRollButton: UIButton!
@IBAction func createAssets(_ sender: Any) {
// Present Albums.swift over the current UIViewController and make it appear as if it's overlayed over the current context
let albumsVC = self.storyboard?.instantiateViewController(withIdentifier: "albumsVC") as! Albums
// MARK: - CSNavigationController
let csNavigationController = CSNavigationController.init(rootViewController: albumsVC)
self.navigationController?.tabBarController?.present(csNavigationController, animated: true, completion: nil)
}
@IBOutlet weak var newTextButton: UIButton!
@IBAction func createNewText(_ sender: Any) {
// Present NewTextPost.swift over the current UIViewController and make it appear as if it's overlayed over the current context
let newTextPostVC = self.storyboard?.instantiateViewController(withIdentifier: "newTextPostVC") as! NewTextPost
// MARK: - CSNavigationController
let csNavigationController = CSNavigationController.init(rootViewController: newTextPostVC)
self.navigationController?.tabBarController?.present(csNavigationController, animated: true, completion: nil)
}
@IBOutlet weak var swapCameraButton: UIButton!
@IBAction func switchCamera(_ sender: Any) {
// MARK: - NextLevel
NextLevel.shared.flipCaptureDevicePosition()
// Set UIImage
if NextLevel.shared.devicePosition == .back {
swapCameraButton.setImage(UIImage(named: "SwapCamera"), for: .normal) // Back Camera
} else {
swapCameraButton.setImage(UIImage(named: "Stickers"), for: .normal) // Front Camera
}
}
/// Function: UIGestureRecognizer that manages the tap-to-focus.
func handleFocusTapGestureRecognizer(_ gestureRecognizer: UIGestureRecognizer) {
let tapPoint = gestureRecognizer.location(in: self.previewView)
let focusView = UIImageView(frame: CGRect(x: 0.0, y: 0.0, width: 30.0, height: 30.0))
focusView.image = UIImage(named: "Camera")
focusView.center.x = tapPoint.x
focusView.center.y = tapPoint.y
view.addSubview(focusView)
// Animate focusView
UIView.animate(withDuration: 0.25, delay: 0.0, options: .curveEaseInOut, animations: {
focusView.alpha = 1.0
focusView.transform = CGAffineTransform(scaleX: 1.25, y: 1.25)
}, completion: { (success) in
UIView.animate(withDuration: 0.15, delay: 0.5, options: .curveEaseInOut, animations: {
focusView.alpha = 0.0
focusView.transform = CGAffineTransform(translationX: 0.6, y: 0.6)
}, completion: { (success) in
focusView.removeFromSuperview()
})
})
/// MARK: - NextLevel; Focus the view.
let adjustedPoint = NextLevel.shared.previewLayer.captureDevicePointOfInterest(for: tapPoint)
NextLevel.shared.focusExposeAndAdjustWhiteBalance(atAdjustedPoint: adjustedPoint)
}
/// Function: Zoom into the frame of the video.
func zoomFrame(_ gestureRecognizer: UIPinchGestureRecognizer) {
let zoomScale = gestureRecognizer.scale
NextLevel.shared.videoZoomFactor = Float(zoomScale)
if gestureRecognizer.state == .ended {
gestureRecognizer.scale = 1.0
NextLevel.shared.videoZoomFactor = 1.0
}
}
/// Function: Capture an image from the current video frame.
func captureImage(_ gestureRecognizer: UIGestureRecognizer) {
// Play system camera shutter sound.
AudioServicesPlaySystemSoundWithCompletion(SystemSoundID(1108), nil)
// MARK: - NextLevel; Capture photo from video
NextLevel.shared.capturePhotoFromVideo()
}
/// Function: Record a video with the current video session.
func recordVideo(_ gestureRecognizer: UIGestureRecognizer) {
if gestureRecognizer.state == .began {
// Change video orientation if recording w the front camera.
if NextLevel.shared.devicePosition == .front {
NextLevel.shared.mirroringMode = .on
}
// MARK: - NextLevel
NextLevel.shared.record()
self._panStartPoint = gestureRecognizer.location(in: self.view)
self._panStartZoom = CGFloat(NextLevel.shared.videoZoomFactor)
} else if gestureRecognizer.state == .changed {
let newPoint = gestureRecognizer.location(in: self.view)
let scale = (self._panStartPoint.y / newPoint.y)
let newZoom = (scale * self._panStartZoom)
NextLevel.shared.videoZoomFactor = Float(newZoom)
} else if gestureRecognizer.state == .ended || gestureRecognizer.state == .cancelled || gestureRecognizer.state == .failed {
// MARK: - NextLevel
NextLevel.shared.pause()
}
}
/// MARK: - View Life Cycle
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
// Hide UINavigationBar
navigationController?.setNavigationBarHidden(true, animated: false)
// Hide UIStatusBar
UIApplication.shared.isStatusBarHidden = false
UIApplication.shared.statusBarStyle = .lightContent
self.setNeedsStatusBarAppearanceUpdate()
// MARK: - CSExtensions; Configure UITabBar
navigationController?.tabBarController?.tabBar.designTabBar(withShadow: false)
// MARK: - NextLevel; Begin Camera Session
if NextLevel.shared.authorizationStatus(forMediaType: AVMediaTypeVideo) == .authorized &&
NextLevel.shared.authorizationStatus(forMediaType: AVMediaTypeAudio) == .authorized {
do {
try NextLevel.shared.start()
// NextLevel.shared.unfreezePreview()
} catch let error {
print("NextLevel, failed to start camera session: \(error.localizedDescription as Any)")
}
} else {
NextLevel.shared.requestAuthorization(forMediaType: AVMediaTypeVideo)
NextLevel.shared.requestAuthorization(forMediaType: AVMediaTypeAudio)
}
}
override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
// Hide UINavigationBar
navigationController?.setNavigationBarHidden(true, animated: false)
// Hide UIStatusBar
UIApplication.shared.isStatusBarHidden = false
UIApplication.shared.statusBarStyle = .lightContent
self.setNeedsStatusBarAppearanceUpdate()
// MARK: - CSExtensions; Configure UITabBar
navigationController?.tabBarController?.tabBar.designTabBar(withShadow: false)
}
override func viewDidLoad() {
super.viewDidLoad()
// Add record video method with UILongPressGestureRecognizer on captureButton
let pressRecord = UILongPressGestureRecognizer(target: self, action: #selector(recordVideo(_:)))
pressRecord.minimumPressDuration = 0.5
pressRecord.allowableMovement = 10.0
pressRecord.delegate = self
captureButton.addGestureRecognizer(pressRecord)
// Add capture photo method with UITapGestureRecognizer on captureButton
let captureTap = UITapGestureRecognizer(target: self, action: #selector(captureImage(_:)))
captureTap.numberOfTapsRequired = 1
captureButton.addGestureRecognizer(captureTap)
// Add zoom method with UIPinchGestureRecognizer on the previewView
let pinchGesture = UIPinchGestureRecognizer(target: self, action: #selector(zoomFrame(_:)))
pinchGesture.delegate = self
previewView.addGestureRecognizer(pinchGesture)
// Add focus method with UITapGestureRecognizer on the previewView
let focusTap = UITapGestureRecognizer(target: self, action: #selector(handleFocusTapGestureRecognizer(_:)))
focusTap.numberOfTapsRequired = 1
previewView.addGestureRecognizer(focusTap)
// Add switch camera method with UITapGestureRecognizer on the previewView
let doubleTap = UITapGestureRecognizer(target: self, action: #selector(switchCamera(_:)))
doubleTap.numberOfTapsRequired = 2
previewView.addGestureRecognizer(doubleTap)
// Configure previewView
previewView.backgroundColor = UIColor.black
NextLevel.shared.previewLayer.frame = previewView.bounds
// MARK: - NextLevel; Add PreviewLayer
previewView.layer.addSublayer(NextLevel.shared.previewLayer)
// Bring buttons to front
/// Buttons ([Any]) --> MUST ADD NEW BUTTONS HERE
let buttons = [flashButton, swapCameraButton, searchButton, cameraRollButton,
newTextButton, exitButton, captureButton] as [Any]
for b in buttons {
// MARK: - CSExtensions
(b as AnyObject).layer.applyShadow()
((b as AnyObject) as! UIView).translatesAutoresizingMaskIntoConstraints = true
view.addSubview((b as AnyObject) as! UIView)
view.bringSubview(toFront: (b as AnyObject) as! UIView)
}
// MARK: - NextLevel Delegate Methods
let nextLevel = NextLevel.shared
nextLevel.delegate = self
nextLevel.deviceDelegate = self
nextLevel.flashDelegate = self
nextLevel.videoDelegate = self
nextLevel.photoDelegate = self
nextLevel.captureMode = .video
nextLevel.photoStabilizationEnabled = true
nextLevel.videoStabilizationMode = .auto
nextLevel.mirroringMode = .auto
nextLevel.automaticallyConfiguresApplicationAudioSession = false // DONT manage audio with running session.
// nextLevel.automaticallyUpdatesDeviceOrientation = true // Allow update of device orientation.
// nextLevel.isVideoCustomContextRenderingEnabled = true // Allow analysis of frame buffers.
// MARK: - NextLevel Video Delegate
nextLevel.videoConfiguration.bitRate = 2000000
nextLevel.videoConfiguration.scalingMode = AVVideoScalingModeResizeAspectFill
nextLevel.videoConfiguration.maximumCaptureDuration = CMTimeMakeWithSeconds(10.0, Int32(1))
nextLevel.videoConfiguration.preset = AVCaptureSessionPresetHigh
// MARK: - NextLevel Audio Delegate
nextLevel.audioConfiguration.bitRate = 96000
}
override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
// MARK: - NextLevel
NextLevel.shared.stop()
}
}
extension Camera: NextLevelDelegate, NextLevelVideoDelegate, NextLevelPhotoDelegate, NextLevelFlashAndTorchDelegate, NextLevelDeviceDelegate {
// MARK: - NextLevel Delegate Methods *******************************************************************************************
// Permission
func nextLevel(_ nextLevel: NextLevel, didUpdateAuthorizationStatus status: NextLevelAuthorizationStatus, forMediaType mediaType: String) {
print("• NL updated authorization status for: \(mediaType).")
}
// Configuration
func nextLevel(_ nextLevel: NextLevel, didUpdateVideoConfiguration videoConfiguration: NextLevelVideoConfiguration) {
print("NL updated video configuration.")
}
func nextLevel(_ nextLevel: NextLevel, didUpdateAudioConfiguration audioConfiguration: NextLevelAudioConfiguration) {
print("NL updated audio configuration.")
}
// Session
func nextLevelSessionWillStart(_ nextLevel: NextLevel) { print("NL session will start.") }
func nextLevelSessionDidStart(_ nextLevel: NextLevel) { print("NL session started.") }
func nextLevelSessionDidStop(_ nextLevel: NextLevel) { print("NL session stopped.") }
// Session Interruption
func nextLevelSessionWasInterrupted(_ nextLevel: NextLevel) { print("NL session was interrupted.") }
func nextLevelSessionInterruptionEnded(_ nextLevel: NextLevel) { print("NL session interruption ended.") }
// Preview
func nextLevelWillStartPreview(_ nextLevel: NextLevel) { print("NL will start preview.") }
func nextLevelDidStopPreview(_ nextLevel: NextLevel) { print("NL did stop preview.") }
// Mode
func nextLevelCaptureModeWillChange(_ nextLevel: NextLevel) { print("NL capture mode will change.") }
func nextLevelCaptureModeDidChange(_ nextLevel: NextLevel) { print("NL capture mode did change.") }
// *******************************************************************************************************************************
// MARK: - NextLevel Video Delegate Methods ***************************************************************************************
// Video Zoom
func nextLevel(_ nextLevel: NextLevel, didUpdateVideoZoomFactor videoZoomFactor: Float) {
print("Did update video zoom factor to \(videoZoomFactor).")
}
// Video Processing
func nextLevel(_ nextLevel: NextLevel, willProcessRawVideoSampleBuffer sampleBuffer: CMSampleBuffer, onQueue queue: DispatchQueue) {
print("NL will process raw video sample buffer.")
}
func nextLevel(_ nextLevel: NextLevel, renderToCustomContextWithImageBuffer imageBuffer: CVPixelBuffer, onQueue queue: DispatchQueue) {
print("NL will render to custom context with image buffer.")
}
// Video Recording Session
func nextLevel(_ nextLevel: NextLevel, didSetupVideoInSession session: NextLevelSession) { print("NL did setup video in session.") }
func nextLevel(_ nextLevel: NextLevel, didSetupAudioInSession session: NextLevelSession) { print("NL did setup audio in session.") }
func nextLevel(_ nextLevel: NextLevel, didStartClipInSession session: NextLevelSession) {
print("NL did start clip in session.")
// Hide UIButtons
}
func nextLevel(_ nextLevel: NextLevel, didCompleteClip clip: NextLevelClip, inSession session: NextLevelSession) {
// Export recorded video...
session.mergeClips(usingPreset: AVAssetExportPresetHighestQuality, completionHandler: { (url: URL?, error: Error?) in
if let _ = url {
// Show UIButtons
self.cameraRollButton.isHidden = false
self.newTextButton.isHidden = false
self.flashButton.isHidden = false
self.searchButton.isHidden = false
self.swapCameraButton.isHidden = false
// Push to CapturedVideo
let capturedVideoVC = self.storyboard?.instantiateViewController(withIdentifier: "capturedVideoVC") as! CapturedVideo
capturedVideoVC.capturedURL = url
self.navigationController?.pushViewController(capturedVideoVC, animated: false)
} else if let _ = error {
print(error?.localizedDescription as Any)
// MARK: - CSHelpers
self.csHelpers.showError(withTitle: "Error recording video.")
}
})
}
func nextLevel(_ nextLevel: NextLevel, didAppendVideoSampleBuffer sampleBuffer: CMSampleBuffer, inSession session: NextLevelSession) {
print("NL did append video sample buffer.")
}
func nextLevel(_ nextLevel: NextLevel, didAppendAudioSampleBuffer sampleBuffer: CMSampleBuffer, inSession session: NextLevelSession) {
print("NL did append audio sample buffer.")
}
func nextLevel(_ nextLevel: NextLevel, didSkipVideoSampleBuffer sampleBuffer: CMSampleBuffer, inSession session: NextLevelSession) {
print("NL did skip video sample buffer.")
}
func nextLevel(_ nextLevel: NextLevel, didSkipAudioSampleBuffer sampleBuffer: CMSampleBuffer, inSession session: NextLevelSession) {
print("NL did skip audio sample buffer.")
}
func nextLevel(_ nextLevel: NextLevel, didCompleteSession session: NextLevelSession) {
print("NL did complete session.")
}
// Capture photo from vide frame
func nextLevel(_ nextLevel: NextLevel, didCompletePhotoCaptureFromVideoFrame photoDict: [String : Any]?) {
// Export captured photo from live video frame.
if let dictionary = photoDict {
if let photoData = dictionary[NextLevelPhotoJPEGKey] {
if let data = photoData as? Data {
if NextLevel.shared.devicePosition == .front {
/// Pass UIImage
let stillVC = self.storyboard?.instantiateViewController(withIdentifier: "stillVC") as! CapturedStill
stillVC.stillImage = UIImage(cgImage: UIImage(data: data)!.cgImage!, scale: 1.0, orientation: .upMirrored)
self.navigationController?.pushViewController(stillVC, animated: false)
} else {
/// Pass UIImage
let stillVC = self.storyboard?.instantiateViewController(withIdentifier: "stillVC") as! CapturedStill
stillVC.stillImage = UIImage(data: data)
self.navigationController?.pushViewController(stillVC, animated: false)
}
}
}
}
}
// *******************************************************************************************************************************
// MARK: - NextLevel Photo Delegate Methods **************************************************************************************
func nextLevel(_ nextLevel: NextLevel, willCapturePhotoWithConfiguration photoConfiguration: NextLevelPhotoConfiguration) {
print("NL will capture photo with configuration.")
}
func nextLevel(_ nextLevel: NextLevel, didCapturePhotoWithConfiguration photoConfiguration: NextLevelPhotoConfiguration) {
print("NL did capture photo with configuration.")
}
func nextLevel(_ nextLevel: NextLevel, didProcessPhotoCaptureWith photoDict: [String: Any]?, photoConfiguration: NextLevelPhotoConfiguration) {
print("NL did process photo capture.")
}
func nextLevel(_ nextLevel: NextLevel, didProcessRawPhotoCaptureWith photoDict: [String: Any]?, photoConfiguration: NextLevelPhotoConfiguration) {
print("NL did process raw photo capture with.")
}
func nextLevelDidCompletePhotoCapture(_ nextLevel: NextLevel) {
print("NL did complete photo capture.")
}
// *******************************************************************************************************************************
// MARK: - NextLevel Flash Delegate Methods **************************************************************************************
func nextLevelDidChangeFlashMode(_ nextLevel: NextLevel) {
print("NL did change flash mode.")
}
func nextLevelDidChangeTorchMode(_ nextLevel: NextLevel) {
print("NL did change torch mode.")
}
func nextLevelFlashActiveChanged(_ nextLevel: NextLevel) {
print("NL flash active changed.")
}
func nextLevelTorchActiveChanged(_ nextLevel: NextLevel) {
print("NL torch active changed.")
}
func nextLevelFlashAndTorchAvailabilityChanged(_ nextLevel: NextLevel) {
print("NL torch active changed.")
}
// *******************************************************************************************************************************
// MARK: - NextLevel Device Delegate Methods *************************************************************************************
// Position and orientation
func nextLevelDevicePositionWillChange(_ nextLevel: NextLevel) {
print("NL device position will change.")
}
func nextLevelDevicePositionDidChange(_ nextLevel: NextLevel) {
print("NL device position did change.")
}
func nextLevel(_ nextLevel: NextLevel, didChangeDeviceOrientation deviceOrientation: NextLevelDeviceOrientation) {
print("NL did change device orientation.")
}
// Format
func nextLevel(_ nextLevel: NextLevel, didChangeDeviceFormat deviceFormat: AVCaptureDeviceFormat) {
print("NL did change device format.")
}
// Aperature
func nextLevel(_ nextLevel: NextLevel, didChangeCleanAperture cleanAperture: CGRect) {
print("NL did change clean aperture.")
}
// Focus, exposure, and white balance.
func nextLevelWillStartFocus(_ nextLevel: NextLevel) {
print("NL will start focus.")
}
func nextLevelDidStopFocus(_ nextLevel: NextLevel) {
print("NL did stop focus.")
}
func nextLevelWillChangeExposure(_ nextLevel: NextLevel) {
print("NL will change exposure.")
}
func nextLevelDidChangeExposure(_ nextLevel: NextLevel) {
print("NL did change exposure.")
}
func nextLevelWillChangeWhiteBalance(_ nextLevel: NextLevel) {
print("NL will change white balance.")
}
func nextLevelDidChangeWhiteBalance(_ nextLevel: NextLevel) {
print("NL did change white balance.")
}
// *******************************************************************************************************************************
}