ios-jsc

ios-jsc copied to clipboard

I have a requirement to do preloading of video in our app when a thumbnail comes into view. Prescribed way to do this is by using AVAssetDownloadURLSession.

We used this link for reference: https://developer.apple.com/library/content/documentation/AudioVideo/Conceptual/MediaPlaybackGuide/Contents/Resources/en.lproj/HTTPLiveStreaming/HTTPLiveStreaming.html

Upon implementation of this, there is a significant drop in FPS that degrades the user experience massively. If preloading is turned off, UI is always above 50FPS. As soon as we introduce this, while preloading, UI drops down 10-30FPS.

I've tried putting this into a worker and it resulted in no difference.

Please find code below that executes the preloading once called. It is in TypeScript and we are using ng2.

import { isAndroid, isIOS } from "platform";

import { usingSimulator } from "../shared/utils/using-simulator.util";
import { osVersion } from "../shared/utils/os-version.util";

export class SpotAssetManager {

    assets: SpotAssetContainer[] = [];

    private _AVAssetDownloadDelegate: any;
    private _AVAssetDownloadConfig: any;
    private _downloadSession: AVAssetDownloadURLSession;
    
    private constructor() {
        
    }

    init() {
        if (isIOS) {
            this._setupDelegate();
        }

    }

    private _setupDelegate() {
        this._AVAssetDownloadDelegate = AVAssetDownloadDelegateImpl.new();
        this._AVAssetDownloadDelegate.wrapper = this;
        this._AVAssetDownloadConfig = NSURLSessionConfiguration.backgroundSessionConfigurationWithIdentifier("downloadIdentifier");
        this._downloadSession = AVAssetDownloadURLSession.sessionWithConfigurationAssetDownloadDelegateDelegateQueue(this._AVAssetDownloadConfig, this._AVAssetDownloadDelegate, null);
    }

    public preloadAsset(asset: FeatureAssetView | TrailerAssetView, percentageToPreload: number): void {

        // Downloading files like this does not work on iOS less than 10 or the simulator.
        // Simulator check only runs if environment is NOT prod to ensure no potential issues.
        if (isIOS) {
            if (osVersion() >= 10 && !usingSimulator()) {
                this.downloadIOSFile(_asset);
            }
    }

    public downloadIOSFile(asset: FeatureAssetView | TrailerAssetView) {
        let _externalFile = this._createExternalFile(asset.src);
        let _downloadTask = AVAssetDownloadURLSession.prototype.assetDownloadTaskWithURLAssetAssetTitleAssetArtworkDataOptions.apply(this._downloadSession, [_externalFile, asset.id, null, null])
        _downloadTask.taskDescription = asset.id;
        _downloadTask.resume();
        // Asset is directly pushed to array as AVFoundation is smart enough to link up this asset and the localFile once its downloaded
        this.assets.push(new SpotAssetContainer(asset, _externalFile, SpotAssetContainerStatus.loading, 0, false));
    }

    private _createExternalFile(externalFileUrl: string): AVURLAsset {
        console.log("externalFileUrl " + externalFileUrl);
        let videoUrl: NSURL = NSURL.URLWithString(externalFileUrl);
        let externalFile = AVURLAsset.alloc().initWithURLOptions(videoUrl, null);
        return externalFile;
    }

export class AVAssetDownloadDelegateImpl extends NSObject implements AVAssetDownloadDelegate {

    wrapper;

    public static ObjCProtocols = [AVAssetDownloadDelegate] // define our native protocols

    URLSessionAssetDownloadTaskDidFinishDownloadingToURL(session: NSURLSession, assetDownloadTask: AVAssetDownloadTask, location: NSURL): void {
        console.log("Finished Downloading " + assetDownloadTask.taskDescription);
    }

    URLSessionAssetDownloadTaskDidLoadTimeRangeTotalTimeRangesLoadedTimeRangeExpectedToLoad?(session: NSURLSession, assetDownloadTask: AVAssetDownloadTask, timeRange: CMTimeRange, loadedTimeRanges: NSArray<NSValue>, timeRangeExpectedToLoad: CMTimeRange): void {

        console.log("Downloading " + assetDownloadTask.taskDescription);
    }

}

Hey @bnussey,

According to Apple documentation when you pass null to the DelegateQueue parameter of sessionWithConfigurationAssetDownloadDelegateDelegateQueue method you are using a serial queue. Serial queues are executing one task at a time, blocking the Main thread until that task is finished. So you could try initializing an AVAssetDownloadURLSession session in the following way AVAssetDownloadURLSession.sessionWithConfigurationAssetDownloadDelegateDelegateQueue(this._AVAssetDownloadConfig, this._AVAssetDownloadDelegate, NSOperationQueue.mainQueue);

If this does not work for you you could share a sample project that demonstrates your problem

Hey @KristinaKoeva I've uploaded a sample project here - https://www.dropbox.com/s/6yto3og7171wa9h/preload-video.zip?dl=0

If you run this on a device older than the 7, you will notice a significant frame rate drop. All the relevant code is in item/preload.ts