`NetStream::dispose()` (and maybe `NetStream::play()`, `NetStream::close()`) blocking operations

[itlancer]

trafficstars

Problem Description

NetStream::dispose() (and maybe NetStream::play(), NetStream::close()) blocking operations. Any start/stop video playback cause huge render lag with performance hit and it cannot be used where need smooth media content playback.

Tested with multiple AIR versions, even with latest AIR 51.2.1.2 with multiple different devices, different platforms, OS versions with different applications and different videos. Issue exists even with small low-resolution videos. Tested with Video and VideoTexture. The same issue in all cases. Most critical for Android and Windows.

Related issues: https://github.com/airsdk/Adobe-Runtime-Support/issues/3770 https://github.com/airsdk/Adobe-Runtime-Support/issues/3750 https://github.com/airsdk/Adobe-Runtime-Support/issues/3281 https://github.com/airsdk/Adobe-Runtime-Support/issues/3260 https://github.com/airsdk/Adobe-Runtime-Support/issues/3115 https://github.com/airsdk/Adobe-Runtime-Support/issues/2869 https://github.com/airsdk/Adobe-Runtime-Support/issues/2503 https://github.com/airsdk/Adobe-Runtime-Support/issues/2402 https://github.com/airsdk/Adobe-Runtime-Support/issues/2268 https://github.com/airsdk/Adobe-Runtime-Support/issues/2163 https://github.com/airsdk/Adobe-Runtime-Support/issues/2162 https://github.com/airsdk/Adobe-Runtime-Support/issues/1939 https://github.com/airsdk/Adobe-Runtime-Support/issues/1984 https://github.com/airsdk/Adobe-Runtime-Support/discussions/1316#discussioncomment-9622309 https://github.com/airsdk/Adobe-Runtime-Support/issues/1291 https://github.com/airsdk/Adobe-Runtime-Support/issues/646 https://github.com/airsdk/Adobe-Runtime-Support/issues/307 https://github.com/airsdk/Adobe-Runtime-Support/issues/224 https://github.com/airsdk/Adobe-Runtime-Support/issues/155 https://github.com/airsdk/Adobe-Runtime-Support/issues/151 https://github.com/airsdk/Adobe-Runtime-Support/issues/87 https://github.com/airsdk/Adobe-Runtime-Support/issues/15

Steps to Reproduce

Launch application with code below. It playback video in a loop and rotate red rectangle per each frame.

Application example with sources, sample of video and Scout log attached. netstream_blocking_bug.zip

package {
	import flash.display.Sprite;
	import flash.events.Event;
	import flash.net.NetConnection;
	import flash.net.NetStream;
	import flash.media.Video;
	import flash.events.NetStatusEvent;
	
	public class NetStreamBlockingBug extends Sprite {
		private var nc:NetConnection;
		private var ns:NetStream;
		private var video:Video;
		private var sprite:Sprite = new Sprite();
		
		public function NetStreamBlockingBug() {
			addEventListener(Event.ADDED_TO_STAGE, init);
		}
		
		private function init(e:Event):void {
			removeEventListener(Event.ADDED_TO_STAGE, init);
			
			sprite.graphics.beginFill(0xff0000);
			sprite.graphics.drawRect(0, 0, 100, 100);
			sprite.graphics.endFill();
			sprite.x = 100;
			sprite.y = 500;
			
			addEventListener(Event.ENTER_FRAME, enterFrame);
			
			startVideo();
		}
	
		private function startVideo():void {
			removeChildren();
			
			addChild(sprite);
			
			if (ns != null){
				video.attachNetStream(null);
				ns.removeEventListener(NetStatusEvent.NET_STATUS, nsHandler);
				ns.dispose();
			}
		
			if (video != null){
				video.clear();
			}
			
			video = new Video(640, 480);
			addChild(video);
			
			if (nc != null){
				nc.removeEventListener(NetStatusEvent.NET_STATUS, ncHandler);
				nc.close();
			}
			
			nc = new NetConnection();
			nc.addEventListener(NetStatusEvent.NET_STATUS, ncHandler);
			nc.connect(null);
		}
		
		private function ncHandler(e:NetStatusEvent):void {
			if (e.info.code == "NetConnection.Connect.Success"){
				ns = new NetStream(nc);
				ns.checkPolicyFile = false;
				ns.addEventListener(NetStatusEvent.NET_STATUS, nsHandler);				
				ns.client = {onMetaData:getMeta, onPlayStatus:onPlayStatus};
				
				video.attachNetStream(ns);
				ns.play("video.mp4");
			}
		}

		private function nsHandler(e:NetStatusEvent):void {
			trace(e.info.code);
			if (e.info.code == "NetStream.Play.Stop"){
				startVideo();
			}
		}

		private function getMeta(mdata:Object):void { }
	
		private function onPlayStatus(infoObject:Object):void {
			trace("onPlayStatus", infoObject.code);
		}
	
		public function enterFrame(e:Event):void {
			sprite.rotation += 1;
		}
	}
}

Actual Result: On video stop/start you can see huge render lag (red rectangle stuck for ~100 milliseconds) (in archive, without conversion to see it clear): netstream_blocking_bug_video.zip

In Scout you can see huge peaks: Especially for NetStream::dispose() you can see that it blocks other render/logic at least for enormous 40-70 milliseconds (just one this call!) even with high-performance Intel i9 CPUs.

Expected Result: Smooth render. Video playback start/stop should not affect render in main thread.

Known Workarounds

none *write your own native extension to playback video

[ajwfrost]

We've made a couple of changes that improve things a little here - the time for the NetStream::dispose() call varies between 5ms-24ms now. There are some things it's doing that will take some time, and without a complete rearchitecture, we're unlikely to eliminate this. Mostly there are threads that are being shut down and which were each taking a bit of time to do that - but rather than waiting for them to actually exit, we can just orphan them and let them terminate themselves. Plus the closing down of the Windows media transform decoder takes a bit of time (but we can push that to another thread and again just let it happen separately).

Definitely something we can look at further though, if we do end up trying to rearchitect the way the runtime threads are organised, per that 307 issue..

[itlancer]

@ajwfrost Great. If it could cause some side effects for now (before complete rearchitect) - maybe add new parameter like NetStream::dispose(inAnotherThread:Boolean=false) for now... This is most critical for Android and Windows right now.

[itlancer]

@ajwfrost With AIR 51.2.1.4 with Windows it works better now. At least with high-end devices. We will made more tests soon. But nothing changes for Android. Absolutely the same lags/consumtion time even with high-end Android devices:

Scout log attached: netstream_blocking_bug_Android_51.2.1.4.zip

[itlancer]

Well, it much better with latest AIR versions. At least for Windows and Android. It still not ideal. Especially for complex applications with animations and low-end devices launch/stop video cause visual lags for a whole application. I hope it will be improved in the future to completely eliminate this affect. Alongside with https://github.com/airsdk/Adobe-Runtime-Support/issues/307