Drawing `Video` (or `SoundMixer.computeSpectrum()`) using `NetStream` in "Data Generation Mode" cause `SecurityError`

[itlancer]

trafficstars

Problem Description

Drawing Video (or SoundMixer.computeSpectrum() call) using NetStream in "Data Generation Mode" cause SecurityError: Error #2123. It critical for some video streams playback tasks or providing FLV bytes to NetStream. Many years ago it was mentioned here: https://stackoverflow.com/questions/27464444/getting-bitmap-from-video-decoded-with-nestream-appendbytes-as3

Tested with multiple AIR versions, even with latest AIR 51.0.0.3. With multiple different Windows devices with different OS versions with different applications and architectures. Same issue in all cases.

Related issue: https://github.com/airsdk/Adobe-Runtime-Support/issues/224 https://github.com/airsdk/Adobe-Runtime-Support/issues/180 https://github.com/airsdk/Adobe-Runtime-Support/issues/171 https://github.com/airsdk/Adobe-Runtime-Support/issues/87

Steps to Reproduce

Launch application with code below and click anywhere on stage. It will try to make screenshot of video (using BitmapData::draw()).

Application example with sources and sample of video attached. netstream_play_null_draw_bug.zip

package {
	import flash.display.Sprite;
	import flash.events.Event;
	import flash.net.NetConnection;
	import flash.net.NetStream;
	import flash.media.Video;
	import flash.events.NetStatusEvent;
	import flash.events.MouseEvent;
	import flash.display.Bitmap;
	import flash.display.BitmapData;
	import flash.utils.ByteArray;
	import flash.net.NetStreamAppendBytesAction;
	import flash.media.SoundMixer;
	
	public class NetStreamPlayNullDrawBug extends Sprite {
		private var nc:NetConnection;
		private var ns:NetStream;
		private var video:Video = new Video(640, 480);
		private var bitmap:Bitmap = new Bitmap();
		
		[Embed(source = "video.flv", mimeType = "application/octet-stream")]
		public var VideoFile:Class;
		
		public function NetStreamPlayNullDrawBug() {
			addEventListener(Event.ADDED_TO_STAGE, init);
		}
		
		private function init(e:Event):void {
			removeEventListener(Event.ADDED_TO_STAGE, init);
			
			addChild(video);
			
			bitmap.x = 650;
			addChild(bitmap);
			
			nc = new NetConnection();
			nc.addEventListener(NetStatusEvent.NET_STATUS, ncHandler);
			nc.connect(null);
			
			stage.addEventListener(MouseEvent.CLICK, drawScreenshot);
		}
		
		private function drawScreenshot(e:MouseEvent = null):void {
			var bd:BitmapData = new BitmapData(video.width, video.height, true, 0x00000000);
			bd.draw(video);//This line cause SecurityError: Error #2123: Security sandbox violation: BitmapData.draw: app:/netstream_play_null_draw_bug.swf cannot access null. No policy files granted access.
			//SoundMixer.computeSpectrum(new ByteArray());//This line also cause SecurityError: Error #2123: Security sandbox violation: SoundMixer.computeSpectrum: app:/netstream_play_null_draw_bug.swf cannot access null. No policy files granted access.
			
			bitmap.bitmapData = bd;
		}
		
		private function ncHandler(e:NetStatusEvent):void {
			trace("e.info.code", e.info.code);			
			if (e.info.code == "NetConnection.Connect.Success"){
				ns = new NetStream(nc);
				ns.client = {onMetaData:getMeta};
				ns.addEventListener(NetStatusEvent.NET_STATUS, nsHandler);
				video.attachNetStream(ns);
				ns.play(null);
				//ns.play("");//Uncomment this line for workaround
				
				var videoBytes:ByteArray = new VideoFile();
				ns.appendBytesAction(NetStreamAppendBytesAction.RESET_SEEK);
				ns.appendBytesAction(NetStreamAppendBytesAction.RESET_BEGIN);
				ns.appendBytes(new VideoFile());
			}
		}

		private function nsHandler(e:NetStatusEvent):void {
			trace(e.info.code);
		}

		private function getMeta(mdata:Object):void { }
	}
}

Actual Result: Exception:

SecurityError: Error #2123: Security sandbox violation: BitmapData.draw: app:/netstream_play_null_draw_bug.swf cannot access null. No policy files granted access.
	at flash.display::BitmapData/draw()

Expected Result: Screenshot of video will be made and displayed.

Known Workarounds

After ns.play(null) call ns.play(""):

ns.play(null);
ns.play("");

But it could cause incorrect video playback or false-positive NetStream.Play.StreamNotFound events.

[ajwfrost]

So .. I think that workaround is actually a bug i.e. that only works because there's a value not being reset properly.

The security error is being thrown deliberately in this mode - any server-based connection should result in the same thing, local files are set up so you can have access to them for snapshots etc, but unless an RTMP server has explicitly granted the access, anything streamed would have this restriction.

In this instance, there's nothing to tell the NetStream object that the byte array isn't just data that's retrieved via some online stream, hence the restriction will apply.

Is this something you need specifically when the video source is an embedded character in a SWF file? or are you looking for snapshots of any random FLV file that's pulled from somewhere online?

thanks

[itlancer]

@ajwfrost I'm using ffmpeg to "convert" local video files or video streams (which AIR not supported: AVI, H.265, RTSP, ...) "on the fly" to FLV container (with H.264 or H.263 codec) to provide it to NetStream for playback. So it's not RTMP in this "problem" case. I need to take screenshot from "locally provided FLV video bytes".

[ajwfrost]

Okay thanks .. will check whether there are any restrictions we have to abide by here, although I would have thought if you're already able to convert things via ffmpeg then it would also be possible for you to rip off the video frames in the same way!

So perhaps the appendBytes() mechanism is one where we can allow the screenshot, whilst still preventing it if you're using an RTMP stream...

[itlancer]

although I would have thought if you're already able to convert things via ffmpeg then it would also be possible for you to rip off the video frames in the same way!

Unfortunately it not suitable for me in this case cause I need not only a video frame but "stage screenshot" (what actually right now on the stage with all other rendered objects).

[ajwfrost]

Yes, sorry, I didn't mean as a solution to this problem, I just meant there doesn't seem any sense in having AIR throw a security error when you're trying to do something that's possible via other mechanisms already.

So we can adjust it to provide this capability when it's using the appendBytes thing, pretty trivial change...

[itlancer]

@ajwfrost Fixed with AIR 51.0.0.4 for BitmapData::draw() usage. But the same issue still exists using SoundMixer.computeSpectrum(new ByteArray());:

SecurityError: Error #2123: Security sandbox violation: SoundMixer.computeSpectrum: app:/netstream_play_null_draw_bug.swf cannot access null. No policy files granted access.
	at flash.media::SoundMixer$/computeSpectrum()

[itlancer]

@ajwfrost Issue with SoundMixer.computeSpectrum(new ByteArray()); still exists using AIR 50.2.4.5.

[itlancer]

@ajwfrost Issue with SoundMixer.computeSpectrum(new ByteArray()); still exists using AIR 50.2.5.1.

[itlancer]

Finally fixed with latest AIR 51.0.1.1. Thanks!