me 学习 rtmp 开发 (Part 4: flash and ams)

学习 rtmp 开发 (Part 4: flash and ams)

Open nonocast opened this issue 2 years ago • 0 comments

很多同学，包括网上大量文章，在非农rtmp的时候都是直接拿起wireshark和spec一顿操作，我倒是有个建议不妨站远点去理解rtmp，会有一个非常不同的视角。

v2-1dd39f0fe20402a2307b4d7048ffde20_1440w

1993, FutureWave Software在SmartSketch的基础上开发了FutureSplash Animator
1995, Adobe拒绝了FutureWave的出售邀约
1996, Macromedia收购FutureWave，并将其FutureSplash Animator命名为Macromedia Flash 1.0
2005, Adobe收购Macromedia, 同时将Flash, Dreamweaver和Fireworks当年的三剑客纳入Adobe旗下
2017, 终结

其中最主要的几个概念：

fla and swf: Flash的源文件，只能用Flash打开编辑，编辑后导出swf
flash player: 借助浏览器NPAPI，flash player将swf成功放到浏览器中
as: ActionScript的缩写，通过这个script可以扩展fla的能力
flv: 一开始swf是直接存视频文件，当时过于大以后，flv成了单独的视频外置资源

最早flash只是一个带交互的video clip，然后通过as扩展了交互性，也不仅局限在browser local的限制，于是有了AMF0和AMF3，其实对标的就是Web Service/SOAP以及RMI，通过这个去访问外部或者后台数据。

再然后flash就想着在浏览器里面实现视频会议的功能，所以2002年就开发了FMS(Flash Media Server)，然后被Adobe收购后才改为AMS(Adobe Media Server)，AMS需要被Flash所调用，所以就有了RTMP+AMF，我们看下flash是如何发布本地摄像头和声音的代码你就明白RTMP协议之根本:

nc = new NetConnection();
nc.connect("rtmp://localhost/aaaa");
nsOut = new NetStream(nc);
nsOut.attachVideo(Camera.get());
nsOut.attachAudio(Microphone.get());
nsOut.publish("1", "live");

推流的API非常直白，和RTMP的协议纹丝合缝，所以这里就有了几个概念:

flash swf是客户端，闭源
ams是流媒体服务器，闭源
rtmp是ams的流媒体协议，被迫开放
flv是rtmp的录像文件格式

最早逆向破解RTMP是为了跳开flash来实现流媒体服务，而后再是有了srs, red5, nginx-rtmp实现了rtmp server取代AMS。

我们再来看看RTMP和FLV之间密不可分的关系，

2002年发布Flash 6 Player，同一时刻包含了Flash Video, RTMP和AMF，这里的Video是指在swf嵌入video文件
2003年发布Flash 7 Player中支持了FLV文件

从Adobe的Fideo File Format Specification中可以看到，flv很大程度上就是前置了rtmp，

Starting with SWF files published for Flash Player 6, Flash Player can exchange audio, video, and data over RTMP connections with the Adobe Flash Media ServerTM. One way to feed data to Flash Media Server (and thus on to Flash Player clients) is from files in the FLV file format. Starting with SWF files published for Flash Player 7, Flash Player can also play FLV files directly with MIME type video/x-flv.

回顾一下FLV的结构:

The FLV header
The FLV file body
- FLV tags
  - Audio tags (TagType == 0x08)
  - Video tags (TagType == 0x09)
  - Script data (TagType == 0x12)

我开始就好奇为什么是8,9,18,而不是1, 2, 3，其实这里就和RTMP的Message Type对上了:

0x01 = Set Packet Size Message.
0x02 = Abort.
0x03 = Acknowledge.
0x04 = Control Message.
0x05 = Server Bandwidth
0x06 = Client Bandwidth.
0x07 = Virtual Control.
0x08 = Audio Packet.
0x09 = Video Packet.
0x0F = Data Extended.
0x10 = Container Extended.
0x11 = Command Extended (An AMF3 type command).
0x12 = Data (Invoke (onMetaData info is sent as such)).
0x13 = Container.
0x14 = Command (An AMF0 type command).
0x15 = UDP
0x16 = Aggregate
0x17 = Present

所以，

rtmp可以抽象理解为message based，handshake和chunk过程交给librtmp处理就ok
flv可以抽象理解为tag based，只需要将audio(0x08),video(0x09),data(0x12)的message通过tag封装进flv即可
AVC codec出来的是AVCC(nalu-len nalu nalu-len nalu...)就是video中的payload
h264 annexb的格式就是将nalu大散，通过start code(0x00 0x00 0x00 0x01)隔开

最后通过rtmpdump验证如下:

rtmpdump -V -r rtmp://your-domain/app/foo -o out.flv
RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
DEBUG: Parsing...
DEBUG: Parsed protocol: 0
DEBUG: Parsed host    : play.nonocast.cn
DEBUG: Parsed app     : app
DEBUG: Protocol : RTMP
DEBUG: Hostname : play.nonocast.cn
DEBUG: Port     : 1935
DEBUG: Playpath : foo
DEBUG: tcUrl    : rtmp://play.nonocast.cn:1935/app
DEBUG: app      : app
DEBUG: live     : no
DEBUG: timeout  : 30 sec
DEBUG: Setting buffer time to: 36000000ms
Connecting ...
DEBUG: RTMP_Connect1, ... connected, handshaking
DEBUG: HandShake: Type Answer   : 03
DEBUG: HandShake: Server Uptime : -745572240
DEBUG: HandShake: FMS Version   : 0.0.0.0
DEBUG: RTMP_Connect1, handshaked
DEBUG: Invoking connect
INFO: Connected...
DEBUG: HandleServerBW: server BW = 5000000
DEBUG: HandleClientBW: client BW = 5000000 2
DEBUG: HandleChangeChunkSize, received: chunk size change to 8000
DEBUG: RTMP_ClientPacket, received: invoke 190 bytes
DEBUG: (object begin)
DEBUG: (object begin)
DEBUG: Property: <Name:             fmsVer, STRING:	FMS/3,0,1,123>
DEBUG: Property: <Name:       capabilities, NUMBER:	31.00>
DEBUG: (object end)
DEBUG: (object begin)
DEBUG: Property: <Name:              level, STRING:	status>
DEBUG: Property: <Name:               code, STRING:	NetConnection.Connect.Success>
DEBUG: Property: <Name:        description, STRING:	Connection succeeded.>
DEBUG: Property: <Name:     objectEncoding, NUMBER:	0.00>
DEBUG: (object end)
DEBUG: (object end)
DEBUG: HandleInvoke, server invoking <_result>
DEBUG: HandleInvoke, received result for method call <connect>
DEBUG: sending ctrl. type: 0x0003
DEBUG: Invoking createStream
DEBUG: RTMP_ClientPacket, received: invoke 29 bytes
DEBUG: (object begin)
DEBUG: Property: NULL
DEBUG: (object end)
DEBUG: HandleInvoke, server invoking <_result>
DEBUG: HandleInvoke, received result for method call <createStream>
DEBUG: SendPlay, seekTime=0, stopTime=0, sending play: foo
DEBUG: Invoking play
DEBUG: sending ctrl. type: 0x0003
DEBUG: RTMP_ClientPacket, received: invoke 96 bytes
DEBUG: (object begin)
DEBUG: Property: NULL
DEBUG: (object begin)
DEBUG: Property: <Name:              level, STRING:	status>
DEBUG: Property: <Name:               code, STRING:	NetStream.Play.Start>
DEBUG: Property: <Name:        description, STRING:	Start live>
DEBUG: (object end)
DEBUG: (object end)
DEBUG: HandleInvoke, server invoking <onStatus>
DEBUG: HandleInvoke, onStatus: NetStream.Play.Start
Starting download at: 0.000 kB
DEBUG: RTMP_ClientPacket, received: notify 24 bytes
DEBUG: (object begin)
DEBUG: (object end)
DEBUG: HandleCtrl, received ctrl. type: 0, len: 6
DEBUG: HandleCtrl, Stream Begin 1
DEBUG: RTMP_ClientPacket, received: notify 573 bytes
DEBUG: (object begin)
DEBUG: (object begin)
DEBUG: Property: <Name:              width, NUMBER:	200.00>
DEBUG: Property: <Name:             height, NUMBER:	200.00>
DEBUG: Property: <Name:       displayWidth, NUMBER:	200.00>
DEBUG: Property: <Name:      displayHeight, NUMBER:	200.00>
DEBUG: Property: <Name:           duration, NUMBER:	0.00>
DEBUG: Property: <Name:          framerate, NUMBER:	10.00>
DEBUG: Property: <Name:                fps, NUMBER:	10.00>
DEBUG: Property: <Name:      videodatarate, NUMBER:	2500.00>
DEBUG: Property: <Name:       videocodecid, NUMBER:	7.00>
DEBUG: Property: <Name:      audiodatarate, NUMBER:	160.00>
DEBUG: Property: <Name:       audiocodecid, NUMBER:	10.00>
DEBUG: Property: <Name:            profile, STRING:	>
DEBUG: Property: <Name:              level, STRING:	>
DEBUG: Property: <Name:             Server, STRING:	Tengine>
DEBUG: Property: <Name:     videocodecreal, NUMBER:	0.00>
DEBUG: Property: <Name:           fileSize, NUMBER:	0.00>
DEBUG: Property: <Name:    audiosamplerate, NUMBER:	48000.00>
DEBUG: Property: <Name:    audiosamplesize, NUMBER:	16.00>
DEBUG: Property: <Name:      audiochannels, NUMBER:	2.00>
DEBUG: Property: <Name:             stereo, BOOLEAN:	TRUE>
DEBUG: Property: <Name:                2.1, BOOLEAN:	FALSE>
DEBUG: Property: <Name:                3.1, BOOLEAN:	FALSE>
DEBUG: Property: <Name:                4.0, BOOLEAN:	FALSE>
DEBUG: Property: <Name:                4.1, BOOLEAN:	FALSE>
DEBUG: Property: <Name:                5.1, BOOLEAN:	FALSE>
DEBUG: Property: <Name:                7.1, BOOLEAN:	FALSE>
DEBUG: Property: <Name:            encoder, STRING:	obs-output module (libobs version 27.2.4)>
DEBUG: (object end)
DEBUG: (object end)
INFO: Metadata:
INFO:   width                 200.00
INFO:   height                200.00
INFO:   displayWidth          200.00
INFO:   displayHeight         200.00
INFO:   duration              0.00
INFO:   framerate             10.00
INFO:   fps                   10.00
INFO:   videodatarate         2500.00
INFO:   videocodecid          7.00
INFO:   audiodatarate         160.00
INFO:   audiocodecid          10.00
INFO:   Server                Tengine
INFO:   videocodecreal        0.00
INFO:   fileSize              0.00
INFO:   audiosamplerate       48000.00
INFO:   audiosamplesize       16.00
INFO:   audiochannels         2.00
INFO:   stereo                TRUE
INFO:   2.1                   FALSE
INFO:   3.1                   FALSE
INFO:   4.0                   FALSE
INFO:   4.1                   FALSE
INFO:   5.1                   FALSE
INFO:   7.1                   FALSE
INFO:   encoder               obs-output module (libobs version 27.2.4)
815.438 kB / 38.55 sec^C
Caught signal: 2, cleaning up, just a second...
816.304 kB / 38.59 sec
DEBUG: RTMP_Read returned: 443
Download may be incomplete (downloaded about 0.00%), try resuming
DEBUG: Closing connection.

DEBUG: Invoking deleteStream

参考阅读

May 17 '22 15:05 nonocast

me me copied to clipboard

学习 rtmp 开发 (Part 4: flash and ams)

参考阅读

me
me copied to clipboard