amazon-kinesis-video-streams-producer-sdk-cpp
amazon-kinesis-video-streams-producer-sdk-cpp copied to clipboard
Published
20 hours ago •
awslabs
awslabs
[QUESTION] Streaming MP4 file data to Kinesis Video Streams results in Transferred a partial file error with CURL
I am using AWS KVS to stream a live camera feed. As a quick MVP I have opted to replicate the camera feed with a MP4 file which I manually send to the AWS KVS stream frame-by-frame (I use OpenCV to read the MP4 and convert it to frames) using the putFrame function. I am not seeing any errors initially from logs, but after a couple seconds after putFrameing all the video frames I am given an error of the form:
[video_streamer_node-1] WARN - curlCompleteSync(): [<stream-name>] curl perform failed for url https://s-ca658586.kinesisvideo.ap-southeast-2.amazonaws.com/putMedia with result Transferred a partial file: transfer closed with outstanding read data remaining
[video_streamer_node-1] WARN - curlCompleteSync(): [<stream-name>] HTTP Error 0 : Response: (null)
On the AWS KVS CloudWatch metrics side I am able to see spikes in PutMedia Active Connections, PutMedia Success, HLS and MPEG-DASH Requests, and HLS and MPEG-DASH Success. I am getting 100% success rates with PutMedia Success, but 0% success rates with HLS and MPEG-DASH Success.
Could anyone give me any ideas on what could be causing this error? Would the data I send in the frame object I pass to putFrame be non-ideal - I have read that you can stream non-video data as well (which would mean what data I send shouldn't really affect the streaming?). Does the MP4 video encoding have any affect? I have uploaded the full logs below.
Logs
[video_streamer_node-1] [INFO] [1720758264.804119277] [kinesis_video_streamer_node]: Initialising streamer node...
[video_streamer_node-1] [INFO] [1720758264.804229816] [kinesis_video_streamer_node]: Initialising AWS Kinesis SDK
[video_streamer_node-1] [INFO] [1720758264.879923571] [kinesis_video_streamer_node]: Initialising kinesis video producer...
[video_streamer_node-1] INFO - createKinesisVideoClient(): Creating Kinesis Video Client
[video_streamer_node-1] WARN - fixupClientInfo(): Connection timeout is invalid...setting to default
[video_streamer_node-1] WARN - fixupClientInfo(): Completion timeout is invalid...setting to default
[video_streamer_node-1] INFO - heapInitialize(): Initializing native heap with limit size 268435456, spill ratio 0% and flags 0x00000001
[video_streamer_node-1] INFO - heapInitialize(): Creating AIV heap.
[video_streamer_node-1] INFO - heapInitialize(): Heap is initialized OK
[video_streamer_node-1] DEBUG - getSecurityTokenHandler invoked
[video_streamer_node-1] DEBUG - Refreshing credentials. Force refreshing: 0 Now time is: 1720758264881859265 Expiration: 0
[video_streamer_node-1] INFO - createDeviceResultEvent(): Create device result event.
[video_streamer_node-1] DEBUG - clientReadyHandler invoked
[video_streamer_node-1] [INFO] [1720758264.882003494] [kinesis_video_streamer_node]: Initialising kinesis video stream...
[video_streamer_node-1] INFO - Creating Kinesis Video Stream <stream-name>
[video_streamer_node-1] INFO - createKinesisVideoStream(): Creating Kinesis Video Stream.
[video_streamer_node-1] INFO - logStreamInfo(): SDK version:
[video_streamer_node-1] DEBUG - logStreamInfo(): Kinesis Video Stream Info
[video_streamer_node-1] DEBUG - logStreamInfo(): Stream name: <stream-name>
[video_streamer_node-1] DEBUG - logStreamInfo(): Streaming type: STREAMING_TYPE_REALTIME
[video_streamer_node-1] DEBUG - logStreamInfo(): Content type: video/h264
[video_streamer_node-1] DEBUG - logStreamInfo(): Max latency (100ns): 0
[video_streamer_node-1] DEBUG - logStreamInfo(): Fragment duration (100ns): 20000000
[video_streamer_node-1] DEBUG - logStreamInfo(): Key frame fragmentation: Yes
[video_streamer_node-1] DEBUG - logStreamInfo(): Use frame timecodes: Yes
[video_streamer_node-1] DEBUG - logStreamInfo(): Absolute frame timecodes: Yes
[video_streamer_node-1] DEBUG - logStreamInfo(): Nal adaptation flags: 40
[video_streamer_node-1] DEBUG - logStreamInfo(): Average bandwidth (bps): 4194304
[video_streamer_node-1] DEBUG - logStreamInfo(): Framerate: 25
[video_streamer_node-1] DEBUG - logStreamInfo(): Buffer duration (100ns): 1200000000
[video_streamer_node-1] DEBUG - logStreamInfo(): Replay duration (100ns): 400000000
[video_streamer_node-1] DEBUG - logStreamInfo(): Connection Staleness duration (100ns): 300000000
[video_streamer_node-1] DEBUG - logStreamInfo(): Store Pressure Policy: 1
[video_streamer_node-1] DEBUG - logStreamInfo(): View Overflow Policy: 1
[video_streamer_node-1] DEBUG - logStreamInfo(): Allow stream creation: Yes
[video_streamer_node-1] DEBUG - logStreamInfo(): Segment UUID: NULL
[video_streamer_node-1] DEBUG - logStreamInfo(): Frame ordering mode: 0
[video_streamer_node-1] DEBUG - logStreamInfo(): Track list
[video_streamer_node-1] DEBUG - logStreamInfo(): Track id: 1
[video_streamer_node-1] DEBUG - logStreamInfo(): Track name: kinesis_video
[video_streamer_node-1] DEBUG - logStreamInfo(): Codec id: V_MPEG4/ISO/AVC
[video_streamer_node-1] DEBUG - logStreamInfo(): Track type: TRACK_INFO_TYPE_VIDEO
[video_streamer_node-1] DEBUG - logStreamInfo(): Track cpd: NULL
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: user-agent -> AWS-SDK-KVS/1.5.2 GCC/13.2.0 Linux/5.15.153.1-microsoft-standard-WSL2 x86_64 CPPSDK
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: host -> kinesisvideo.ap-southeast-2.amazonaws.com/describeStream
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: X-Amz-Date -> 20240712T042424Z
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-type -> application/json
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-length -> 27
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: Authorization -> AWS4-HMAC-SHA256 Credential=<credentials>/ap-southeast-2/kinesisvideo/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=<signature>
[video_streamer_node-1] INFO - writeHeaderCallback(): RequestId: 36605001-63ab-4233-ad0c-b2cd5811344c
[video_streamer_node-1] DEBUG - describeStreamCurlHandler(): [<stream-name>] DescribeStream API response: {"StreamInfo":{"CreationTime":1.720751492346E9,"DataRetentionInHours":24,"DeviceName":"Kinesis_Video_Device","IngestionConfiguration":null,"KmsKeyId":"arn:aws:kms:ap-southeast-2:210362323009:alias/aws/kinesisvideo","MediaType":"video/h264","Status":"ACTIVE","StreamARN":"arn:aws:kinesisvideo:ap-southeast-2:210362323009:stream/<stream-name>/1720751492346","StreamName":"<stream-name>","Version":"<version>"}}
[video_streamer_node-1] INFO - describeStreamResultEvent(): Describe stream result event.
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: user-agent -> AWS-SDK-KVS/1.5.2 GCC/13.2.0 Linux/5.15.153.1-microsoft-standard-WSL2 x86_64 CPPSDK
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: host -> kinesisvideo.ap-southeast-2.amazonaws.com/tagStream
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: X-Amz-Date -> 20240712T042425Z
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-type -> application/json
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-length -> 132
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: Authorization -> AWS4-HMAC-SHA256 Credential=<credentials>/ap-southeast-2/kinesisvideo/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=<signature>
[video_streamer_node-1] INFO - writeHeaderCallback(): RequestId: e8905042-9e0f-418f-ba60-b9915bb54ca6
[video_streamer_node-1] INFO - tagResourceResultEvent(): Tag resource result event.
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: user-agent -> AWS-SDK-KVS/1.5.2 GCC/13.2.0 Linux/5.15.153.1-microsoft-standard-WSL2 x86_64 CPPSDK
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: host -> kinesisvideo.ap-southeast-2.amazonaws.com/getDataEndpoint
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: X-Amz-Date -> 20240712T042425Z
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-type -> application/json
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-length -> 52
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: Authorization -> AWS4-HMAC-SHA256 Credential=<credentials>/ap-southeast-2/kinesisvideo/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=<signature>
[video_streamer_node-1] INFO - writeHeaderCallback(): RequestId: 77971e4e-a881-4344-bbb3-4b2871fcd48e
[video_streamer_node-1] DEBUG - getStreamingEndpointCurlHandler(): [<stream-name>] GetStreamingEndpoint API response: {"DataEndpoint":"https://s-ca658586.kinesisvideo.ap-southeast-2.amazonaws.com"}
[video_streamer_node-1] INFO - getStreamingEndpointResultEvent(): Get streaming endpoint result event.
[video_streamer_node-1] DEBUG - getStreamingTokenHandler invoked
[video_streamer_node-1] DEBUG - Refreshing credentials. Force refreshing: 1 Now time is: 1720758265390606953 Expiration: 1720758564
[video_streamer_node-1] INFO - getStreamingTokenResultEvent(): Get streaming token result event.
[video_streamer_node-1] DEBUG - streamReadyHandler invoked
[video_streamer_node-1] [INFO] [1720758265.390754832] [kinesis_video_streamer_node]: Initialising image transport callback...
<stream some data with putFrame>
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: user-agent -> AWS-SDK-KVS/1.5.2 GCC/13.2.0 Linux/5.15.153.1-microsoft-standard-WSL2 x86_64 CPPSDK
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: x-amzn-stream-name -> <stream-name>
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: x-amzn-producer-start-timestamp -> 1720758394.270
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: x-amzn-fragment-acknowledgment-required -> 1
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: x-amzn-fragment-timecode-type -> ABSOLUTE
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: transfer-encoding -> chunked
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: connection -> keep-alive
[video_streamer_node-1] INFO - putStreamResultEvent(): Put stream result event. New upload handle 0
[video_streamer_node-1] WARN - notifyDataAvailable(): [<stream-name>] Failed to un-pause curl with error: 43. Curl object 0x7f12fc010b50
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: host -> s-ca658586.kinesisvideo.ap-southeast-2.amazonaws.com/putMedia
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: X-Amz-Date -> 20240712T042634Z
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: content-type -> application/json
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: Authorization -> AWS4-HMAC-SHA256 Credential=<credentials>/ap-southeast-2/kinesisvideo/aws4_request, SignedHeaders=connection;host;transfer-encoding;user-agent;x-amz-date;x-amzn-fragment-acknowledgment-required;x-amzn-fragment-timecode-type;x-amzn-producer-start-timestamp;x-amzn-stream-name, Signature=<signature>
<stream some data with putFrame>
[video_streamer_node-1] INFO - writeHeaderCallback(): RequestId: c7cbe5da-833c-2c79-98c1-5766cd414979
<stream more data with putFrame>
[video_streamer_node-1] DEBUG - postReadCallback(): [AbiCam] Pausing CURL read for upload handle: 0
<stream more data with putFrame>
[video_streamer_node-1] WARN - curlCompleteSync(): [<stream-name>] curl perform failed for url https://s-ca658586.kinesisvideo.ap-southeast-2.amazonaws.com/putMedia with result Transferred a partial file: transfer closed with outstanding read data remaining
[video_streamer_node-1] WARN - curlCompleteSync(): [<stream-name>] HTTP Error 0 : Response: (null)
[video_streamer_node-1] Request URL: https://s-ca658586.kinesisvideo.ap-southeast-2.amazonaws.com/putMedia
[video_streamer_node-1] Request Headers:
[video_streamer_node-1] Authorization: AWS4-HMAC-SHA256 Credential=<credentials>/20240712/ap-southeast-2/kinesisvideo/aws4_request, SignedHeaders=connection;host;transfer-encoding;user-agent;x-amz-date;x-amzn-fragment-acknowledgment-required;x-amzn-fragment-timecode-type;x-amzn-producer-start-timestamp;x-amzn-stream-name, Signature=<signature>
[video_streamer_node-1] DEBUG - putStreamCurlHandler(): Network thread for Kinesis Video stream: <stream-name> with upload handle: 0 exited. http status: 0
[video_streamer_node-1] WARN - putStreamCurlHandler(): [<stream-name>] Stream with streamHandle 20284784 uploadHandle 0 has exited without triggering end-of-stream. Service call result: 10006
[video_streamer_node-1] INFO - kinesisVideoStreamTerminated(): Stream 0x1358570 terminated upload handle 0 with service call result 10006.
[video_streamer_node-1] DEBUG - defaultStreamStateTransitionHook(): Stream state machine retry count: 0
[video_streamer_node-1] DEBUG - defaultStreamStateTransitionHook():
[video_streamer_node-1] KinesisVideoStream base result is [10006]. Executing KVS retry handler of retry strategy type [1]
[video_streamer_node-1] DEBUG - defaultStreamStateTransitionHook(): Stream state machine retry count: 1
[video_streamer_node-1] DEBUG - defaultStreamStateTransitionHook():
[video_streamer_node-1] KinesisVideoStream base result is [10006]. Executing KVS retry handler of retry strategy type [1]
[video_streamer_node-1] DEBUG - setRequestHeader(): Appending header to request: user-agent -> AWS-SDK-KVS/1.5.2 GCC/13.2.0 Linux/5.15.153.1-microsoft-standard-WSL2 x86_64 CPPSD
