Mavic 3E stream ffmpeg

[sehadnassim]

Is it possible using the PSDK to retrieve the video stream from the DJI camera and play it on the platform using fflpay or stream it to an RTMP server using ffmpeg ?

[dji-dev]

Agent comment from Leon in Zendesk ticket #119296:

Hello, you can use PSDK to obtain the video stream data of the drone. They are video streams in H264 format. You can push them twice or do other processing on them.

°°°

[sehadnassim]

Any example of how to play the stream using ffplay. I currently have the following code but it says address already in use.

Log mentions : 8.079][channel]-[Debug]-[DjiStreamChannelUdp_RecvDataThread:248) Receive udp data from 192.168.107.1.30081 -> 192.168.90.2.30081, protocol: raw_data [8.083][channel]-[Debug]-[DjiStreamChannelUdp_RecvDataThread:248) Receive udp data from 192.168.107.1.30082 -> 192.168.90.2.30082, protocol: raw_data [8.088][channel]-[Debug]-[DjiStreamChannelUdp_RecvDataThread:248) Receive udp data from 192.168.107.1.30083 -> 192.168.90.2.30083, protocol: raw_data [8.093][channel]-[Debug]-[DjiStreamChannelUdp_RecvDataThread:248) Receive udp data from 192.168.107.1.30084 -> 192.168.90.2.30084, protocol: raw_data [8.098][channel]-[Debug]-[DjiStreamChannelUdp_RecvDataThread:248) Receive udp data from 192.168.107.1.30085 -> 192.168.90.2.30085, protocol: raw_data [8.102][linker]-[Debug]-[DjiCommand_RegRecvCmdHandler:674) Reg command handle list success, index:10

#include <liveview/test_liveview_entry.hpp>
#include <perception/test_perception_entry.hpp>
#include <flight_control/test_flight_control.h>
#include <gimbal/test_gimbal_entry.hpp>
#include <hms/test_hms.h>
#include <waypoint_v2/test_waypoint_v2.h>
#include <waypoint_v3/test_waypoint_v3.h>
#include <gimbal_manager/test_gimbal_manager.h>
#include "application.hpp"
#include "fc_subscription/test_fc_subscription.h"
#include <gimbal_emu/test_payload_gimbal_emu.h>
#include <camera_emu/test_payload_cam_emu_media.h>
#include <camera_emu/test_payload_cam_emu_base.h>
#include <dji_logger.h>
#include "widget/test_widget.h"
#include "widget/test_widget_speaker.h"
#include <power_management/test_power_management.h>
#include "data_transmission/test_data_transmission.h"
#include <camera_manager/test_camera_manager.h>
#include "camera_manager/test_camera_manager_entry.h"
#include "dji_liveview.h" // Ensure this includes the necessary types and functions

#include <iostream>
#include <cstdio>
#include <cstdlib>
#include <unistd.h>
#include <string>
#include <thread>
#include <chrono>
#include <sys/types.h>  // for socket types
#include <sys/socket.h> // for socket(), connect(), sendto()
#include <netinet/in.h> // for sockaddr_in
#include <arpa/inet.h>  // for inet_addr()
#include <unistd.h>     // for close()

static void StartFFplayStream(const std::string& streamUrl);
static void DjiTest_FpvCameraStreamCallback(E_DjiLiveViewCameraPosition position, const uint8_t* data, uint32_t dataSize);

int main(int argc, char **argv)
{
    Application application(argc, argv);
    T_DjiReturnCode returnCode;
    returnCode = DjiLiveview_Init();
    if (returnCode != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
        std::cerr << "Liveview init failed, error code: 0x" << std::hex << returnCode << std::endl;
        return returnCode;
    }
    returnCode = DjiLiveview_StartH264Stream(DJI_LIVEVIEW_CAMERA_POSITION_FPV, 
                                             DJI_LIVEVIEW_CAMERA_SOURCE_DEFAULT,
                                             DjiTest_FpvCameraStreamCallback);
    if (returnCode != DJI_ERROR_SYSTEM_MODULE_CODE_SUCCESS) {
        std::cerr << "Request h264 of FPV failed, error code: 0x" << std::hex << returnCode << std::endl;
        return returnCode;
    }


    StartFFplayStream("udp://192.168.90.2:30082"); // Modify the address as needed


    while (true) {

        std::this_thread::sleep_for(std::chrono::seconds(1));
    }

    // Stop the stream and deinitialize live view before exiting
    DjiLiveview_StopH264Stream(DJI_LIVEVIEW_CAMERA_POSITION_FPV, DJI_LIVEVIEW_CAMERA_SOURCE_DEFAULT);
    DjiLiveview_Deinit();
    return 0;
}

static void DjiTest_FpvCameraStreamCallback(E_DjiLiveViewCameraPosition position, const uint8_t* data, uint32_t dataSize) {
    // Write data to a UDP socket to stream to ffplay
    int sock = socket(AF_INET, SOCK_DGRAM, 0);
    if (sock < 0) {
        std::cerr << "Failed to create socket!" << std::endl;
        return;
    }

    struct sockaddr_in destAddr;
    destAddr.sin_family = AF_INET;
    destAddr.sin_port = htons(30082); // Port for ffplay
    destAddr.sin_addr.s_addr = inet_addr("127.0.0.1");

    ssize_t sentBytes = sendto(sock, data, dataSize, 0, (struct sockaddr*)&destAddr, sizeof(destAddr));
    if (sentBytes < 0) {
        std::cerr << "Failed to send data!" << std::endl;
    }

    close(sock); // Close the socket
}

static void StartFFplayStream(const std::string& streamUrl) {
    std::string command = "export XDG_RUNTIME_DIR=/run/user/" + std::to_string(getuid()) + " && ffplay " + streamUrl + " &";
    system(command.c_str()); // Launch ffplay as a subprocess
}

[dji-dev]

Agent comment from Leon in Zendesk ticket #119296:

Hello, if you want to see the video stream directly on the terminal, you can refer to the processing in C++sample: payload-sdk\samples\sample_c++\module_sample\liveview\test_liveview.cpp

°°°

[sehadnassim]

The example code for live display does not do anything in MAVIC 3E :

[dji-dev]

Agent comment from Leon in Zendesk ticket #119296:

Hello, you can check your ffmpeg and opencv versions. We have officially recommended versions. In addition, you need to confirm whether the libusb component is successfully installed. This component will also affect the normal use of this function. Finally, it is recommended that you use the code of the manifold2 project. The codes of other projects may not be updated in time. In order to avoid these effects, it is recommended that you use the code of this project. https://developer.dji.com/doc/payload-sdk-tutorial/en/quick-start/config-develop-environment.html

°°°