Mobile-SDK-Android-V5 Gimbal Freezing After Zoom Level

Environment: Platform: Android

**SDK Version: DJI SDK 5.9.1 Drone Model: Mavic 3 Enterprise

Firmware Version: Aircraft: 10.01.0503 Controller: 02.01.0517

Problem: When the zoom level exceeds 10x, the gimbal freezes and does not respond to further commands. The issue occurs consistently under the following conditions:

Smooth zoom (applySmoothZoom) is used to increment the zoom. The target zoom level (zoomSetpoint) is set above 10x.

Code Context: The following method is responsible for incrementally setting the zoom level:

private fun applySmoothZoom(setPoint: Double) {
    var targetZoom = currentZoom
    if (setPoint > targetZoom) {
        targetZoom += zoomStep
        if (targetZoom > setPoint)
            targetZoom = setPoint
    } else if (setPoint < targetZoom) {
        targetZoom -= zoomStep
        if (targetZoom < setPoint)
            targetZoom = setPoint
    }
    targetZoom = targetZoom.coerceIn(MIN_ZOOM, MAX_ZOOM)
    Log.v(TAG, "Zoom ramp $targetZoom setpoint $zoomSetpoint")
    setZoom(targetZoom)
}

The setZoom method uses the KeyManager to update the zoom ratio value:


private fun setZoomRatioValue(zoomRatio: Double) {
    KeyManager.getInstance()
        .setValue(cameraZoomRatiosKey, zoomRatio, object : CompletionCallback {
            override fun onSuccess() {
                isSetZoomOk = true
                Log.i(TAG, "Zoom set successfully to $zoomRatio")
            }

            override fun onFailure(error: IDJIError) {
                isSetZoomOk = false
                Log.e(TAG, "Error setting zoom: ${error.description()}")
            }
        })
}

Steps to Reproduce: Initialize the DJI SDK and connect to the drone. Start a smooth zoom using the applySmoothZoom method with a target zoom level above 10x (e.g., 15x). Observe the gimbal behavior when the zoom level passes 10x.

Expected Behavior: The gimbal should continue to respond to zoom commands and reach the target zoom level smoothly.

Actual Behavior: The gimbal freezes when the zoom level exceeds 10x. Further commands to the gimbal or zoom controls do not have any effect.

Dec 18 '24 20:12 Cherobin

Agent comment from SHENRONG.LE in Zendesk ticket #124873:

Dear developer, Hello, thank you for contacting DJI Innovations.

I just used the example code to reproduce and found that there is no such problem. According to your reproduction process, as the multiplier increases, it will only become slower, but it will not completely stop jamming.

Thank you for your email. Wishing you a happy life! Best Regards, DJI Innovation SDK Technical Support

°°°

Dec 19 '24 08:12 dji-dev

Dear DJI Innovation SDK Technical Support,

Thank you for your response.

I would like to provide additional context regarding the issue we are facing. The problem occurs only during flight operations when the drone is actively processing image data alongside gimbal and zoom controls. Specifically, we are running image processing algorithms to detect objects within a bounding box. The gimbal and zoom are used to focus on this bounding box, which requires precise and smooth adjustments.

Here is a summary of the setup and behavior:

Concurrent Image Processing: While flying, the drone processes real-time image data, identifies objects, and computes a bounding box for the target. Zoom and Gimbal Coordination: We use the applySmoothZoom method to adjust the zoom incrementally, and the gimbal is directed to center the bounding box target using pitch and yaw adjustments (setGimbalPitch and djiRotateByAngleAsync methods).

Issue Description: -The issue arises when the zoom level exceeds > 20~ during this concurrent operation. -The gimbal freezes and becomes unresponsive to further commands. -The problem does not occur when the drone is stationary on the ground, even with image processing running.

Our workflow demands real-time adjustments to maintain the bounding box in the center of the frame, so smooth zoom and precise gimbal control are critical.

Could this issue be related to:

Firmware constraints under concurrent workloads during flight? Any limitations in the zoom or gimbal systems when operating beyond >20x zoom?

We appreciate your support and are willing to perform specific tests or provide logs to help diagnose the problem further.

Thank you for your time and assistance.

Here is the code:

// Setup Gimbal - Gimbal Mode Configuration
KeyManager.getInstance().setValue(
    KeyTools.createKey(GimbalKey.KeyGimbalMode),
    GimbalMode.YAW_FOLLOW,
    object : CommonCallbacks.CompletionCallback {
        override fun onSuccess() {
            continuation.resume(Result.success(Unit))
        }

        override fun onFailure(error: IDJIError) {
            val errorMessage = "Error setting Gimbal Mode: ${error.description()}"
            continuation.resumeWithException(GimbalException(errorMessage))
        }
    }
)

// Setup Gimbal - Pitch Smoothing Factor Configuration
KeyManager.getInstance().setValue(
    KeyTools.createKey(GimbalKey.KeyPitchSmoothingFactor),
    0,
    object : CommonCallbacks.CompletionCallback {
        override fun onSuccess() {
            continuation.resume(Result.success(Unit))
        }

        override fun onFailure(error: IDJIError) {
            val errorMessage = "Error setting Pitch Smoothing Factor: ${error.description()}"
            continuation.resumeWithException(GimbalException(errorMessage))
        }
    }
)

// Rotate Gimbal By Angle
private suspend fun djiRotateByAngleAsync(
    pitch: Float,
    yaw: Float,
    movementDuration: Long,
    waitTime: Long
): Result<Unit> {
    return suspendCancellableCoroutine { continuation ->
        GimbalKey.KeyRotateByAngle.create()
            .action(
                GimbalAngleRotation(
                    GimbalAngleRotationMode.ABSOLUTE_ANGLE,
                    pitch.toDouble(),
                    0.0,
                    yaw.toDouble(),
                    false,
                    true,
                    false,
                    movementDuration.toDouble(),
                    true,
                    waitTime.toInt()
                ),
                onSuccess = {
                    continuation.resume(Result.success(Unit))
                },
                onFailure = { error: IDJIError ->
                    val errorMessage = "Error rotating Gimbal by angle: ${error.description()}"
                    continuation.resumeWithException(GimbalException(errorMessage))
                }
            )
    }
}

Dec 19 '24 11:12 Cherobin

Agent comment from SHENRONG.LE in Zendesk ticket #124873:

Dear developers, Hello, thank you for contacting DJI Innovation.

Can you provide videos of sample code test reproductions? I'll show it to R&D.

Thank you for your email and have a nice life! Best Regards, Dji innovation SDK technical support

°°°

Dec 19 '24 11:12 dji-dev

Thank you for your prompt response and assistance.

I have uploaded a video demonstrating the issue for your review: https://drive.google.com/file/d/1MSxEGnGcDfj6MfJXsF2MZvqg3q_NyMdM/view?usp=sharing

Please note the abrupt jerking or "jolt" at the end of the video during the gimbal's movement. This behavior appears to coincide with the conditions we described earlier, where the zoom level exceeds >20x and the gimbal starts to exhibit inconsistent behavior.

We hope this video provides clarity for your R&D team, and we look forward to any insights or recommendations you might have to resolve this.

Thank you for your continued support!

Dec 19 '24 13:12 Cherobin

To provide further clarity, I have included an additional video demonstrating a different test scenario: https://drive.google.com/file/d/1_tcB1oCLOnDSWgQCqXHUAEILgv2BnW2i/view?usp=sharing

In this test, we focused on vertical (up and down) gimbal movements. Please note:

The gimbal movement only occurs when the stick is pressed. If the stick is not actively pressed, the gimbal does not respond. Since we control the drone via the VirtualStick, we’re curious if any internal mechanisms might be interfering after a certain zoom level.

For instance: Is there any process or restriction that gets enabled at a higher zoom (e.g., above 20x) that might block both the VirtualStick and gimbal from functioning? Could this be related to another concurrent process? Could virtual stabilization be causing conflicts at higher zoom levels?

We appreciate any insights or guidance you can provide regarding these observations.

Thank you for your continued support and assistance.

Dec 19 '24 13:12 Cherobin

Agent comment from SHENRONG.LE in Zendesk ticket #124873:

Dear developers, Hello, thank you for contacting DJI Innovation.

Please save the video to the following cloud storage link and note which email corresponds to each of the two videos: https://pan-sec.djicorp.com/s/Xp5iSWms8pmbQmG In fact, theoretically, the pan tilt zoom is not related to the zoom factor. You need to check what the parameters of the pan tilt zoom are when the zoom exceeds 20x?

Thank you for your email and have a nice life! Best Regards, Dji innovation SDK technical support

°°°

Dec 20 '24 02:12 dji-dev

Hello,

I have uploaded the requested videos to the provided link.

video 1) app_test.webm video 2) movement_test.webm

In the videos, you will be able to see the logged information related to pan, tilt parameters as the zoom factor exceeds 20x.

If you need anything else, please let me know.

Best Regards,

Dec 20 '24 03:12 Cherobin

Agent comment from SHENRONG.LE in Zendesk ticket #124873:

Dear developers, Hello, thank you for contacting DJI Innovation.

I have checked the video named "app_test" you submitted. At the end of the video, while zoom, the head seemed to be out of control. Please confirm if this is your fault problem. Can you submit the log to the R & D department for analysis according to the corresponding time of this video?

Please provide the following logs (different types of logs are saved in separate folders) -> Remote control log (must) + aircraft log (must) + MSDK(5.10.0 ↑) + CSDK(must) + PKU load log (must); The Web disk link:https://pan-sec.djicorp.com/s/Xp5iSWms8pmbQmG

In addition, you are required to complete the following information (required) 1.【MSDK version and Firmware version 】 : 2. 【Drone model and Firmware version】 : 3.【Detailed fault Symptom 】 : 4.【Specific time period when the fault occurs]】:

Thank you for your email and have a nice life! Best Regards, Dji innovation SDK technical support
使用Pilot 2完成日志上传.docx
提交日志模板.jpeg

°°°

Dec 20 '24 04:12 dji-dev

Dear Support Team,

I have reviewed the video titled "app_test" and would like to provide further clarification about the issue observed.

In the video, while the gimbal is moving, we are able to send information to the drone without issues. However, at a certain point, the drone unexpectedly stops accepting commands. This behavior is consistent across our tests. The drone suddenly ceases to respond to commands sent by the app and only resumes responding if we use the physical sticks. Once the sticks are released, the drone again stops receiving commands from both the gimbal and zoom functionalities.

Additionally, the final jolt seen in the video occurs without any input from our side. This seems to be an automatic or unintended reaction from the drone.

To help investigate the issue further, I will provide the corresponding logs for this video shortly. Please confirm if there is any additional information required for analysis.

SDK Version: DJI SDK 5.9.1 Drone Model: Mavic 3 Enterprise

Firmware Version: Aircraft: 10.01.0503 Controller: 02.01.0517

Dec 20 '24 11:12 Cherobin

Agent comment from SHENRONG.LE in Zendesk ticket #124873:

Dear developers, Hello, thank you for contacting DJI Innovation.

Ok, please provide the log according to the format described in my last email. Please select the corresponding time in the "app_test" video for the log

==》 I have checked the video named "app_test" you submitted. At the end of the video, while zoom, the head seemed to be out of control. Please confirm if this is your fault problem. Can you submit the log to the R & D department for analysis according to the corresponding time of this video?

Please provide the following logs (different types of logs are saved in separate folders) -> Remote control log (must) + aircraft log (must) + MSDK(5.10.0 ↑) + CSDK(must) + PKU load log (must); The Web disk link:https://pan-sec.djicorp.com/s/Xp5iSWms8pmbQmG

In addition, you are required to complete the following information (required) 1.【MSDK version and Firmware version 】 : 2. 【Drone model and Firmware version】 : 3.【Detailed fault Symptom 】 : 4.【Specific time period when the fault occurs]】:

Thank you for your email and have a nice life! Best Regards, Dji innovation SDK technical support

°°°

Dec 20 '24 11:12 dji-dev

Dear DJI Innovation Technical Support,

Thank you for your email and for looking into the app issue I reported in the "app_test" video.

I apologize for the delay, but I won't be able to access the logs and respond until after the holidays. I'll be back in the office on January 6th, 2025, and I can collect the requested logs (remote control, aircraft, MSDK, CSDK, and PKU load) at that time.

In the meantime, I'd appreciate some clarification on the following log types:

CSDK (must): Could you please explain what the CSDK log captures specifically? PKU load log (must): I'm not familiar with the PKU load log. Could you provide a brief description of its purpose? Understanding these logs will help me collect the most relevant information for your analysis.

Thank you for your patience and understanding. I look forward to providing the logs and additional details on January 6th.

Best regards,

Dec 26 '24 18:12 Cherobin

Agent comment from SHENRONG.LE in Zendesk ticket #124873:

Dear customer Hello, thank you for contacting DJI Innovation

csdk log is actually the library used by msdk, msdk research and development colleagues need to check whether you have the problem of using errors.

Load logging is needed because you are using the scaling process, so see if there is a problem with the camera.

These logs are necessary if you need us to check the problem about the "app_test" video. The logs required to be submitted within the failure time should be provided correctly. In addition, the failure occurred a long time ago, and you can only start to get the logs on January 6th. I think you need to revisit the issue and re-shoot the video, as various previous logs may have washed out with time or use.

Please provide the following logs (different types of logs are saved in separate folders) -> Remote control log (must) + aircraft log (must) + MSDK(5.10.0 ↑) + CSDK(must) + PKU load log (must); The Web disk link:https://pan-sec.djicorp.com/s/Xp5iSWms8pmbQmG (If you provide a new video that repeats, note the name of the new video corresponding to the fault log)

In addition, you are required to complete the following information (required) 1.【MSDK version and Firmware version 】 : 2. 【Drone model and Firmware version】 : 3.【Detailed fault Symptom 】 : 4.【Specific time period when the fault occurs]】:

如何导出MSDK V5的日志？

Thank you for your understanding and support, I wish you a happy life! Best Regards Dji innovation SDK technical support
image.png
使用Pilot 2完成日志上传.docx
提交日志文件夹规范模板.png

°°°

Dec 27 '24 06:12 dji-dev

Hello DJI Innovation Technical Support,

My name is Guilherme and I work with Cherobin, who opened this issue. Can you tell me which folders I need to access to obtain each of the logs mentioned? Thank you.

Jan 06 '25 12:01 GuiABP

Dear DJI Support,

Thank you for your response and detailed instructions.

I have uploaded the requested logs. Please note that the logs correspond to two videos.

SDK Version: DJI SDK 5.9.1 Drone Model: Mavic 3 Enterprise

Firmware Version: Aircraft: 10.01.0503 Controller: 02.01.0517

Should you require any additional information or further details, please let me know. I look forward to your feedback.

Best regards,

Jan 06 '25 19:01 Cherobin

Agent comment from SHENRONG.LE in Zendesk ticket #124873:

Dear customer Hello, thank you for contacting DJI Innovation

Which video does the log content you submitted in the screenshot correspond to? What's the fault? What time does it correspond to? The information you submitted is not complete, msdk related logs are also added. Can you add the necessary information below? If the information is missing, the R&D department will not analyze it, because the huge log does not know when to start the analysis.

Please provide the following logs (different types of logs are saved in separate folders) -> Remote control log (must) + aircraft log (must) + MSDK(5.10.0 ↑) + CSDK(must) + PKU load log (must); The Web disk link:https://pan-sec.djicorp.com/s/Xp5iSWms8pmbQmG (If you provide a new video that repeats, note the name of the new video corresponding to the fault log)

In addition, you are required to complete the following information (required) 1.【MSDK version and Firmware version 】 : 2. 【Drone model and Firmware version】 : 3.【Detailed fault Symptom 】 : 4.【Specific time period when the fault occurs]】:

如何导出MSDK V5的日志？

Thank you for your understanding and support, I wish you a happy life! Best Regards Dji innovation SDK technical support
image.png

°°°

Jan 07 '25 04:01 dji-dev

Dear DJI Support,

Thank you for your feedback. Below are the necessary details to assist the R&D team in their analysis:

Video and Log Details

Video Name: app_test Log Timestamp: December 9th, 10:24:29

MSDK Version and Firmware Version: MSDK 5.9.1, Aircraft 10.01.0503, Controller 02.01.0517. Drone Model and Firmware Version: Mavic 3 Enterprise, 10.01.0503.

Fault Observed The drone stops receiving zoom commands unexpectedly. Shortly after, the gimbal freezes and abruptly moves to the bottom-right position without any input from our side. The logs are organized, and the provided timestamp (10:24:29 on December 9th) corresponds to the exact moment in the "app_test" video where the issue occurs.

Fault Context This fault occurs only during flight operations when concurrent zoom and gimbal commands are executed. It does not occur when the drone is stationary on the ground, even with similar commands.

Additional Observations The abrupt gimbal movement appears to be an automatic reaction, unrelated to any external inputs. If further information or specific tests are required, please let me know. I appreciate your time and support in resolving this matter.

Best regards,

Jan 07 '25 12:01 Cherobin

Agent comment from SHENRONG.LE in Zendesk ticket #124873:

Dear customer Hello, thank you for contacting DJI Innovation

I have submitted the information you provided, and I will contact you if I need to supplement the log or draw conclusions in the future.

Thank you for your understanding and support, I wish you a happy life! Best Regards Dji innovation SDK technical support

°°°

Jan 08 '25 09:01 dji-dev

Agent comment from SHENRONG.LE in Zendesk ticket #124873:

Dear customer Hello, thank you for contacting DJI Innovation

R & D feedback: M3 needs to develop its own image stabilization at high magnifications, at this time, MSDK program control PTZ is not supported, and can only be controlled by remote control.

Thank you for your understanding and support, I wish you a happy life! Best Regards Dji innovation SDK technical support

°°°

Jan 22 '25 02:01 dji-dev

Dear DJI Innovation Technical Support,

Thank you for the R&D team's feedback ...

This limitation presents a significant challenge for our application, as we've invested considerable resources in developing automated zoom and gimbal control features.

We have some important questions:

Is there any way to disable this automatic image stabilization feature when operating at high magnifications? This would allow us to maintain programmatic control via MSDK.
At what specific zoom level does this limitation begin? Understanding this threshold would help us adapt our application accordingly.
This limitation isn't documented in the SDK documentation. Could you please provide any additional technical documentation about this behavior?
Are there any alternative approaches or workarounds that would allow us to maintain programmatic control of the gimbal at high magnifications while ensuring stable image quality?

We appreciate your continued support in helping us find a solution that meets our requirements while working within the system's capabilities.

Best regards

Jan 23 '25 12:01 Cherobin

Agent comment from SHENRONG.LE in Zendesk ticket #124873:

Dear customer Hello, thank you for contacting DJI Innovation

here is currently no way to disable this automatic anti-shake feature

I need further confirmation on the specific zoom level and will let you know.

Sorry, MSDK currently does not have the relevant interface to deal with this problem, the customization of the current solution has gone beyond the scope of technical support, usually by the developer to find a solution to stabilize the head and image quality.

Thank you for your understanding and support, I wish you a happy life! Best Regards Dji innovation SDK technical support

°°°

Jan 24 '25 10:01 dji-dev

Gimbal Freezing After Zoom Level > ~10