How to change front camera resolution?

[zweistein]

I'm using the Ar.Drone 2.0. with Firmware: 2.2.6. The front camera can be used with resolution : 360p (640x360) or 720p (1280*720). The default setup is the first one, 360p. I would like to use the 720p. How can I change it? Is there a parameter in the launch file?

Is there any update in camera togle issue?

[mani-monaj]

The front camera resolution can not be changed on the fly, because the image memory buffer size is now fixed. The dynamic memory allocation should fix the problem though. Here is how 720p video stream can be enabled on the current code. The glitch is, the driver should be recompiled.

1. In file src/video.h:

Change MAX_STREAM_WIDTH and MAX_STREAM_HEIGHT to 1280 and 720 respectively. Also, change D2_STREAM_WIDTH and D2_STREAM_HEIGHT (lines 32,33) to 1280 and 720 respectively.

2. In file src/ardrone_sdk.cpp

Change ardrone_application_default_config.video_codec = H264_360P_CODEC (line 46) to H264_720P_CODEC

3. Recompile with rosmake --pre-clean.

Please note that 1) The change in default video_codec may be overwritten by rosparam parameter of the same name. 2) This is a hack!

[truongngoctuan]

I followed your instruction, but there are many errors on decoding video stream. I don't know how to fix it

Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a I Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a P Frame Decoding failed for a I Frame Decoding failed for a P Frame Decoding failed for a P Frame I tried with increasing the bitrate... but useless

[mani-monaj]

Can you please paste the git diff of your changes?

[krisdestruction]

The comment in /src/ardrone_drive.cpp says that there's no 720p support for now :/ is this true?

[mani-monaj]

@krisdestruction As I described in this thread, you can try 720p by changing the source code.

[krisdestruction]

I tried this and it seems to be spamming as noted above: Decoding failed for a P Frame

[boris-il-forte]

same problem for me...

[thomas-bamford]

To increase video resolution to 720p, I tried the following steps as instructed above:

1. In file include/ardrone_autonomy/video.h:

Change MAX_STREAM_WIDTH and MAX_STREAM_HEIGHT (lines 33, 34) to 1280 and 720 respectively. Also, change D2_STREAM_WIDTH and D2_STREAM_HEIGHT (lines 58, 59) to 1280 and 720 respectively.

2. In file src/ardrone_sdk.cpp:

Change ardrone_application_default_config.video_codec = H264_360P_CODEC (line 71) to H264_720P_CODEC

3. Recompile with catkin_make.

The results after compilation and launching ardrone.launch with an AR.Drone 2.0 connected on wifi are shown below with an image saved from the /ardrone/image_raw rostopic.

...
Starting thread video_stage
Starting thread video_recorder

video stage thread initialisation

Video multisocket : init 2 sockets
Video multisocket : connecting socket 0 on port 0 UDP
Video recorder thread initialisation
Video multisocket : connecting socket 1 on port 5555 TCP
...
PA : MEMORY SPACE ALLOWED : 40 MB
...
Sending default CAT_APPLI settings
Sending default CAT_USER settings
Sending default CAT_SESSION settings
Decoding failed for a I Frame
Decoding failed for a P Frame
...
Decoding failed for a P Frame
...
Decoding failed for a P Frame
...
Decoding failed for a P Frame
...
Decoding failed for a P Frame
...
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
...
Decoding failed for a I Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
[ INFO] [1460046443.794771981]:   SEND: CAT_COMMON/outdoor = 1.000000 (DEFAULT = 0.000000)
[ INFO] [1460046443.794889905]:   SEND: CAT_COMMON/enemy_colors = 3.000000 (DEFAULT = 1.000000)
[ INFO] [1460046443.794940702]:   SEND: CAT_APPLI/navdata_options = 268435456.000000 (DEFAULT = 65537.000000)
[ INFO] [1460046443.795005714]:   SEND: CAT_USER/euler_angle_max = 0.210000 (DEFAULT = 0.209440)
[ INFO] [1460046443.795087181]:   SEND: CAT_USER/control_yaw = 1.750000 (DEFAULT = 1.745329)
[ INFO] [1460046443.795192413]:   SEND: CAT_SESSION/video_codec = 131.000000 (DEFAULT = 32.000000)
[ INFO] [1460046443.795296483]:   SEND: CAT_APPLI/bitrate = 4000.000000 (DEFAULT = 1000.000000)
[ INFO] [1460046443.795445789]:   SEND: CAT_SESSION/max_bitrate = 4000.000000 (DEFAULT = 1000.000000)
[ INFO] [1460046443.795539563]:   SEND: CAT_SESSION/detect_type = 10.000000 (DEFAULT = 3.000000)
[ INFO] [1460046443.795625731]:   SEND: CAT_SESSION/detections_select_h = 32.000000 (DEFAULT = 0.000000)
[ INFO] [1460046443.795717897]:   SEND: CAT_SESSION/detections_select_v_hsync = 128.000000 (DEFAULT = 0.000000)
Decoding failed for a P Frame
Decoding failed for a P Frame
[ INFO] [1460046443.835659236]: Successfully connected to 'My ARDrone' (AR-Drone 2.0 - Firmware: 2.4.8) - Battery(%): 86
[ INFO] [1460046443.835779725]: Navdata Publish Settings:
[ INFO] [1460046443.835858745]:     Legacy Navdata Mode: On
[ INFO] [1460046443.835951245]:     ROS Loop Rate: 50 Hz
[ INFO] [1460046443.836020931]:     Realtime Navdata Publish: Off
[ INFO] [1460046443.836095017]:     Realtime Video Publish: Off
[ INFO] [1460046443.836165093]:     Drone Navdata Send Speed: 200Hz (navdata_demo=0)
Decoding failed for a P Frame
Decoding failed for a P Frame
...
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a I Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
Decoding failed for a P Frame
...

[mani-monaj]

@thomas-bamford Oh, I missed this update. Did you manage to fix the issue?

[Fawzikhattar]

Has anyone managed to fix this issue?

[thomas-bamford]

Sorry, was not able to fix this issue. I upgraded my hardware to the Parrot Bebop 2 and it streams at 720p.

[Fawzikhattar]

I was able to find a hack to it (although sometimes the H264 decoder needs some time to start decoding properly).

In addition to the steps above add the following in the video_stage_tcp.c file in devel/src/ardronelib/ARDroneLib/Soft/Lib/ardrone_tool/Video

function declaration:

static void changeSPS(uint8_t * buf)
{ int i=0;
  int dontbreak=1;
  int numberOfBytesToMove=20;
   while(dontbreak)
      {
        if(buf[i]==0x67)
        { dontbreak=0;
         memmove (buf+i+1, buf+i,numberOfBytesToMove);
     
        }
   i=i+1;
       }
}

inside function video_stage_tcp_transform add this after the first two lines:

if(frameIsIFrame(in->buffers[in->indexBuffer])) changeSPS(in->buffers[in->indexBuffer]);

That did it for me although sometimes the decoder takes some time to work properly.

[mani-monaj]

@Fawzikhattar Thank you for the update. Does this change break regular (non-720p) streaming in your tests?

[Fawzikhattar]

I didn't test if it breaks it or n as im only intrested in the 720p video stream. However a simple if condition on the codec used can avoid this problem in case it exist. Also note that a better more robust way to solve this problem might exist (I dont have too much experience in decoding videos).

[shlapion]

Hello, I have tried @Fawzikhattar method and it works. As mentioned the decoder needs some time to start properly decoding. In my test took around (45s-1m). After that the video is stable. I also recommend a re-calibration of the camera to have it work properly.

Personally, I have little to no experience in decoding videos, so I also can't recommend a better approach to solving this problem.

@mani-monaj do you think this issue could be addressed in the near future?

@Fawzikhattar many thanks for your solution !

[chinnapong]

The problem comes from the nature of H.264 codec at 720P resolution. The decoder cannot start unless the SPS and PPS frames are properly arrived.

The ARDone sends SPS and PPS frame regularly so, normally, there won't be a problem. However, the actual problem lies in the library that AR_Drone autonomy used for decoding the video, which is the FFMPEG.

We can say that it's the bug of FFMPEG library (or there may be some reason why the programmer of FFMPEG leaves this bug inside). AR-Drone (and most of the H.264 streaming application) sends the SPS and PPS in the first 20 bytes within every IDR frame. The location of these SPS and PPS are in the first 20 bytes of the frame. However, FFMPEG expects that the first byte of the IDR frame should always be 0, and the SPS and PPS should starts from the second byte.

You can see that the function "ChangeSPS" as provided by Fawzi works by shifting the first 20 bytes of the video frame to the right by one byte. Thus, FFMPEG can finally receive the SPS, PPS properly for decoding the video.

The resason why sometimes it takes 45-1 minutes before the video could be start is because it's the period where the drone does not send the IDR frame yet. We cannot do any much about it because the frequency of sending IDR frame is hard-coded and cannot be controlled. ><"

I hope that my explanation helps.

[shlapion]

Hello @chinnapong ,

thank you for the detailed explanation. Helped me to understand the issue better: ).

[Kikkomann]

Is it only me that can't make it work? I get the same result as @thomas-bamford had. I have followed @mani-monaj and @Fawzikhattar guides. Are there other some implicit details that I should know (I am new to ROS and c/c++)?

[Fawzikhattar]

Did you recompile and clean the project after all the changes? if yes how?

[Kikkomann]

I have tried rosmake --pre-clean in the ardron_autonomy folder and make in the drone root folder (containing ardrone_autonomy and my own drone project).

[cyberphantom]

I've tried the steps above. I got 4 x 640x360 frames inside a 1280x720 (as the same in the comment of @thomas-bamford above). Please update the correct steps to get this resolution change works, or advise if this is not a practical solution.

Thanks,

[cyberphantom]

It worked after rosmake --pre-clean

[TengFeiHan0]

@cyberphantom if(frameIsIFrame(in->buffers[in->indexBuffer])) changeSPS(in->buffers[in->indexBuffer]); I can't understand what did he mean "inside function video_stage_tcp_transform add this after the first two lines:" could you tell me the specific location where these codes should add?

[cyberphantom]

Go to "video_stage_tcp.c" file inside "devel/src/ardronelib/ARDroneLib/Soft/Lib/ardrone_tool/Video" directory (inside your catkin for instance)

Go to "C_RESULT video_stage_tcp_transform(video_stage_tcp_config_t *cfg, vp_api_io_data_t *in, vp_api_io_data_t *out)" function

it has only two lines and a comment.

Add "if(frameIsIFrame(in->buffers[in->indexBuffer])) changeSPS(in->buffers[in->indexBuffer]);" just inside it before the comment

[joshi-bharat]

Hello @cyberphantom @Fawzikhattar,

I am getting buffer full issue and also the ardrone_driver node crashes after like 2-3 minutes. Did you get the same problem? If yes, how did you solve the problem?

[cyberphantom]

No actually, I remember running it for a long time without any issues.

[joshi-bharat]

@mani-monaj I am getting buffer full issue and it's coming from video_stage_tcp.c file in devel/src/ardronelib/ARDroneLib/Soft/Lib/ardrone_tool/Video, probably due to high resolution images. I tried to increase the buffer size at line 62 ( BUFFER_SIZE=cfg->maxPFramesPerIFrame * cfg->frameMeanSize ) by multiplying by 4. But it's giving me segmentation fault. I am not sure what exactly the buffer size should be. Can you look into this, please?