Jpeg files written from captured frames are corrupted

[moriyas1984]

Hi.

I captured some frames and saved it to files. And I copied the files to local machine and tried to open it by photo viewer, but it said that "The format is not supported.". By using ImageMagick's imdisplay command, I can see the image with a warning dialog says that some extraneous bytes before marker 0xdxx.

The output of lsusb command is as follows.

$ lsusb
Bus 001 Device 005: ID 046d:0825 Logitech, Inc. Webcam C270
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. SMSC9512/9514 Fast Ethernet Adapter
Bus 001 Device 002: ID 0424:9514 Standard Microsystems Corp. SMC9514 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

My source is as follows.

let mut camera = Camera::new("/dev/video0").unwrap();
camera.start(&Config {
    interval: (1, 30),
    resolution: (640, 480),
    format: b"MJPG",
    ..Default::default()
}).expect("Failed to start the camera device.")

for i in 0..10 {
    let frame = camera.capture().unwrap();
    let mut file = File::create(&format!( "assets/frame-{}.jpg", i )).unwrap();
    file.write_all(&frame).unwrap();
}

Binary data of one of the files are as follows.

00000000: ff d8 ff e0 00 21 41 56 49 31 00 01 01 01 00 78 |   .....!AVI1.....x 
00000010: 00 78 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |   .x..............
00000020: 00 00 00 00 00 ff db 00 43 00 08 05 06 07 06 05 |   ........C.......
00000030: 08 07 06 07 09 08 08 09 0c 14 0d 0c 0b 0b 0c 18 |   ................
00000040: 11 12 0e 14 1d 19 1e 1e 1c 19 1c 1f 20 24 2e 27 |   ............ $.'
00000050: 20 22 2b 22 1b 1c 28 34 28 2b 2f 31 33 34 33 1f |    "+"..(4(+/1343.
00000060: 26 38 3c 38 32 3c 2e 32 33 31 ff db 00 43 01 08 |   &8<82<.231...C..
00000070: 09 09 0c 0a 0c 17 0d 0d 17 31 21 1c 21 31 31 31 |   .........1!.!111
00000080: 31 31 31 31 31 31 31 31 31 31 31 31 31 31 31 31 |   1111111111111111
00000090: 31 31 31 31 31 31 31 31 31 31 31 31 31 31 31 31 |   1111111111111111
000000a0: 31 31 31 31 31 31 31 31 31 31 31 31 31 31 31 ff |   111111111111111.
...(omit)...
000064e0: 15 ec 63 e5 90 6d 63 ef da b9 9a 51 77 43 62 a9 |   ..c..mc....QwCb.
000064f0: c7 34                                           |   .4

Jpeg files start at ffd8 and end at ffd9, so I think that just a part of the captured data is written to the file. (Additionally, it might be strange that a number of 00 bytes and 31 bytes are repeated...)

What should I do about it? Have I missed any dependencies or procedures?

[leo60228]

All JPEG files are valid MJPEG frames, but the other way around isn't true. Most cameras will give valid JPEG files when asked to output MJPEG frames, but this isn't true for all cameras. https://lightbits.github.io/v4l2_huffman/ contains simple C code to convert them.

[leo60228]

#include <string.h>
#include <stdint.h>
uint32_t mjpg_to_jpg(uint8_t *mjpg,
                     uint32_t mjpg_size,
                     uint8_t *jpg)
{
    static uint8_t huffman[] = { 0xff, 0xc4, 0x01, ... , 0xfa };

    uint32_t i = 0;
    while (!(mjpg[i] == 0xff && mjpg[i+1] == 0xc0))
        i++;

    memcpy(jpg,                   mjpg,    i);
    memcpy(jpg+i,                 huffman, sizeof(huffman));
    memcpy(jpg+i+sizeof(huffman), mjpg+i,  mjpg_size-i);

    uint32_t jpg_size = mjpg_size+sizeof(huffman);
    return jpg_size;
}

[leo60228]

If you can't read C, that puts a (truncated in that snippet) array before the first occurrence of 0xffc0.

[leo60228]

The full array is at https://github.com/lightbits/usbcam/blob/master/mjpg_to_jpg.cpp#L28-L64.

[leo60228]

Also note that libv4l only guarantees RGB24, BGR24, YUV420, and YVU420. For best compatibility, you should use one of those four and convert it to jpeg using image-rs.

Untested:

use image::RgbImage;

let mut camera = rscam::new("/dev/video0").unwrap();

camera.start(&rscam::Config {
    interval: (1, 30),      // 30 fps.
    resolution: (1280, 720),
    format: b"RGB3",
    ..Default::default()
}).unwrap();

for i in 0..10 {
    let frame = camera.capture().unwrap();
    let image = 
        RgbImage::from_vec(frame.resolution.0, frame.resolution.1, Vec::from(&*frame)).unwrap();
    image.save(&format!("frame-{}.jpg", i)).unwrap();
}

[TheSalarKhan]

Hi @leo60228 I am having a very similar problem. I captured some frames from a tcp stream and they seem corrupted too. I have not been able to identify the manufacturer of the camera but all I know is that it has AV1 line.

I tried to follow your method by adding huffman tables to the header but it seemed to have no affect. So I tried looking at the image structure using exiv2 and this is what it outputs

$ exiv2 -pS image.jpg 
STRUCTURE OF JPEG FILE: image.jpg
 address | marker       |  length | data
       0 | 0xffd8 SOI  
       2 | 0xffe0 APP0  |      33 | AVI1.....x.x....................
      37 | 0xffdb DQT   |      67 
     106 | 0xffdb DQT   |      67 
     175 | 0xffc4 DHT   |      31 
     208 | 0xffc4 DHT   |     181 
     391 | 0xffc4 DHT   |      31 
     424 | 0xffc4 DHT   |     181 
     607 | 0xffdd DRI   |       4 
     613 | 0xffe1 APP1  |       4 | ....
     619 | 0xffc0 SOF0  |      17 
     638 | 0xffda SOS

If I'm not wrong it seems like there's already some DHTs in there.

Here is the file that I got by cropping from ff d8 to ff d9.

This file is not totally corrupted the background is actually my room's ceiling the only corrupt part are these horizontal patches.

And the client application for the webcam displays the video just fine without any artifacts. I would love to have your thoughts on this.

Thanks

PS: Here's the hex dump of this file.

00000000: ffd8 ffe0 0021 4156 4931 0001 0101 0078  .....!AVI1.....x
00000010: 0078 0000 0000 0000 0000 0000 0000 0000  .x..............
00000020: 0000 0000 00ff db00 4300 0604 0405 0403  ........C.......
00000030: 0605 0405 0606 0607 090f 0909 0808 0912  ................
00000040: 0d0d 0a0f 1513 1616 1513 1514 181b 221d  ..............".
00000050: 1819 2019 1415 1e28 1e20 2324 2627 2617  .. ....(. #$&'&.
00000060: 1c2a 2d2a 252d 2225 2625 ffdb 0043 0106  .*-*%-"%&%...C..
00000070: 0606 0907 0911 0909 1125 1815 1825 2525  .........%...%%%
00000080: 2525 2525 2525 2525 2525 2525 2525 2525  %%%%%%%%%%%%%%%%
00000090: 2525 2525 2525 2525 2525 2525 2525 2525  %%%%%%%%%%%%%%%%
000000a0: 2525 2525 2525 2525 2525 2525 2525 25ff  %%%%%%%%%%%%%%%.
000000b0: c400 1f00 0001 0501 0101 0101 0100 0000  ................
000000c0: 0000 0000 0001 0203 0405 0607 0809 0a0b  ................
000000d0: ffc4 00b5 1000 0201 0303 0204 0305 0504  ................
000000e0: 0400 0001 7d01 0203 0004 1105 1221 3141  ....}........!1A
000000f0: 0613 5161 0722 7114 3281 91a1 0823 42b1  ..Qa."q.2....#B.
00000100: c115 52d1 f024 3362 7282 090a 1617 1819  ..R..$3br.......
00000110: 1a25 2627 2829 2a34 3536 3738 393a 4344  .%&'()*456789:CD
00000120: 4546 4748 494a 5354 5556 5758 595a 6364  EFGHIJSTUVWXYZcd
00000130: 6566 6768 696a 7374 7576 7778 797a 8384  efghijstuvwxyz..
00000140: 8586 8788 898a 9293 9495 9697 9899 9aa2  ................
00000150: a3a4 a5a6 a7a8 a9aa b2b3 b4b5 b6b7 b8b9  ................
00000160: bac2 c3c4 c5c6 c7c8 c9ca d2d3 d4d5 d6d7  ................
00000170: d8d9 dae1 e2e3 e4e5 e6e7 e8e9 eaf1 f2f3  ................
00000180: f4f5 f6f7 f8f9 faff c400 1f01 0003 0101  ................
00000190: 0101 0101 0101 0100 0000 0000 0001 0203  ................
000001a0: 0405 0607 0809 0a0b ffc4 00b5 1100 0201  ................
000001b0: 0204 0403 0407 0504 0400 0102 7700 0102  ............w...
000001c0: 0311 0405 2131 0612 4151 0761 7113 2232  ....!1..AQ.aq."2
000001d0: 8108 1442 91a1 b1c1 0923 3352 f015 6272  ...B.....#3R..br
000001e0: d10a 1624 34e1 25f1 1718 191a 2627 2829  ...$4.%.....&'()
000001f0: 2a35 3637 3839 3a43 4445 4647 4849 4a53  *56789:CDEFGHIJS
00000200: 5455 5657 5859 5a63 6465 6667 6869 6a73  TUVWXYZcdefghijs
00000210: 7475 7677 7879 7a82 8384 8586 8788 898a  tuvwxyz.........
00000220: 9293 9495 9697 .....