Wrong shape of Python Aruco dictionary `bytesList`?

[abetusk]

System information (version)

• OpenCV => 4.0.0-dev
• Operating System / Platform => debian 9.4 (64 Bit)
• Compiler => gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516

Detailed description

I'm not sure this is a bug or not but I'm filing in case it is.

When investigating loading a custom Aruco Dictionary for Python (2.7) I noticed the bytesList looks (to me) as if it's been reshaped improperly, going from a 4 row by 5 column array in the predefined_dictionaries.hpp to a 5 row by 4 column array in Python.

For example, here is a short Python snippet:

>>> print cv2.aruco.Dictionary_get(cv2.aruco.DICT_6X6_250).bytesList
[[[ 30  61 216  42]
  [  6 227 186  70]
  [ 49   9 101  65]
  [187 199   8 152]
  [198  37 220   7]]
...
>>> print cv2.aruco.Dictionary_get(cv2.aruco.DICT_6X6_250).bytesList.shape
(250, 5, 4)

And the relevant code snippet in predefined_dictionaries.hpp:

...
static unsigned char DICT_6X6_1000_BYTES[][4][5] =
    { { { 30, 61, 216, 42, 6 },
        { 227, 186, 70, 49, 9 },
        { 101, 65, 187, 199, 8 },
        { 152, 198, 37, 220, 7 }, },
...

Note that the data is all there, it just looks like the shape of the matrix is wrong in Python.

I would like to load a custom Aruco dictionary through Python which presumably means I would need to feed in a bytesList with the oddly shaped format in Python. Since, as far as I can tell, bytesList is part of the specification and needs to be populated for custom Aruco dictionaries, even if it's being used consistently in the OpenCV library the portion exposed seems incorrect.

Steps to reproduce

See above, but for redundancies sake (in Python):

>>> print cv2.aruco.Dictionary_get(cv2.aruco.DICT_6X6_250).bytesList
[[[ 30  61 216  42]
  [  6 227 186  70]
  [ 49   9 101  65]
  [187 199   8 152]
  [198  37 220   7]]
...
>>> print cv2.aruco.Dictionary_get(cv2.aruco.DICT_6X6_250).bytesList.shape
(250, 5, 4)

[paroj]

I think this is just because of row vs column major addressing between C and numpy

[dogod621]

in fact aruco use these byte data as 1-d array: 30, 61, 216, 42, 6,....

it group 4 bytes as a row is just for storing the byteList as Mat image (RGBA, 4 bytes)

so the shape affect nothing

Notice how aruco use the byteList, it directlly use raw pointer of Mat opencv_contrib/modules/aruco/src/dictionary.cpp line 236

Mat Dictionary::getBitsFromByteList(const Mat &byteList, int markerSize) {
    CV_Assert(byteList.total() > 0 &&
              byteList.total() >= (unsigned int)markerSize * markerSize / 8 &&
              byteList.total() <= (unsigned int)markerSize * markerSize / 8 + 1);
    Mat bits(markerSize, markerSize, CV_8UC1, Scalar::all(0));

    unsigned char base2List[] = { 128, 64, 32, 16, 8, 4, 2, 1 };
    int currentByteIdx = 0;
    // we only need the bytes in normal rotation
    unsigned char currentByte = byteList.ptr()[0];
    int currentBit = 0;
    for(int row = 0; row < bits.rows; row++) {
        for(int col = 0; col < bits.cols; col++) {
            if(currentByte >= base2List[currentBit]) {
                bits.at< unsigned char >(row, col) = 1;
                currentByte -= base2List[currentBit];
            }
            currentBit++;
            if(currentBit == 8) {
                currentByteIdx++;
                currentByte = byteList.ptr()[currentByteIdx];
                // if not enough bits for one more byte, we are in the end
                // update bit position accordingly
                if(8 * (currentByteIdx + 1) > (int)bits.total())
                    currentBit = 8 * (currentByteIdx + 1) - (int)bits.total();
                else
                    currentBit = 0; // ok, bits enough for next byte
            }
        }
    }
    return bits;
}

[AleksandrPanov]

bytesList storing as 2-dimensions Mat with 4-th channels (CV_8UC4 type was used) and contains the marker codewords where:

• bytesList.rows is the dictionary size
• each marker is encoded using nbytes = ceil(markerSize*markerSize/8.)
• each row contains all 4 rotations of the marker, so its length is 4*nbytes
• the byte order in the bytesList[i] row:
  • ||Bytes without rotation| |bytes with rotation 1| |bytes with rotation 2| |bytes with rotation 3||
• To get j-th byte in i-th marker with k-th rotation in С++ use: unsingned char byte = bytesList.ptr(i)[k*nbytes + j]
• To get j-th byte in i-th marker with k-th rotation in Python use: byte = aruco_dict.bytesList[id].ravel()[k*nbytes + j]
• Attention, in Python shape of bytesList is dictionary_size x nbytes x 4, but you need to use two-dimensional indexing (see the example above).