Question: Replicating C bitfield structs with multi byte fields?

[TheDan64]

Hi, I'm trying to recreate some bitfield structs from C in rust with C FFI compatibility in mind. I'm having some trouble. Here are some examples:

Suppose we have this super compact date bitfield struct:

struct date {
   unsigned char d: 5;
   unsigned char m: 4;
   unsigned short y;
};

Clang tells me the layout (on my machine) is such:

*** Dumping AST Record Layout
        0 | struct date
    0:0-4 |   unsigned char d
    1:0-3 |   unsigned char m
        2 |   unsigned short y
          | [sizeof=4, align=2]

Ok, easy enough to replicate:

#[derive(PackedStruct)]
#[packed_struct(bit_numbering="msb0")]
pub struct Date {
    #[packed_field(bits="0..=4")]
    pub day: u8,
    #[packed_field(bits="8..=11")]
    pub month: u8,
    #[packed_field(bytes="2..=3", endian="msb")]
    pub year: u16,
}

But now I try something much less compact but still C compliant:

struct date2 {
   unsigned long d: 5;
   unsigned short m: 4;
   unsigned short y;
};

and get this layout from clang:

*** Dumping AST Record Layout
        0 | struct date
    0:0-4 |   unsigned long d
    0:5-8 |   unsigned short m
        2 |   unsigned short y
          | [sizeof=8, align=8]

But when I try and replicate this one:

#[derive(PackedStruct)]
#[packed_struct(bit_numbering="msb0")]
pub struct Date2 {
    #[packed_field(bits="0..=4")]
    pub day: u32,
    #[packed_field(bits="5..=8")]
    pub month: u16,
    #[packed_field(bytes="2..=3", endian="msb")]
    pub year: u16,
}

I get a lot of unhelpful error messages:

error[E0277]: the trait bound `packed_struct::types::Integer<u32, packed_struct::types::bits::Bits5>: std::convert::From<u32>` is not satisfied
  --> bf.rs:49:10
   |
49 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^ the trait `std::convert::From<u32>` is not implemented for `packed_struct::types::Integer<u32, packed_struct::types::bits::Bits5>`
   |
   = help: the following implementations were found:
             <packed_struct::types::Integer<i32, packed_struct::types::bits::Bits25> as std::convert::From<i32>>
             <packed_struct::types::Integer<i32, packed_struct::types::bits::Bits32> as std::convert::From<i32>>
             <packed_struct::types::Integer<i64, packed_struct::types::bits::Bits46> as std::convert::From<i64>>
             <packed_struct::types::Integer<i8, packed_struct::types::bits::Bits8> as std::convert::From<i8>>
           and 124 others
   = note: required because of the requirements on the impl of `bitfield::Into<packed_struct::types::Integer<u32, packed_struct::types::bits::Bits5>>` for `u32`

error[E0599]: no method named `pack` found for type `&packed_struct::types::MsbInteger<_, _, packed_struct::types::Integer<u32, packed_struct::types::bits::Bits5>>` in the current scope
  --> bf.rs:49:10
   |
49 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^
   |
   = note: the method `pack` exists but the following trait bounds were not satisfied:
           `packed_struct::types::MsbInteger<_, _, packed_struct::types::Integer<u32, packed_struct::types::bits::Bits5>> : packed_struct::PackedStruct<_>`
           `packed_struct::types::MsbInteger<_, _, packed_struct::types::Integer<u32, packed_struct::types::bits::Bits5>> : packed_struct::PackedStruct<_>`

error[E0277]: the trait bound `packed_struct::types::Integer<u16, packed_struct::types::bits::Bits4>: std::convert::From<u16>` is not satisfied
  --> bf.rs:49:10
   |
49 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^ the trait `std::convert::From<u16>` is not implemented for `packed_struct::types::Integer<u16, packed_struct::types::bits::Bits4>`
   |
   = help: the following implementations were found:
             <packed_struct::types::Integer<i32, packed_struct::types::bits::Bits25> as std::convert::From<i32>>
             <packed_struct::types::Integer<i32, packed_struct::types::bits::Bits32> as std::convert::From<i32>>
             <packed_struct::types::Integer<i64, packed_struct::types::bits::Bits46> as std::convert::From<i64>>
             <packed_struct::types::Integer<i8, packed_struct::types::bits::Bits8> as std::convert::From<i8>>
           and 124 others
   = note: required because of the requirements on the impl of `bitfield::Into<packed_struct::types::Integer<u16, packed_struct::types::bits::Bits4>>` for `u16`

error[E0599]: no method named `pack` found for type `&packed_struct::types::MsbInteger<_, _, packed_struct::types::Integer<u16, packed_struct::types::bits::Bits4>>` in the current scope
  --> bf.rs:49:10
   |
49 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^
   |
   = note: the method `pack` exists but the following trait bounds were not satisfied:
           `packed_struct::types::MsbInteger<_, _, packed_struct::types::Integer<u16, packed_struct::types::bits::Bits4>> : packed_struct::PackedStruct<_>`
           `packed_struct::types::MsbInteger<_, _, packed_struct::types::Integer<u16, packed_struct::types::bits::Bits4>> : packed_struct::PackedStruct<_>`

error[E0277]: the trait bound `packed_struct::types::Integer<u32, packed_struct::types::bits::Bits5>: packed_struct::types::SizedInteger<_, _>` is not satisfied
  --> bf.rs:49:10
   |
49 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^ the trait `packed_struct::types::SizedInteger<_, _>` is not implemented for `packed_struct::types::Integer<u32, packed_struct::types::bits::Bits5>`
   |
   = help: the following implementations were found:
             <packed_struct::types::Integer<i64, packed_struct::types::bits::Bits49> as packed_struct::types::SizedInteger<i64, packed_struct::types::bits::Bits49>>
             <packed_struct::types::Integer<i64, packed_struct::types::bits::Bits61> as packed_struct::types::SizedInteger<i64, packed_struct::types::bits::Bits61>>
             <packed_struct::types::Integer<u32, packed_struct::types::bits::Bits26> as packed_struct::types::SizedInteger<u32, packed_struct::types::bits::Bits26>>
             <packed_struct::types::Integer<u64, packed_struct::types::bits::Bits40> as packed_struct::types::SizedInteger<u64, packed_struct::types::bits::Bits40>>
           and 124 others
   = note: required because of the requirements on the impl of `packed_struct::PackedStruct<[u8; 1]>` for `packed_struct::tinternal error: cargo failed with status 101
ypes::MsbInteger<_, _, packed_struct::types::Integer<u32, packed_struct::types::bits::Bits5>>`
   = note: required by `packed_struct::PackedStruct::unpack`

error[E0614]: type `packed_struct::types::Integer<u32, packed_struct::types::bits::Bits5>` cannot be dereferenced
  --> bf.rs:49:10
   |
49 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^

error[E0277]: the trait bound `packed_struct::types::Integer<u16, packed_struct::types::bits::Bits4>: packed_struct::types::SizedInteger<_, _>` is not satisfied
  --> bf.rs:49:10
   |
49 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^ the trait `packed_struct::types::SizedInteger<_, _>` is not implemented for `packed_struct::types::Integer<u16, packed_struct::types::bits::Bits4>`
   |
   = help: the following implementations were found:
             <packed_struct::types::Integer<i64, packed_struct::types::bits::Bits49> as packed_struct::types::SizedInteger<i64, packed_struct::types::bits::Bits49>>
             <packed_struct::types::Integer<i64, packed_struct::types::bits::Bits61> as packed_struct::types::SizedInteger<i64, packed_struct::types::bits::Bits61>>
             <packed_struct::types::Integer<u32, packed_struct::types::bits::Bits26> as packed_struct::types::SizedInteger<u32, packed_struct::types::bits::Bits26>>
             <packed_struct::types::Integer<u64, packed_struct::types::bits::Bits40> as packed_struct::types::SizedInteger<u64, packed_struct::types::bits::Bits40>>
           and 124 others
   = note: required because of the requirements on the impl of `packed_struct::PackedStruct<[u8; 1]>` for `packed_struct::types::MsbInteger<_, _, packed_struct::types::Integer<u16, packed_struct::types::bits::Bits4>>`
   = note: required by `packed_struct::PackedStruct::unpack`

error[E0614]: type `packed_struct::types::Integer<u16, packed_struct::types::bits::Bits4>` cannot be dereferenced
  --> bf.rs:49:10
   |
49 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^

error[E0277]: the trait bound `u32: std::convert::From<i32>` is not satisfied
   --> bf.rs:116:11
    |
116 |         day: 31.into(),
    |                 ^^^^ the trait `std::convert::From<i32>` is not implemented for `u32`
    |
    = help: the following implementations were found:
              <u32 as std::convert::From<char>>
              <u32 as std::convert::From<std::net::Ipv4Addr>>
              <u32 as std::convert::From<std::num::NonZeroU32>>
              <u32 as std::convert::From<u16>>
            and 18 others
    = note: required because of the requirements on the impl of `bitfield::Into<u32>` for `i32`

Any idea what I'm doing wrong, and how to get this example working?

[rudib]

The errors really aren't helpful, but it boils down to that we need to explicitly define the bit widths of every integer field so that the structures fields are protected for overflows (or at least masked). Try using types like these:

#[derive(PackedStruct)]
#[packed_struct(bit_numbering="msb0")]
pub struct Date {
    #[packed_field(bits="0..=4")]
    pub day: Integer<u8, packed_bits::Bits5>,
    #[packed_field(bits="8..=11")]
    pub month: Integer<u8, packed_bits::Bits4>,
    #[packed_field(bytes="2..=3", endian="msb")]
    pub year: Integer<u8, packed_bits::Bits2>,
}

That's where you'll also need the .into() conversion as the sized integer wrapper type performs integer masking as the part of the conversion.

[TheDan64]

Thank you so much for the quick reply!

A couple questions:

1. Isn't that a different version of Date? That example worked fine, it's the 2nd example (Date2) that didn't compile.

2. Interestingly, with that code there is a compile error too:

error[E0308]: mismatched types
  --> bf.rs:60:10
   |
60 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^ expected an array with a fixed size of 1 elements, found one with 2 elements
   |
   = note: expected type `&[u8; 1]`
              found type `&[u8; 2]

3. Why do the int types in Integer<u8, ...> have to be bytes? What if I'd like the field to be a u16, u32, etc?

[rudib]

Sorry, I didn't actually compile your example to get it fully working, I'll try to look into that in the following days - your compile error looks new to me. Try looking into the tests and the docs how the small integer fields are implemented.

The in-memory type of a field is defined with the first generic parameter type and the packing code will happily compile if the appropriate traits are implemented. You can see a list of implementations in the docs:

https://docs.rs/packed_struct/0.3.0/packed_struct/types/trait.SizedInteger.html

But I'm not sure if having a u16 and packing it down to two bits is supported and even sensible.

[TheDan64]

Sorry, I didn't actually compile your example to get it fully working, ...

No worries!

But I'm not sure if having a u16 and packing it down to two bits is supported and even sensible.

It wasn't two bits, but two bytes:

    #[packed_field(bytes="2..=3", endian="msb")]
    pub year: u16,

...that said, I think it is sensible since C seems to allow you to do this. You can define a bitfield to be any arbitrary int size for the sake of assignments with ints of that type, even if only a certain number of bits are actually used.

[rudib]

Oops, I speed-read it as two bits. Of course, then just use a plain u16 and it will work, the long syntax would need to use a wrapper with Bits16. 😄

[TheDan64]

This also surprisingly doesn't work which is just two bytes:

#[derive(PackedStruct)]
#[packed_struct(bit_numbering="msb0")]
pub struct TwoBytes {
    #[packed_field(bytes="0..=1", endian="msb")]
    pub tb: Integer<u16, Bytes2>,
}

error[E0599]: no method named `pack` found for type `&packed_struct::types::Integer<u16, packed_struct::types::bits::Bytes2>` in the current scope
  --> bf.rs:34:10
   |
34 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^

error[E0277]: the size for values of type `[u8]` cannot be known at compilation time
  --> bf.rs:34:10
   |
34 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^ doesn't have a size known at compile-time
   |
   = help: the trait `std::marker::Sized` is not implemented for `[u8]`
   = note: to learn more, visit <https://doc.rust-lang.org/book/second-edition/ch19-04-advanced-types.html#dynamically-sized-types-and-the-sized-trait>
   = note: all local variables must have a statically known size
   = help: unsized locals are gated as an unstable feature

error[E0599]: no function or associated item named `unpack` found for type `packed_struct::types::Integer<u16, packed_struct::types::bits::Bytes2>` in the current scope
  --> bf.rs:34:10
   |
34 | #[derive(PackedStruct)]
   |          ^^^^^^^^^^^^ function or associated item not found in `packed_struct::types::Integer<u16, packed_struct::types::bits::Bytes2>`

but works if you replace Bytes2 with Bits16, but one would expect them to be interchangeable