RFC: `#[late]` statics

[japaric]

Summary

#[late] static are statics that are initialized at runtime, before the entry point is called and after RAM initialization happens.

Motivation

Consider the allocator example from the quickstart template:

#[global_allocator]
static ALLOCATOR: CortexMHeap = CortexMHeap::empty();

const HEAP_SIZE: usize = 1024; // in bytes

#[entry]
fn main() -> ! {
    // Initialize the allocator BEFORE you use it
    unsafe { ALLOCATOR.init(cortex_m_rt::heap_start() as usize, HEAP_SIZE) }

    let xs = vec![0, 1, 2];

    // ..
}

Quite a few things can go wrong there: you could forget to initialize the allocator or you could initialize it too late. In either case you end up with OOM (abort).

This example can be simplified using a #[late] static:

#[late]
#[global_allocator]
static ALLOCATOR: SyncAllocator = unsafe {
    static mut BYTES: [u8; 1024] = [0; 1024];

    // you are familiar with this transformation
    let bytes: &'static mut [u8] = BYTES;

    // `SyncAllocator` is a newtype over `Mutex<SomeAllocator>`
    SyncAllocator::new(bytes)
}

#[entry]
fn main() -> ! {
    // Growable array allocated on the heap
    let xs = vec![0, 1, 2];

    // ..
}

Design

#[late] statics are not lazy statics. Lazy statics are initialized on first use, they have runtime cost on each access and memory overhead. #[late] statics are statics that are initialized before the #[entry] function is called, accessing them is zero cost and they have no memory overhead.

The above example expands to something like this:

// For details about the `.uninint` section see RFC rust-embedded/cortex-m#398
#[link_section = ".uninit.<random-hash>"]
#[global_allocator]
static ALLOCATOR: Late<SyncAllocator> = {
    #[link_section = ".late.<random-hash>"] // new linker section
    static INITIALIZER: unsafe fn() = initialize;

    unsafe fn initialize() {
        let val = {
            // (proc-macro transformation)
            let BYTES: &'static mut _ = {
                static mut BYTES: [u8; 1024] = [0; 1024];

                &mut BYTES
            };

            // (user code)
            // you are familiar with this transformation
            let bytes: &'static mut [u8] = BYTES;

            // `SyncAllocator` is a newtype over `Mutex<Allocator>`
            SyncAllocator::new(bytes)
        };

        ALLOCATOR.initialize(val);
    }

    Late::uninitialized()
};

The Late newtype will be provided by this crate:

/// IMPLEMENTATION DETAIL. DO NOT USE
#[doc(hidden)]
pub struct Late<T> {
    inner: UnsafeCell<MaybeUninit<T>>,
}

unsafe impl<T> Sync for Late<T> where T: Sync {}

impl<T> Late<T> {
    /// IMPLEMENTATION DETAIL. DO NOT USE
    #[doc(hidden)]
    pub const fn uninitialized() -> Self {
        Late {
            inner: UnsafeCell::new(MaybeUninit::uninitialized()),
        }
    }

    /// IMPLEMENTATION DETAIL. DO NOT USE
    #[doc(hidden)]
    pub unsafe fn initialize(&self, value: T) {
        (*self.inner.get()).set(value)
    }
}

impl<T> Deref for Late<T> {
    type Target = T;

    fn deref(&self) -> &T {
        unsafe { (*self.inner.get()).get_ref() }
    }
}

impl<T> DerefMut for Late<T> {
    fn deref_mut(&mut self) -> &mut T {
        unsafe { (*self.inner.get()).get_mut() }
    }
}

As you can see Deref and DerefMut perform no runtime checks. However, dereferencing is only valid after Late has been initialize-d.

#[late] statics are initialized before #[entry] / main and after RAM initialization using r0::run_init_array or its equivalent.

fn Reset() -> ! {
    __pre_init();

    r0::zero_bss(&mut __sbss, &mut __ebss);
    r0::init_data(&mut __sdata, &mut __edata, & __sidata);
    r0::run_init_array(&__slate, &__elate);

    main()
}

#[global_allocator]

To compose with the built-in global_allocator attribute we need to provide the implementation below in this crate.

unsafe impl<T> GlobalAlloc for Late<T>
where
    T: GlobalAlloc,
{
    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
        T::alloc(self.deref(), layout)
    }

    unsafe fn dealloc(&self, p: *mut u8, layout: Layout) {
        T::dealloc(self.deref(), p, layout)
    }
}

Why unsafe?

Using normal statics within a #[late] static initializer is sound, but using any #[late] static within the initializer is not because there are no guarantees about the execution order of #[late] initializers.

Furthermore for global resources like global_allocator trying to use the resource before it's initialized is also undefined behavior.

#[late]
#[global_allocator]
static ALLOCATOR: SyncAllocator = unsafe {
    static mut BYTES: [u8; 1024] = [0; 1024];

    let xs = vec![0, 1, 2]; // undefined behavior
    ALLOCATOR.alloc(..); // also undefined behavior

    SyncAllocator::new(BYTES)
}

It's not possible to prevent all these issues at compile time so unsafe must be used. Arguably, we could prevent the first issue by making the static unnameable (by assigning it a randomly generated name) but this would greatly reduce the flexibility of #[late] statics.

---

Thoughts? Are there use cases other than global allocators?

[korken89]

I think this was never discussed?

From my side I see great usage for this, for example when calculating encryption keys for the system on startup.

[jonas-schievink]

I personally feel uneasy about bringing the static initialization order fiasco to Rust, but I can see that this would be useful. However, "not forgetting to call a function" doesn't really strike me as the most important use case of this feature either.

It seems like this would effectively bring RTFM's late resources to the non-RTFM world, which would be pretty great, but I'm sure there must be some way of doing that without the unsafety and initialization order problems – RTFM also manages to do this after all.

[thejpster]

This sounds like C++ constructors for statics - oh the fun I've had trying to get the compiler to call the constructors in the right order. In the past I used C++ static objects to represent singletons like serial ports, and I could see a use case for that in Rust.