Translate nullable pointers to Option<T>

[jameysharp]

Right now, Corrode translates C pointer types to *const T or *mut T (the raw pointer types) or unsafe extern fn(...) (for function pointers). This has some issues around null pointers.

Rust's raw pointers are allowed to be null (conveniently for us), and can be cast to and from integers. The standard library provides ptr::null(), ptr::null_mut(), and p.is_null() functions for creating and checking for null pointers. However, we can't call ptr::null() in initializers for static variables, because that isn't declared as a const function, so right now we generate null pointer initializers by casting 0 to the appropriate pointer type.

Rust's function pointers, on the other hand, are not allowed to be null, or cast to or from integers. One consequence is that we can't construct a correct zero-initializer for them at all.

An alternative encoding for null is to wrap pointers in Option. Rust's "null pointer optimization" (I can't find a good reference for it, but everyone who talks about it uses that exact phrase) makes the runtime representation of Option exactly the same size as a pointer when it's used with borrows, raw pointers, or function pointers. So that sounds like a nice general way to clean up Corrode's translation. Instead of p.is_null(), we'd use p.is_none() or p.is_some(); and we could construct a null pointer at any time with None, even in a static initializer.

I think this encoding would also enable us to generate better code if the C source uses the C99 restrict keyword. I believe any pointer declaration that uses restrict is equivalent to a possibly-null borrow in Rust. (Whether it's a mutable borrow depends on whether the const keyword is also present.) There might be caveats though; for instance, if the program does any pointer arithmetic on the address then we probably have to translate it as a raw pointer anyway.

We can also use the GCC function attributes nonnull and returns_nonnull to omit the Option wrapper when the programmer promises that's safe. That's more complicated, since we have to wrap and unwrap the Option type as values flow between nullable and non-null pointers, but I think it's worth doing someday.

My hope is that the combination of restrict and nonnull will give programmers the means to clean up their C source such that Corrode will produce more idiomatic Rust from it, while still doing something sensible in the absence of either keyword.

But in the short term, I think we need to at least wrap all function pointers in Option, and in that case maybe we might as well do it for non-function pointers too.

[tcr]

If you want a source, null-pointer optimization is described in the second to last paragraph here: https://doc.rust-lang.org/nomicon/repr-rust.html

[jameysharp]

Thanks @tcr! Also, I've been informed that the null-pointer optimization doesn't apply to raw pointers unless they're wrapped in core::nonzero::NonZero, because otherwise the compiler wouldn't be able to distinguish between None and Some(ptr::null()).

So I guess we'll either wind up with types like Option<NonZero<*mut u8>>, or just continue using nullable raw pointers. I kind of like the more complex type because inserting .unwrap() calls at each pointer dereference would transform unsafe null-pointer dereferences into "safe" panics. But maybe that should be a post-processing pass that does more static analysis, propagating knowledge about non-null pointers along control-flow paths.

I think no matter what we'll still need to wrap function pointers in Option, because those don't (as far as I can tell) permit null values in Rust.