Guarantee soundness of pointer-to-int transmutes

[joshlf]

Resolves https://github.com/rust-lang/unsafe-code-guidelines/issues/286 by guaranteeing that it is sound to transmute a pointer to a sequence of u8s of the same size (also known as a "pointer-to-integer" or "ptr2int" transmute).

See the discussion starting here. The current consensus is that:

• ptr2int will likely always be well-defined in practice
• The only concerns regarding committing to soundness are:
  • Possible future optimizations which would be impossible given that ptr2int is sound (no specific optimizations have been mentioned)
  • Possible future changes to LLVM which would make ptr2int unsound
• It will remain UB to round-trip a pointer through a usize or other non-pointer type and then to dereference, as provenance will be lost
• Most code in the ecosystem which uses ptr2int does so in order to round-trip back to a pointer, and so this code will remain unsound even in the face of ptr2int being sound
• The main benefits of guaranteeing the soundness of ptr2int are:
  • It enables abstractions like the ones provided by zerocopy to be defined in a modular way, rather than having to have special carve-outs for pointers
  • It results in a simpler mental model of transmutes (this has not been discussed much, so it's possible that it's overly-strong to call it the consensus view)

[ehuss]

@joshlf Just checking if you are still interested in working on this?

In general it would be helpful for these kinds of PRs to have a short summary about what it is changing instead of just linking to an issue. It would also be helpful to indicate what consensus has already been gathered with the opsem or lang team, or doing the legwork to gather that consensus, and making it clear what work needs to happen if the PR isn't ready. These changes will eventually need to be brought up with the lang team for a final review, and it is very helpful to have a summary of the context so that they can process it more easily.

[joshlf]

@joshlf Just checking if you are still interested in working on this?

Yes, I'm still interested. It may take a while to drive this towards consensus as it's a fairly abstract discussion. I can't promise that I'll have consistent cycles to push the discussion forward (my work goes through phases regarding what I can focus on), but I will push it forward periodically.

In general it would be helpful for these kinds of PRs to have a short summary about what it is changing instead of just linking to an issue. It would also be helpful to indicate what consensus has already been gathered with the opsem or lang team, or doing the legwork to gather that consensus, and making it clear what work needs to happen if the PR isn't ready. These changes will eventually need to be brought up with the lang team for a final review, and it is very helpful to have a summary of the context so that they can process it more easily.

I've updated the PR description; let me know if the updated text looks good, or if you'd like to see other changes.

[ia0]

By the way, this can be considered an incremental part of the larger effort started in #1664. Commenting to create the link between both PRs since there was none yet.