Global resource handling

[jethrogb]

Often, global decisions affecting the entire dependency tree need to be made. Current examples of this in std are panic behavior and the global allocator. Outside of std there are logging and event loops, standard RNG.

This issue proposes to come up with a generic system that works for all such cases.

[Ericson2314]

The perfect general solution need be ML modules, or something else that's just as big a change. But there is a more incremental solution on top of https://github.com/rust-lang/rust/issues/27389 . Basically the pattern there is there is some ZST which implements a trait by calling the methods on the user-defined static. We could replace #![global_allocator] with #![global(MyTrait)] to expose that pattern in its full generality.

[jethrogb]

@aturon In our meeting you mentioned event loops are a global resource. Why? If you are currently a task running on an event loop, you should be able to figure that out somehow (e.g. Task::current()). If you are not currently on an event loop, you're probably in charge of instantiating it.

[jethrogb]

https://github.com/rust-lang/rust/issues/27389#issuecomment-342285805

[jethrogb]

Missing from the current allocator API design is a good way to hook into the selected global singleton. There's a lot of compiler magic to make it work for what's tagged #[global_allocator], including adding global functions that just forward to the calling the trait methods on the singleton. However, this is only done if you explicitly choose a global allocator, and not when using the standard allocator.

See https://github.com/rust-lang/rust/blob/master/src/librustc_allocator/expand.rs and https://github.com/rust-lang/rust/blob/master/src/librustc/middle/allocator.rs

I guess the only way to do this kind of dependency inversion in Rust today is with a extern { fn } kludge.

[jethrogb]

How about leveraging extern statics?

// liballoc
pub mod allocator {
  pub unsafe trait Alloc { /* ... */ }
}

pub mod heap {
  extern {
    #[global="allocator"]
    static HEAP: impl ::allocator:Alloc;
  }
}

// liballoc_system
use alloc::allocator::Alloc;

struct System;

impl Alloc for System { /* ... */ }

#[global="allocator"]
static SYSTEM_ALLOCATOR: System = System;

[pierzchalski]

Is there an issue using existing/"boring" extern statics? Eg.

pub mod heap {
  extern {
    // Static trait object because an anonymous `impl` type can't be simply linked
    static HEAP_IMPL: &::allocator::Alloc;
  }
}

#[no_mangle]
static HEAP_IMPL: &Alloc = &System;

Naively this looks like it would have some runtime overhead from the vtable, but... it's static? I would hope it would be eligible for inlining or LTO or whatever. Not sure if we can do good error reporting if someone includes two global allocators, though.

[jethrogb]

I guess that's possible. I'm not a big fan because:

1. Accessing normal extern statics is unsafe.
2. Normal extern statics are subject to the improper_ctypes lint.
3. Dynamic dispatch. LTO might pick it up but I'm not sure.
4. Having the linker do collision detection is not very user-friendly

[pierzchalski]

Yeah, fair enough. What's your vision on how the compiler should see an extern static X: impl Y? That's spiritually the kind of existential type that an ML-style module system would provide, it's just being filled via a magic global name instead of module inheritance.

[Ericson2314]

https://github.com/rust-lang/rfcs/pull/2492 is an RFC for this.

[Ericson2314]

(@pitdicker I'm sorry to say I didn't include the rand example is it already does a decent job by spliting the create into two with a trait, but yes we could overhaul the thing such that the core trait has both OsRng and ThreadRng as extern existential types, and then each OS's implementation has its own crate.