tokio-rs
consider adding `task::Id` to `u64` conversion
Is your feature request related to a problem? Please describe.
Currently, Tokio's task::Id is an opaque newtype around a NonZeroU64. They implement Debug, Display, Eq, and Hash, making them usable for a variety of purposes within Rust code in the same program --- they can be printed or used as map keys. However, when exposing task IDs to external observability systems, the opaqueness of the newtype is an issue: it may be desirable to serialize or otherwise export them as integers, which isn't really possible at present in safe code. The only options available to code that needs to expose task IDs as integers to an external system are to either format the ID as a string and then parse the integer out of it, or do an unsafe cast to transmute it into a u64. The first option has the overhead of allocating a string and parsing it, which might be unacceptable in a hot path, while the second option runs the risk of breaking if Tokio changes the layout of task::Id.
In tokio-dtrace, I'm currently doing the unsafe transmute because I didn't want to add a string allocation in every task poll. I've added some checks to ensure that tokio-dtrace returns an error on setup rather than doing an unsound cast, but it still feels Not Great.
Describe the solution you'd like
It would be really nice if there was a TaskId::as_u64, Into<u64>, or Into<NonZeroU64> conversion that just gives you the integer value. We may prefer making the API provide the task ID as a u64 rather than a NonZeroU64 to avoid committing to it being non-zero in the public API.
Describe alternatives you've considered
Keep doing the current unsafe thing, or allocate a string on every task poll to parse the integer out of. As described above, these are both quite unfortunate in different ways.
Additional context
Adding a conversion to u64 means we would be stabilizing a commitment to continuing to represent task::Id as a u64, or at least as "something that can be easily turned into a u64". However, we've had task IDs for three years now and haven't changed the representation away from being a 64-bit integer, so I think we're probably safe committing to it.
Alternatively, we could make the task ID to integer conversion require tokio_unstable if we're uncomfortable committing to the idea that they'll be u64s forever at this point. That would be fine for my use case, since I'm using the runtime hooks and require unstable features anyway.
On the go, so quick question. The reason for needing it isn’t clear. Why is using a string or u64 repr the only option? Could you elaborate on the problem without getting into the solution?
Hi @carllerche, I'd like to help clarify the problem:
The Problem: External observability tools (like DTrace, eBPF, OpenTelemetry) need to correlate Tokio tasks with their internal tracking systems. These tools typically work at the system level and only understand primitive types - they can't interpret Rust's opaque types.
Why strings/u64 are the only options:
- System-level tools limitation: DTrace probes, eBPF programs, and similar tools can only work with C-compatible types (integers, pointers, strings)
- Cross-process correlation: When correlating events across different processes or systems, you need a serializable identifier
- Performance constraints: In hot paths (like every task poll), allocating strings for ID conversion adds overhead
Current workarounds and their problems:
-
String parsing:
format!("{}", task_id)then parse - adds allocation + parsing overhead on every poll -
Unsafe transmute: Relies on internal layout of
Id, breaks if Tokio changes implementation
Real-world example:
In tokio-dtrace, tracking task polls requires passing the ID to DTrace on every poll event. With thousands of polls per second, the string allocation overhead becomes significant.
The request is simply to expose the existing internal as_u64() method publicly, allowing zero-cost conversion for observability tools while maintaining Tokio's type safety for regular Rust code.
Hi! I will like to start contributing to this issue if this is still relevant. Thanks.
There isn't consensus on this issue, so I would not suggest implementing anything at this stage.
I've been using the chromium tracing format - to get some visualization with perfetto - and it needs some integer identifier to distinguish between thread/tracks running within a process.
Although it's possible to just "roll my own" incrementing counter here, that loses all information about tasks when tracing, which is kinda a bummer. Would be really nice if I could use the actual task ID that tokio is using.
(TL;DR: One small extra case where having a u64 exposed here would be handy!)
