tracing
tracing copied to clipboard
Consider changing Event struct to have non-'static metadata (Cow?)
ref. #922
Currently the Event
type in tracing-core
requires a &'static Metadata<'static>
.
I've run into this limitation several times and been stumped because it's not possible to dynamically construct a tracing::Metadata
or tracing::Event
due to the use of &'static str
strings.
For a real world example, I've got a WebAssembly module which uses the tracing
crate and a custom Subscriber
for logging and I'd like to pass logged messages to the host's tracing
subscriber. This requires the guest to serialize an Event
so the host can deserialize it and reconstruct the Event
and Metadata
. However, this isn't possible for the host to do because the fields will be String
s defined at runtime and the Event::new()
constructor requires metadata: &'static Metadata<'static>
.
Leaking+caching the strings to get a &'static str
isn't practical because this application is meant to run for long periods of time and the Metadata
is controlled by the guest, which may not be fully trusted. For example, we could have a DOS issue where large numbers of unique Metadata
objects are dynamically created by the guest, and cause the host to OOM.
I kind of have the same issue, where I need a dynamic Metadata
and Event
.
I ended up with the same kind of tricks as tracing-log
, where I have a generic static Metadata
, and I set all dynamic fields as additional fields of the event, only for the subscriber to pop those fields and rebuild a proper Metadata
object.
This is quite ugly and cumbersome though, as this a lot of copying around and requires a custom subscriber to properly handle. I would definitely love to be able to build dynamic objects to solve those issues.
I think this limitation makes it quite hard to follow the open telemetry convention for rpc span names https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/semantic_conventions/rpc.md
Might be able to do something similar where the subscribers rewrite the events before formatting and/or sending to open telemetry.
I ran into this issue again a couple weeks back and created a thread on the Rust User Forums.
It didn't get any responses, but I think the post itself provides some good concrete examples of what people mean when they want to dynamically generate tracing
events.
I'm working on an application which uses the tracing
library for logging, however a different component will generate events at runtime[^1] and as far as I can tell it's not possible to construct a tracing::Event
object because it requires 'static
metadata (the thing tracking log level, target, etc.).
Here is a snippet showing the rough public API I want to expose:
use tracing::field::ValueSet;
pub enum LogLevel {
Trace,
Debug,
Info,
Warn,
Error,
}
pub struct LogMetadata<'a> {
pub name: &'a str,
pub target: &'a str,
pub level: LogLevel,
pub file: Option<&'a str>,
pub line: Option<u32>,
pub module: Option<&'a str>,
}
pub enum LogValue<'a> {
Null,
Boolean(bool),
Integer(i64),
Float(f64),
String(&'a str),
}
pub type LogValueMap<'a> = Vec<(&'a str, LogValue<'a>)>;
fn is_enabled(meta: LogMetadata<'_>) -> bool {
tracing::dispatcher::get_default(|dispatch| {
let meta = tracing::Metadata::from(meta);
dispatch.enabled(&meta)
})
}
fn log(meta: LogMetadata<'_>, message: &str, data: LogValueMap<'_>) {
let meta = tracing::Metadata::from(meta);
let values = value_set(message, &data);
tracing::Event::dispatch(&meta, &values)
}
impl<'a> From<LogMetadata<'a>> for tracing::Metadata<'a> {
fn from(_: LogMetadata<'a>) -> Self {
todo!();
}
}
fn value_set<'a>(message: &'a str, values: &'a [(&'a str, LogValue<'a>)]) -> ValueSet<'a> {
todo!();
}
The compile error shows that Event::dispatch()
requires a &'static Metadata
.
error[E0759]: `meta` has an anonymous lifetime `'_` but it needs to satisfy a `'static` lifetime requirement
--> src/lib.rs:38:40
|
37 | fn log(meta: LogMetadata<'_>, message: &str, data: LogValueMap<'_>) {
| --------------- this data with an anonymous lifetime `'_`...
38 | let meta = tracing::Metadata::from(meta);
| ^^^^ ...is used here...
39 | let values = value_set(message, &data);
40 | tracing::Event::dispatch(&meta, &values)
| ----- ...and is required to live as long as `'static` here
For more information about this error, try `rustc --explain E0759`.
error: could not compile `playground` due to previous error
Warnings
Does anyone have any solutions or workarounds for my problem?
The main thing I care about is having the log level and target in my final log event so that the dynamically generated log messages look just like any other message (styled differently based on levels, filtering by target/level works, etc.).
I could use unsafe
to pretend my data is all 'static
, but that's unsound and I'd prefer to avoid going down that path if possible.
[^1]: In this case, a WebAssembly guest calls my application's log()
function with things like the log level, target, file, message, structured data, and so on. The data is almost identical to the ValueSet
and Metadata
types used when constructing a tracing::Event
, except everything borrows from WebAssembly linear memory instead of being compile time literals.
Non 'static
things are harder because they might require cloning, lowering the performance for everyone, even when they are not using them. So it's understandable that there might not be supported.
However I have a use case, in which I'm happy to leak the string (one per program), but I don't know how to use it as a name for a span!
.
However I have a use case, in which I'm happy to leak the string (one per program), but I don't know how to use it as a name for a
span!
.
The span!
macro requires not just &'static str
s but, specifically, &'static str
s that are constructable in a const
initializer context, because it uses the name to construct a static with a tracing Metadata
struct. So, unfortunately, you can't really use a leaked string as a span name without substantial changes to how the macros work.
One potential option could be to use the dynamic string as the span's message
field, instead. In that case, tracing_subscriber::fmt
will, at least, format it without a field name, as span_name{the message}
instead of span_name{message="the message"}
, which might be a little nicer. But, it's not actually using the string as the span's name.
Sorry.
I appreciate the workaround suggestion. It might work.