dataloader
dataloader copied to clipboard
graph-gophers
Tracing interface makes proper use of Links impossible
Dataloaders present a unique challenge with tracing, as batching is one of the few instances where the invariant that a span has a single parent is broken. Many different parent spans end up producing just a single child span which performs the operation for all of them.
To support this, most tracing standards have a concept of Links(Open Telemetry). Where a span can specify that is has links to more than one parent span. This is important when trying to compute metrics like %time spent in operation. Without this, only one of the many parent spans gets its time discounted from the overall operation time as the rest have no child span.
In order for this to work, the batch function, when creating the span for its batch work, must have access to the span (context) of each of the calls to Load(), or LoadMany() which is currently not possible given the existing interface / code architecture. I'm curious to hear if this is not a problem for anyone else using the current tracing architecture.
Hi. After concatenating the id, your batch function will make 1 call for multiple keys. You will get the same picture, but one level lower. All functions will have the same time.
For your idea u can use something struct as key, like this
type Key struct {
ctx context.Context
raw interface{}
}
func ContextKey(ctx context.Context, key interface{}) Key {
return contextKey{ctx: ctx, raw: key}
}
func (c contextKey) Context() context.Context {
return c.ctx
}
func (c contextKey) Raw() interface {
return c.raw
}
or
type Key[K comparable] struct {
ctx context.Context
raw K
}
func ContextKey[K comparable](ctx context.Context, key K) Key[K] {
return Key[K]{ctx:ctx, raw:key}
}
func (k Key[K]) Context() context.Context {
return c.ctx
}
func (k Key[K]) Raw() K {
return c.raw
}
@DylanRJohnston-FZ
Hi, I write a new dataloader package, support trace with opentelemetry links. Let me know if it works for you :-)
- https://github.com/sysulq/dataloader-go
