Feature request - correlation out-of-the box

[stijnmoreels]

Is your feature request related to a problem? Please describe. Provide support for out-of-the-box correlation on both the operation & transaction level.

Every request should allow consumers to specify a transaction id by passing the X-Transaction-Id header. If this is not specified, one will be generated.

Internally this transaction id and operation id can be used internally for correlating telemetry.

Every response should return the used correlation information via the following headers:

X-Transaction-Id - The generated or specified transaction id RequestId - The unique id that identifies this operation. This information should also be able to be retrieved further down the application if they wish to use it.

Describe the solution you'd like Middleware should be provided that handles this for every call, and registers the information for others to use.

/// Functionality to correlate HTTP requests with one another.
[<AutoOpen>]
module Correlate =
  /// Correlation type containing the identifiers of the HTTP request.
  type Correlation =
    { /// The unique ID for this request.
      OperationId : string
      /// The ID that makes this request part of the same transaction/set of requests.
      TransactionId : string }

  type HttpContext with
    /// Gets the unique ID for this request.
    member this.OperationId = this.Features.Get<Correlation>().OperationId
    /// Gets the ID that makes this request part of the same transaction/set of requests.
    member this.TransactionId = this.Features.Get<Correlation>().TransactionId

  /// The user-defined correlation options to control how the correlation should happen.
  type CorrelationOptions =
    { /// The request/response header where the operation ID is located, default: `X-Operation-ID`.
      OperationIdHeader : string
      /// The request/response header where the transaction ID is located, default: `X-Transaction-ID`.
      TransactionIdHeader : string 
      /// Indicates whether or not the correlation ID's should be present in the response, default: `true`.
      IncludeInResponse : bool
      /// The function to generate a unique operation ID for the request.
      GenerateOperationId : unit -> string
      /// The function to generate a transaction ID for the request when no `X-Transaction-ID` is present in the request.
      GenerateTransactionId : unit -> string } with
        static member Default =
          { OperationIdHeader = "X-Operation-ID"
            TransactionIdHeader = "X-Transaction-ID"
            IncludeInResponse = true
            GenerateOperationId = Guid.NewGuid().ToString
            GenerateTransactionId = Guid.NewGuid().ToString }

  /// Provides correlation on request/operation level, and transaction/set-of-requests level
  /// with a set of user-defined options to control the correlation.
  let correlateWith setOptions =
    fun next (ctx : HttpContext) -> 
      let options = setOptions CorrelationOptions.Default
      let operationId = 
        Option.ofObj ctx.TraceIdentifier
        |> Option.defaultWith options.GenerateOperationId
      let transactionId =
        ctx.TryGetRequestHeader options.TransactionIdHeader
        |> Option.defaultWith options.GenerateTransactionId
      
      let c = { OperationId = operationId; TransactionId = transactionId }
      ctx.Features.Set c
      if options.IncludeInResponse then
        ctx.Response.OnStarting ((fun state -> 
          let ctx = state :?> HttpContext
          ctx.Response.Headers.Add (options.OperationIdHeader, StringValues c.OperationId)
          ctx.Response.Headers.Add (options.TransactionIdHeader, StringValues c.TransactionId)
          Task.CompletedTask), ctx)
      next ctx

  /// Provides correlation on request/operation level, and transaction/set-of-requests level
  /// by using the `X-Operation-ID` and `X-Transaction-ID` respectively.
  let correlate = correlateWith id

This way, correlation can be added easily to the pipeline without any effort. Only not sure if this is something we want in the Core library, or if this is too much 'application-specific' and should exists in another package/library.

[NinoFloris]

Hi @stijnmoreels

Thanks for creating the issue.

This might interest you https://devblogs.microsoft.com/aspnet/improvements-in-net-core-3-0-for-troubleshooting-and-monitoring-distributed-apps/

[stijnmoreels]

This might interest you https://devblogs.microsoft.com/aspnet/improvements-in-net-core-3-0-for-troubleshooting-and-monitoring-distributed-apps/

That's nice! So we may not need a dedicated correlation approach in Giraffe.

Super library, btw. I maybe suggest some other functionality related to max request body size, request header validation, authentication, ... but not sure if that's really something that would help Giraffe. Maybe to 'application-specific'. idk.

[dustinmoris]

@stijnmoreels Does the link provided by @NinoFloris solve this particular issue or is there still a feature request in here worth discussing?

[stijnmoreels]

I think it's still an option to discus this. But it;s not a problem to discard it either. It seems helpful to have some basic correlation handlers here.

[xperiandri]

Is there a sample that shows Activity usage in a functional way?