haystack icon indicating copy to clipboard operation
haystack copied to clipboard

Add target persona meta information to logs

Open tstadel opened this issue 2 years ago • 0 comments

As a datascientist I don't want my logs to be cluttered with devops infos.

Is your feature request related to a problem? Please describe. Currently we do not separate logs according to the persona. devops infos (e.g. availability of multiprocessing, numba, number of worker processes, telemetry) dilutes the logs a data scientist is interested in. Same applies for devops logs regarding data scientist infos (e.g. warnings about which similarity metric to use for some models).

Describe the solution you'd like We add extra haystack_persona to all logs:

  • value datascience marks logs for data scientists
  • value devops marks logs for devops/data engineers
  • Multiple values are allowed. The type of haystack_persona is an array of values.

Describe alternatives you've considered Another distinction could be (user-facing vs. backend-applications) but although this has almost the same distribution it is way harder to grasp for notebook users.

Additional context Main focus lies on bigger deployments using analytics and monitoring tools and a separate UI. Notebook users should also be able to make use of it. The value must be easily filtered for in standard logging and structlog.

tstadel avatar Aug 09 '22 08:08 tstadel