jetstream icon indicating copy to clipboard operation
jetstream copied to clipboard

Jetstream validation for population ratio should check numerator & denominator

Open dzeber opened this issue 1 year ago • 0 comments

I recently committed updates to the Navigational Suggestions experiment config in metric-hub. As a part of this I renamed the engaged_search_sessions metric to search_sessions, but I forgot to update the name in the denominator for nav_impression_rate and search_engine_rate.

However, the CI validation passed and the rerun-jetstream job completed successfully. I only noticed the error when I found that the statistics_ tables did not contain the nav_impression_rate and search_engine_rate metrics - looks like they were silently dropped because of the denominator error.

The validation should check the numerator & denominator for population ratio metrics and flag if either of these are not defined.

┆Issue is synchronized with this Jira Task

dzeber avatar Jul 20 '23 23:07 dzeber