Anna Scholtz
Anna Scholtz
Pulling in experiments from the Experimenter API takes about 5 minutes. We already import most of this data into the `moz-fx-data-experiments.monitoring.experimenter_experiments_v1` table. This table gets updated every 10 minutes. Pulling...
Currently, each analysis run for each experiment and analysis date calls the Experimenter API to fetch all experiments. With 50 runs or more running in parallel which is more requests...
It would be helpful for debugging if we'd print out some additional information for each experiment, like the dates when weekly and overall results are expected as well as end...
The ETL for retrieving cost information about the queries jetstream is running seems to be lagging more than one day behind. This is likely just a scheduling issue in Airflow...
There are certain types of experiment where as much information as possible needs to be kept confidential and/or confidential data is accessed for running the analysis. Jetstream currently does not...
Currently, all clients that enrolled during the enrollment period will be part of all analysis runs, even if the clients have unenrolled. Ideally, we want to exclude unenrolled clients. Excluding...
> [these cutoffs](https://github.com/mozilla/looker-spoke-default/blob/main/experimentation/views/sample_ratio_mismatch.view.lkml#L12-L54) are dependent on the number of branches in the experiment. These are from a 2-branch experiment but for 3 (or more) branch experiments they give the wrong...
Follow-up to https://mozilla.slack.com/archives/C03JK372UF6/p1654872557375169 There are certain exceptions (like `NoStartDateException`) that should be treated as warning instead of as an error. The jetstream error dashboard has a list of exceptions that...
Currently, validation fails when an experiment has not been launched. It would be useful to validate the SQL query even before the experiment is live. The tricky part here is...
It might make sense to add an option to experiment configs which allow to list the outcomes that should be used for the experiment. Currently, outcomes can only be specified...