matanor
matanor
Today confidence intervals are computed by default for the main_score. This [PR](https://github.com/IBM/unitxt/pull/431) adds the capability of computing confidence intervals for additional scores. We would like to change the confidence interval...
Computing of confidence intervals for `GlobelMetric` objects ([here](https://github.com/IBM/unitxt/blob/main/src/unitxt/metrics.py#L115C6-L115C6)) requires the recalculation of the metric multiple times, depending on the [n_resamples](https://github.com/IBM/unitxt/blob/main/src/unitxt/metrics.py#L62) parameter. This recalculation may be costly in runtime for some...
Requirement: support an `ibm_watsonx_ai.client.APIClient` with specially defined headers within an inference engine. Motivation: This is required for the use of such inference engines within `LLMaaJ` based metrics.