How can I delay the start of llm evaluation within the time frame I request？

[lucas-zan]

Due to the limited computing power of the machine, model resources are limited. I hope to be able to track model calls in real time, but the model related calls evaluated online will be executed in a queue within the time frame I set.

[alexkuzmik]

Hi @lucas-zan! We have opik.evaluate_experiment function which you can use to compute extra metrics for already existing Opik experiment, you may call it with a different set of metrics whenever you want after creating initial evaluation with opik.evaluate.

If it doesn't help could you please share an example script showing how the API for that could look like in theory with some more explanations? At the moment we don't have any ways to schedule the metric calculations during the evaluation.

[vincentkoc]

@lucas-zan we were unable to get a response from you so will be closing this issue, feel free to re-open this PR if you have any on-going issues and we willnbe more than happy to help you out.