[Need Information] Uploading DSPy generated prompt to Opik to assign it to experiment to compare with other experiments.

[ralsofias]

I was wondering if it was possible to upload a DSPy prompt to an experiment produced by an evaluation so we could compare 2 prompts in different experiments. I already know how to upload prompts but the problem is recovering the prompt from DSPy. I was wondering if there was a way (maybe from traces or spans) to obtain the str produced by DPSy with a very basic, almost blank example before executing the evaluation?

[ralsofias]

So far, the workaround I have is to get module.history after 1 execution with a dummy example, get the last message and the system and user message, but i don't know if there is a better way.

_, history = asyncio.run(your_llm_application([HumanMessage(content="Hello")])) # history is module.history

prompt = history[-1]["messages"] # Assign this prompt somehow (is a list of messages with 1 system message, list of demos and then a final user message)

evaluation = evaluate(
    dataset=get_dataset(),
    task=evaluation_task,
    scoring_metrics=[
        SomeMetric()
    ],
    # use prompt or prompts, dont know yet how to do it,
    experiment_config={
        "provider": "provider",
        "model": "model",
        "temperature": None,
        "max_tokens: 123,
    },
    project_name=PROJECT_NAME,
)

[dsblank]

@ralsofias if you want to get a single prompt from a dspy program, perhaps this little function could help:

def get_prompt(program):
    """
    Get the system prompt from the program
    """
    if hasattr(program, "react"):
        instructions = program.react.signature.instructions
    else:
        instructions = program.signature.instructions

    return instructions