AttributeError: 'Predict' object has no attribute 'extended_signature1'

[younes-io]

I'm trying to run the BayesianSignatureOptimizer to compile my pipeline but it fails with the below error:

from dspy.teleprompt import BayesianSignatureOptimizer

model_gpt4=config['MODEL_NAME_GPT4']

llm_prompter = dspy.OpenAI(deployment_id=model_gpt4,
                    api_key=openai_api_key,
                    api_base=azure_endpoint,
                    api_provider="azure",
                    model_type="chat",
                    api_version=openai_api_version,
                    max_tokens=2000)

teleprompter = BayesianSignatureOptimizer(task_model=dspy.settings.lm,
                                          prompt_model=llm_prompter,
                                          metric=llm_metric,
                                          n=5,
                                          verbose=False)

kwargs = dict(num_threads=1, display_progress=True, display_table=0)

bayesian_compiled_classifier = teleprompter.compile(MyModuleClassifier(), devset=devset,
                                                optuna_trials_num=3,
                                                max_bootstrapped_demos=4,
                                                max_labeled_demos=4,
                                                eval_kwargs=kwargs)

bayesian_compiled_classifier.save(path="saves/BayesianSignatureOptimizer.json")

it just throws this:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[133], [line 23](vscode-notebook-cell:?execution_count=133&line=23)
     [15](vscode-notebook-cell:?execution_count=133&line=15) teleprompter = BayesianSignatureOptimizer(task_model=dspy.settings.lm,
     [16](vscode-notebook-cell:?execution_count=133&line=16)                                           prompt_model=llm_prompter,
     [17](vscode-notebook-cell:?execution_count=133&line=17)                                           metric=llm_metric,
     [18](vscode-notebook-cell:?execution_count=133&line=18)                                           n=5,
     [19](vscode-notebook-cell:?execution_count=133&line=19)                                           verbose=False)
     [21](vscode-notebook-cell:?execution_count=133&line=21) kwargs = dict(num_threads=1, display_progress=True, display_table=0)
---> [23](vscode-notebook-cell:?execution_count=133&line=23) bayesian_compiled_classifier = teleprompter.compile(MyModuleClassifier(), devset=devset,
     [24](vscode-notebook-cell:?execution_count=133&line=24)                                                 optuna_trials_num=3,
     [25](vscode-notebook-cell:?execution_count=133&line=25)                                                 max_bootstrapped_demos=4,
     [26](vscode-notebook-cell:?execution_count=133&line=26)                                                 max_labeled_demos=4,
     [27](vscode-notebook-cell:?execution_count=133&line=27)                                                 eval_kwargs=kwargs)
     [29](vscode-notebook-cell:?execution_count=133&line=29) bayesian_compiled_classifier.save(path="saves/BayesianSignatureOptimizer.json")

File [~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:311](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:311), in BayesianSignatureOptimizer.compile(self, student, devset, optuna_trials_num, max_bootstrapped_demos, max_labeled_demos, eval_kwargs, seed, view_data, view_examples)
    [308](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:308)             demo_candidates[id(module_p)].append(candidate_p.demos)
    [310](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:310) # Generate N candidate prompts
--> [311](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:311) instruction_candidates, _ = self._generate_first_N_candidates(module, self.n, view_data, view_examples, demo_candidates, devset)
    [313](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:313) # Initialize variables to store the best program and its score
    [314](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:314) best_score = float('-inf')

File [~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:210](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:210), in BayesianSignatureOptimizer._generate_first_N_candidates(self, module, N, view_data, view_examples, demo_candidates, devset)
    [208](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:208)     basic_prefix = predictor.extended_signature.fields[-1].name
    [209](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:209) else:
--> [210](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:210)     basic_instruction = predictor.extended_signature1.instructions
    [211](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:211)     basic_prefix = predictor.extended_signature1.fields[-1].name
    [212](https://vscode-remote+wsl-002bubuntu.vscode-resource.vscode-cdn.net/home/me/dev/whatever/experiments/~/.pyenv/versions/3.11.6/lib/python3.11/site-packages/dspy/teleprompt/signature_opt_bayesian.py:212) if self.prompt_model: 

AttributeError: 'Predict' object has no attribute 'extended_signature1'

[younes-io]

I think the issue is due to https://github.com/stanfordnlp/dspy/blob/f89b875523b31817adc9adf0d791c8621df19d47/dspy/teleprompt/signature_opt_bayesian.py#L115

The code doesn't check whether extended_signature1 exists in the predictor object as it does for extended_signature. It just assumes it exists; as far as I know, the attribute extended_signature1 exists only in ChainOfThoughtWithHint.

[thomasahle]

Weird. Predict should have an extended signature, so it shouldn't go into the branch mentioning extended signature 1

[okhat]

I think Predict only has signature? Anyway this whole set of signatures is probably going to need to either be removed or at least standardize .signature or .signature() as the public interface for all modules.

Until we do that, @younes-io can you use ChainOfThought? It’s almost always a good idea compared to Predict unless you need multiple output fields

[younes-io]

@okhat yes, Predict only has signature; it has no extended_signatures. I'll switch one of the modules to ChainOfThough as you mentioned until this is solved.

[younes-io]

I think we should have a clear description of how signatures should be defined/handled/composed and avoid having attributes like extended_signature, extended_signature1 and extended_signature2

I can help in the PR if this is clarified

[okhat]

Agreed. Only Predict should actually worry about managing and storing signatures

[neoxelox]

Just an addition; If extended signatures are going to be refactored, can we change how they are extended from using dsp.Types to dspy.Input/OutputFields, so it's more clear whether the new fields are input or output?

[okhat]

Yes. I agree. We start a refactor on Friday this week or on Thursday next week

[okhat]

Maybe I should make an issue with a plan for refactor

[thomasahle]

In my PR extended_signature is a Signature object, not a type (https://github.com/thomasahle/dspy/blob/pydantic/dspy/predict/chain_of_thought.py#L45)

It works by just converting to a dsp.Template right before calling generate (https://github.com/thomasahle/dspy/blob/pydantic/dspy/predict/predict.py#L68)

[younes-io]

From what I understood, a signature is a specification; if we take ChainOfThought, the signature appears to be question -> answer, but internally, it's question -> rationale, answer, so:

• we could have/keep self.signature to be used by consumers / the external world
• and self._signature (which is a computation of self.signature) to be used internally for all the DSPy magic

[younes-io]

I noticed that for ChainOfThoughtWithHint, both extended_signature1 and extended_signature2 are used. Unless I'm missing something, I don't think we need different "versions" of the signature. As long as we have an "internal" signature, internal consumers can use it to compute it with other params and do their job. It would act as the single source of truth, instead of having several versions all over the place