dspy icon indicating copy to clipboard operation
dspy copied to clipboard

List index out of range error after passing long text in input fields

Open aparnajore opened this issue 11 months ago • 7 comments

I am using question->answer signature and trying to generate output fields by passing inputs text in input fields. And trying to use chainofthought to generate response. But it's working fine for small size input text and fails after passing long input text and gives list index out of range error

aparnajore avatar Mar 02 '24 18:03 aparnajore

Using lm as a gemini-1.0-pro. The output generated by dspy chainofthought having rationale field and in that field I want to print only question and reasoning but getting all input fields again in the prediction. Please help me resolve this and generate response using chainofthought with input fields having long text.

aparnajore avatar Mar 02 '24 18:03 aparnajore

Thank you for opening this @aparnajore b Do you mind sharing the full error trace? And a minimal example of what causes the error

okhat avatar Mar 02 '24 19:03 okhat

Thank you very much for quick response @okhat, I am facing below mentioned error:

IndexError Traceback (most recent call last) Cell In[18], line 3 1 question="Department of employee having Emp no. 7369?" 2 answer = dspy.ChainOfThought(question_answer) ----> 3 prediction = answer(input_data_1=input_data_1, input_data_2=input_data_2, question = question) 4 prediction

File ~\dspy\dspy\predict\predict.py:49, in Predict.call(self, **kwargs) 48 def call(self, **kwargs): ---> 49 return self.forward(**kwargs)

File ~\dspy\dspy\predict\chain_of_thought.py:60, in ChainOfThought.forward(self, **kwargs) 58 signature = new_signature 59 # template = dsp.Template(self.signature.instructions, **new_signature) ---> 60 return super().forward(signature=signature, **kwargs)

File ~\dspy\dspy\predict\predict.py:90, in Predict.forward(self, **kwargs) 87 template = signature_to_template(signature) 89 if self.lm is None: ---> 90 x, C = dsp.generate(template, **config)(x, stage=self.stage) 91 else: 92 # Note: query_only=True means the instructions and examples are not included. 93 # I'm not really sure why we'd want to do that, but it's there. 94 with dsp.settings.context(lm=self.lm, query_only=True):

File ~\dspy\dsp\primitives\predict.py:77, in _generate..do_generate(example, stage, max_depth, original_example) 75 # Generate and extract the fields. 76 prompt = template(example) ---> 77 completions: list[dict[str, Any]] = generator(prompt, **kwargs) 78 completions: list[Example] = [template.extract(example, p) for p in completions] 80 # Find the completions that are most complete.

File ~\dspy\dsp\modules\google.py:159, in Google.call(self, prompt, only_completed, return_sorted, **kwargs) 157 for i in range(n): 158 response = self.request(prompt, **kwargs) --> 159 completions.append(response.parts[0].text) 161 return completions

File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\proto\marshal\collections\repeated.py:125, in RepeatedComposite.getitem(self, key) 124 def getitem(self, key): --> 125 return self._marshal.to_python(self._pb_type, self.pb[key])

IndexError: list index out of range

When length of input_data_1 and input_data_2 is around 20k characters then its giving above error.

aparnajore avatar Mar 03 '24 17:03 aparnajore

Hello @okhat, any updates on this?

aparnajore avatar Mar 04 '24 04:03 aparnajore

It seems to come from the google models, whose details I don’t know about. Tagging @CShorten in case he has an insight into this

okhat avatar Mar 04 '24 04:03 okhat

Thank you for adding support for new models! I had the same issue when I walked through the tutorial with gemini model. Was there any update on this?

stlkcmrd avatar Mar 21 '24 04:03 stlkcmrd

Not super helpful but quickfix for me was to navigate to \dspy\dsp\modules\google.py around line 158-160 and add a try catch.

Go from this:

for i in range(n): response = self.request(prompt, **kwargs) completions.append(response.parts[0].text)

to this:

for i in range(n): response = self.request(prompt, **kwargs) try: completions.append(response.parts[0].text) except: print("Failed here")

Try this fix before saying it wont work :). I found that n was 2 for some reason and the second iteration was empty so it has no parts but the first iteration had all the content I wanted.

Tabbing is all outta wack but you can fix that on your end

hodgepodge73 avatar Apr 17 '24 20:04 hodgepodge73

Hi there, I am running into the same problems using Llama 3 8B.

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[36], [line 5](vscode-notebook-cell:?execution_count=36&line=5)
      [3](vscode-notebook-cell:?execution_count=36&line=3) teleprompter = MIPROv2(metric=validate_answer, num_candidates=2, prompt_model=llm_long_output, task_model=llm_short_output)
      [4](vscode-notebook-cell:?execution_count=36&line=4) kwargs = dict(num_threads=1, display_progress=True, display_table=0)
----> [5](vscode-notebook-cell:?execution_count=36&line=5) compiled_prompt_opt = teleprompter.compile(
      [6](vscode-notebook-cell:?execution_count=36&line=6)     report_classification_suggestions,
      [7](vscode-notebook-cell:?execution_count=36&line=7)     trainset=trainset, num_batches=10, max_bootstrapped_demos=2, max_labeled_demos=6, eval_kwargs=kwargs, requires_permission_to_run=False)
      [8](vscode-notebook-cell:?execution_count=36&line=8) eval_score = evaluate(compiled_prompt_opt, devset=evalset, **kwargs)

File ~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:287, in MIPROv2.compile(self, student, trainset, valset, num_batches, max_bootstrapped_demos, max_labeled_demos, eval_kwargs, seed, minibatch, program_aware_proposer, requires_permission_to_run)
    [285](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:285) proposer.use_instruct_history = False
    [286](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:286) proposer.set_history_randomly = False
--> [287](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:287) instruction_candidates = proposer.propose_instructions_for_program(
    [288](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:288)     trainset=trainset,
    [289](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:289)     program=program,
    [290](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:290)     demo_candidates=demo_candidates,
    [291](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:291)     N=self.n,
    [292](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:292)     prompt_model=self.prompt_model,
    [293](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:293)     T=self.init_temperature,
    [294](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:294)     trial_logs={},
    [295](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:295) )
    [296](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:296) for i, pred in enumerate(program.predictors()):
    [297](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/teleprompt/mipro_optimizer_v2.py:297)     instruction_candidates[i][0] = get_signature(pred).instructions

File ~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:302, in GroundedProposer.propose_instructions_for_program(self, trainset, program, demo_candidates, prompt_model, trial_logs, N, T, tip)
    [299](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:299)         if pred_i not in proposed_instructions:
    [300](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:300)             proposed_instructions[pred_i] = []
    [301](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:301)         proposed_instructions[pred_i].append(
--> [302](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:302)             self.propose_instruction_for_predictor(
    [303](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:303)                 program=program,
    [304](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:304)                 predictor=predictor,
    [305](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:305)                 pred_i=pred_i,
    [306](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:306)                 prompt_model=prompt_model,
    [307](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:307)                 T=T,
    [308](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:308)                 demo_candidates=demo_candidates,
    [309](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:309)                 demo_set_i=demo_set_i,
    [310](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:310)                 trial_logs=trial_logs,
    [311](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:311)                 tip=selected_tip,
    [312](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:312)             ),
    [313](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:313)         )
    [314](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:314) return proposed_instructions

File ~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:349, in GroundedProposer.propose_instruction_for_predictor(self, program, predictor, pred_i, prompt_model, T, demo_candidates, demo_set_i, trial_logs, tip)
    [347](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:347) with dspy.settings.context(lm=prompt_model):
    [348](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:348)     prompt_model.kwargs["temperature"] = T
--> [349](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:349)     proposed_instruction = instruction_generator.forward(
    [350](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:350)         demo_candidates=demo_candidates,
    [351](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:351)         pred_i=pred_i,
    [352](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:352)         demo_set_i=demo_set_i,
    [353](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:353)         program=program,
    [354](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:354)         data_summary=self.data_summary,
    [355](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:355)         previous_instructions=instruction_history,
    [356](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:356)         tip=tip,
    [357](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:357)     ).proposed_instruction
    [358](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:358) prompt_model.kwargs["temperature"] = original_temp
    [360](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:360) # Log the trace used to generate the new instruction, along with the new instruction itself

File ~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:201, in GenerateModuleInstruction.forward(self, demo_candidates, pred_i, demo_set_i, program, previous_instructions, data_summary, max_demos, tip)
    [199](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:199)     matches = re.findall(pattern, init_content, re.MULTILINE)
    [200](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:200)     modules = [match[0].strip() for match in matches]
--> [201](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:201)     module_code = modules[pred_i]
    [203](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:203) module_description = self.describe_module(
    [204](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:204)     program_code=self.program_code_string,
    [205](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:205)     program_description=program_description,
   (...)
    [208](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:208)     max_depth=10,
    [209](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:209) ).module_description
    [211](https://vscode-remote+ssh-002dremote-002bhermes-002ehpc-002edhclab-002ei-002ehpi-002ede.vscode-resource.vscode-cdn.net/dhc/home/johann.lieberwirth/mp_lippert_2024/report_pipeline/~/conda3/envs/mp2024/lib/python3.10/site-packages/dspy/propose/grounded_proposer.py:211) # Generate an instruction for our chosen module

IndexError: list index out of range```

JohannKaspar avatar Jul 18 '24 12:07 JohannKaspar

Follow up: I tried to find the error and in grounded_proposer.py I ran into this funny looking string:

self.describe_program(
    program_code=self.program_code_string, program_example=task_demos,
    ).program_description

'the MOVE rational with conditional wors glimps words 위치 yiwo opaque ################################################################################'

self.program_code_string

'ClassifyBodyPart(report -> body_part\n instructions='Classify the body part on which the radiology report was written on and provide the body part as output.'\n report = Field(annotation=str required=True json_schema_extra={'desc': 'The radiology report to be classified.', '__dspy_field_type': 'input', 'prefix': 'Report:'})\n body_part = Field(annotation=str required=True json_schema_extra={'desc': "One of the following values: ['abdomen' 'carotic_arteries' 'chest' 'head' 'kidneys'\n 'liver_or_gallbladder' 'lower_extremity' 'spine']", '__dspy_field_type': 'output', 'prefix': 'Body Part:'})\n)\n\nClassifyBodyPart(report -> body_part\n instructions='Classify the body part on which the radiology report was written on and provide the body part as output.'\n report = Field(annotation=str required=True json_schema_extra={'desc': 'The radiology report to be classified.', '__dspy_field_type': 'input', 'prefix': 'Report:'})\n body_part = Field(annotation=str required=True json_schema_extra={'desc': "One of the following values: ['abdomen' 'carotic_arteries' 'chest' 'head' 'kidneys'\n 'liver_or_gallbladder' 'lower_extremity' 'spine']", '__dspy_field_type': 'output', 'prefix': 'Body Part:'})\n)\n\n\n\nclass Predict(Module, Parameter):\n def init(self, signature, **config):\n self.stage = random.randbytes(8).hex()\n self.signature = ensure_signature(signature)\n self.config = config\n self.reset()\n\n def reset(self):\n self.lm = None\n self.traces = []\n self.train = []\n self.demos = []\n\n def dump_state(self):\n state_keys = ["lm", "traces", "train"]\n state = {k: getattr(self, k) for k in state_keys}\n\n state["demos"] = []\n for demo in self.demos:\n demo = demo.copy()\n\n for field in demo:\n if isinstance(demo[field], BaseModel):\n demo[field] = demo[field].model_dump_json()\n\n state["demos"].append(demo)\n\n # Cache the signature instructions and the last field's name.\n *, last_key = self.signature.fields.keys()\n state["signature_instructions"] = self.signature.instructions\n state["signature_prefix"] = self.signature.fields[last_key].json_schema_extra["prefix"]\n\n # Some special stuff for CoT.\n if hasattr(self, "extended_signature"):\n # Cache the signature instructions and the last field's name.\n state["extended_signature_instructions"] = self.extended_signature.instructions\n state["extended_signature_prefix"] = self.extended_signature.fields[last_key].json_schema_extra['prefix']\n\n return state\n\n def load_state(self, state):\n for name, value in state.items():\n setattr(self, name, value)\n\n # Reconstruct the signature.\n if "signature_instructions" in state:\n instructions = state["signature_instructions"]\n self.signature = self.signature.with_instructions(instructions)\n\n if "signature_prefix" in state:\n prefix = state["signature_prefix"]\n *, last_key = self.signature.fields.keys()\n self.signature = self.signature.with_updated_fields(last_key, prefix=prefix)\n \n # Some special stuff for CoT.\n if "extended_signature_instructions" in state:\n instructions = state["extended_signature_instructions"]\n self.extended_signature = self.extended_signature.with_instructions(instructions)\n\n if "extended_signature_prefix" in state:\n prefix = state["extended_signature_prefix"]\n *_, last_key = self.extended_signature.fields.keys()\n self.extended_signature = self.extended_signature.with_updated_fields(last_key, prefix=prefix)\n\n def call(self, **kwargs):\n return self.forward(**kwargs)\n\n def forward(self, **kwargs):\n assert not dsp.settings.compiling, "It's no longer ever the case that .compiling is True"\n\n # Extract the three privileged keyword arguments.\n new_signature = ensure_signature(kwargs.pop("new_signature", None))\n signature = ensure_signature(kwargs.pop("signature", self.signature))\n demos = kwargs.pop("demos", self.demos)\n config = dict(**self.config, **kwargs.pop("config", {}))\n\n # Get the right LM to use.\n lm = kwargs.pop("lm", self.lm) or dsp.settings.lm\n assert lm is not None, "No LM is loaded."\n\n # If temperature is 0.0 but its n > 1, set temperature to 0.7.\n temperature = config.get("temperature")\n temperature = lm.kwargs["temperature"] if temperature is None else temperature\n\n num_generations = config.get("n")\n if num_generations is None:\n num_generations = lm.kwargs.get("n", lm.kwargs.get("num_generations", 1))\n\n if (temperature is None or temperature <= 0.15) and num_generations > 1:\n config["temperature"] = 0.7\n # print(f"#> Setting temperature to 0.7 since n={num_generations} and prior temperature={temperature}.")\n\n if new_signature is not None:\n signature = new_signature\n\n if not all(k in kwargs for k in signature.input_fields):\n present = [k for k in signature.input_fields if k in kwargs]\n missing = [k for k in signature.input_fields if k not in kwargs]\n print(f"WARNING: Not all input fields were provided to module. Present: {present}. Missing: {missing}.")\n\n if dsp.settings.experimental:\n completions = new_generate(lm, signature, dsp.Example(demos=demos, **kwargs), **config)\n else:\n completions = old_generate(demos, signature, kwargs, config, self.lm, self.stage)\n\n pred = Prediction.from_completions(completions, signature=signature)\n\n if kwargs.pop("_trace", True) and dsp.settings.trace is not None:\n trace = dsp.settings.trace\n trace.append((self, {**kwargs}, pred))\n\n return pred\n\n def update_config(self, **kwargs):\n self.config = {**self.config, **kwargs}\n\n def get_config(self):\n return self.config\n\n def repr(self):\n return f"{self.class.name}({self.signature})"\n\n\nclass Retry(Predict):\n def init(self, module):\n super().init(module.signature)\n self.module = module\n self.original_signature = module.extended_signature if isinstance(module, dspy.ChainOfThought) else module.signature\n self.original_forward = module.forward\n self.new_signature = self._create_new_signature(self.original_signature)\n\n def create_new_signature(self, signature):\n # Add "Past" input fields for each output field\n for key, value in signature.output_fields.items():\n actual_prefix = value.json_schema_extra["prefix"].split(":")[0] + ":"\n signature = signature.append(f"past{key}", dspy.InputField(\n prefix="Previous " + actual_prefix,\n desc=f"past {actual_prefix} with errors",\n format=value.json_schema_extra.get("format"),\n ))\n\n signature = signature.append("feedback", dspy.InputField(\n prefix="Instructions:",\n desc="Some instructions you must satisfy",\n format=str,\n ))\n\n return signature\n\n def forward(self, *, past_outputs, **kwargs):\n # Take into account the possible new signature, as in TypedPredictor\n new_signature = kwargs.pop("new_signature", None)\n if new_signature:\n self.original_signature = new_signature\n self.new_signature = self.create_new_signature(self.original_signature)\n\n # Convert the dict past_outputs={"answer": ...} to kwargs\n # {past_answer=..., ...}\n for key, value in past_outputs.items():\n past_key = f"past{key}"\n if past_key in self.new_signature.input_fields:\n kwargs[past_key] = value\n # Tell the wrapped module to use the new signature.\n # Note: This only works if the wrapped module is a Predict or ChainOfThought.\n kwargs["new_signature"] = self.new_signature\n return self.original_forward(**kwargs)\n\n def call(self, **kwargs):\n cached_kwargs = copy.deepcopy(kwargs)\n kwargs["_trace"] = False\n kwargs.setdefault("demos", self.demos if self.demos is not None else [])\n\n # perform backtracking\n if dspy.settings.backtrack_to == self:\n for key, value in dspy.settings.backtrack_to_args.items():\n kwargs.setdefault(key, value)\n pred = self.forward(**kwargs)\n else:\n pred = self.module(**kwargs)\n\n # now pop multiple reserved keys\n # NOTE(shangyin) past_outputs seems not useful to include in demos,\n # therefore dropped\n for key in ["_trace", "demos", "signature", "new_signature", "config", "lm", "past_outputs"]:\n kwargs.pop(key, None)\n\n if dsp.settings.trace is not None:\n trace = dsp.settings.trace\n trace.append((self, {**kwargs}, pred))\n return pred\n\n\nclass ReportClassification(dspy.Module):\n def init(self):\n super().init()\n self.generate_answer = dspy.Predict(ClassifyBodyPart)\n \n def forward(self, report):\n pred = self.generate_answer(report=report)\n \n """ dspy.Suggest(\n len(pred.body_part) < 50,\n "Answer is too long"\n ) """\n dspy.Suggest(\n evaluate_bodypart(pred.body_part),\n f"Answer should be one of the following elements: {body_parts}"\n )\n #return dspy.Prediction(answer=pred.answer)\n return pred\n'

JohannKaspar avatar Jul 18 '24 12:07 JohannKaspar