AttributeError: 'str' object has no attribute 'items'

[poppingtonic]

Error for example in dev set: 'str' object has no attribute 'inputs'

---

AttributeError                            Traceback (most recent call last)
devset = dl.from_json("/opt/dspy-test/demonstration_dataset_responses.json", fields=["question", "answer", "context"], input_keys=('question'))
dev_set = dspy.Prediction(passages=devset).with_inputs('question')
evaluate = Evaluate(devset=dev_set, num_threads=1, display_progress=True, display_table=5)
evaluate(TestRAG(), metric=llm_metric)

File [/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:139](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:139), in Evaluate.__call__(self, program, metric, devset, num_threads, display_progress, display_table, display, return_all_scores, return_outputs)
    [136](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:136) predicted_devset = sorted(reordered_devset)
    [138](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:138) # data = [{**example, **prediction, 'correct': score} for example, prediction, score in zip(reordered_devset, preds, scores)]
--> [139](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:139) data = [merge_dicts(example, prediction) | {'correct': score} for _, example, prediction, score in predicted_devset]
    [141](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:141) df = pd.DataFrame(data)
    [143](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:143) # Truncate every cell in the DataFrame

File [/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:139](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:139), in <listcomp>(.0)
    [136](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:136) predicted_devset = sorted(reordered_devset)
    [138](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:138) # data = [{**example, **prediction, 'correct': score} for example, prediction, score in zip(reordered_devset, preds, scores)]
--> [139](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:139) data = [merge_dicts(example, prediction) | {'correct': score} for _, example, prediction, score in predicted_devset]
    [141](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:141) df = pd.DataFrame(data)
    [143](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:143) # Truncate every cell in the DataFrame

File [/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:188](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:188), in merge_dicts(d1, d2)
    [186](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:186) def merge_dicts(d1, d2):
    [187](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:187)     merged = {}
--> [188](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:188)     for k, v in d1.items():
    [189](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:189)         if k in d2:
    [190](/opt/dspy-test/venv/lib/python3.10/site-packages/dspy/evaluate/evaluate.py:190)             merged[f"example_{k}"] = v

AttributeError: 'str' object has no attribute 'items'

[arnavsinghvi11]

Hi @poppingtonic , this error happens because your dev_set is not populated correctly with dspy.Example objects. Try following this documentation.

Instead of creating a dspy.Predicion object, you'd want to create a list of dspy.Example objects and then specify the inputs (potentially looking like this):

devset = [Example(**devset[i], dspy_uuid=str(uuid.uuid4())) for i in range(len(devset))]
devset = [x.with_inputs('question') for x in devset]

[poppingtonic]

Hi @arnavsinghvi11 Excellent. This solved it. Now the next error is:

Error for example in dev set: 		 'list' object has no attribute 'passages'

The retriever looks like this: https://github.com/stanfordnlp/dspy/pull/465/files#diff-107f689fa9e6402bc132103823e6d84f36b5c1b95326cf9059a61934e3d50483