llama_index [Bug]: ValueError: The following `model_kwargs` are not used by the model: ['token_type_ids'] (note: typos in the generate arguments will also show up in this list)

[Bug]: ValueError: The following `model_kwargs` are not used by the model: ['token_type_ids'] (note: typos in the generate arguments will also show up in this list)

Open vincent507cpu opened this issue 1 month ago • 2 comments

Bug Description

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[32], [line 3](vscode-notebook-cell:?execution_count=32&line=3)
      [1](vscode-notebook-cell:?execution_count=32&line=1) query_engine = index.as_query_engine(similarity_top_k=3)
----> [3](vscode-notebook-cell:?execution_count=32&line=3) response = query_engine.query('What year was Elizabeth Matory the opponent of Charles Albert Ruppersberger?')

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274, in Dispatcher.span.<locals>.wrapper(func, instance, args, kwargs)
    [270](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:270) self.span_enter(
    [271](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:271)     id_=id_, bound_args=bound_args, instance=instance, parent_id=parent_id
    [272](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:272) )
    [273](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:273) try:
--> [274](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274)     result = func(*args, **kwargs)
    [275](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:275) except BaseException as e:
    [276](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:276)     self.event(SpanDropEvent(span_id=id_, err_str=str(e)))

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/base/base_query_engine.py:53, in BaseQueryEngine.query(self, str_or_query_bundle)
     [51](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/base/base_query_engine.py:51)     if isinstance(str_or_query_bundle, str):
     [52](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/base/base_query_engine.py:52)         str_or_query_bundle = QueryBundle(str_or_query_bundle)
---> [53](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/base/base_query_engine.py:53)     query_result = self._query(str_or_query_bundle)
     [54](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/base/base_query_engine.py:54) dispatch_event(QueryEndEvent(query=str_or_query_bundle, response=query_result))
     [55](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/base/base_query_engine.py:55) return query_result

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274, in Dispatcher.span.<locals>.wrapper(func, instance, args, kwargs)
    [270](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:270) self.span_enter(
    [271](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:271)     id_=id_, bound_args=bound_args, instance=instance, parent_id=parent_id
    [272](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:272) )
    [273](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:273) try:
--> [274](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274)     result = func(*args, **kwargs)
    [275](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:275) except BaseException as e:
    [276](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:276)     self.event(SpanDropEvent(span_id=id_, err_str=str(e)))

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/query_engine/retriever_query_engine.py:190, in RetrieverQueryEngine._query(self, query_bundle)
    [186](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/query_engine/retriever_query_engine.py:186) with self.callback_manager.event(
    [187](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/query_engine/retriever_query_engine.py:187)     CBEventType.QUERY, payload={EventPayload.QUERY_STR: query_bundle.query_str}
    [188](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/query_engine/retriever_query_engine.py:188) ) as query_event:
    [189](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/query_engine/retriever_query_engine.py:189)     nodes = self.retrieve(query_bundle)
--> [190](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/query_engine/retriever_query_engine.py:190)     response = self._response_synthesizer.synthesize(
    [191](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/query_engine/retriever_query_engine.py:191)         query=query_bundle,
    [192](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/query_engine/retriever_query_engine.py:192)         nodes=nodes,
    [193](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/query_engine/retriever_query_engine.py:193)     )
    [194](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/query_engine/retriever_query_engine.py:194)     query_event.on_end(payload={EventPayload.RESPONSE: response})
    [196](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/query_engine/retriever_query_engine.py:196) return response

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274, in Dispatcher.span.<locals>.wrapper(func, instance, args, kwargs)
    [270](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:270) self.span_enter(
    [271](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:271)     id_=id_, bound_args=bound_args, instance=instance, parent_id=parent_id
    [272](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:272) )
    [273](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:273) try:
--> [274](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274)     result = func(*args, **kwargs)
    [275](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:275) except BaseException as e:
    [276](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:276)     self.event(SpanDropEvent(span_id=id_, err_str=str(e)))

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:242, in BaseSynthesizer.synthesize(self, query, nodes, additional_source_nodes, **response_kwargs)
    [236](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:236)     query = QueryBundle(query_str=query)
    [238](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:238) with self._callback_manager.event(
    [239](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:239)     CBEventType.SYNTHESIZE,
    [240](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:240)     payload={EventPayload.QUERY_STR: query.query_str},
    [241](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:241) ) as event:
--> [242](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:242)     response_str = self.get_response(
    [243](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:243)         query_str=query.query_str,
    [244](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:244)         text_chunks=[
    [245](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:245)             n.node.get_content(metadata_mode=MetadataMode.LLM) for n in nodes
    [246](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:246)         ],
    [247](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:247)         **response_kwargs,
    [248](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:248)     )
    [250](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:250)     additional_source_nodes = additional_source_nodes or []
    [251](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/base.py:251)     source_nodes = list(nodes) + list(additional_source_nodes)

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274, in Dispatcher.span.<locals>.wrapper(func, instance, args, kwargs)
    [270](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:270) self.span_enter(
    [271](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:271)     id_=id_, bound_args=bound_args, instance=instance, parent_id=parent_id
    [272](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:272) )
    [273](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:273) try:
--> [274](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274)     result = func(*args, **kwargs)
    [275](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:275) except BaseException as e:
    [276](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:276)     self.event(SpanDropEvent(span_id=id_, err_str=str(e)))

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py:43, in CompactAndRefine.get_response(self, query_str, text_chunks, prev_response, **response_kwargs)
     [39](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py:39) # use prompt helper to fix compact text_chunks under the prompt limitation
     [40](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py:40) # TODO: This is a temporary fix - reason it's temporary is that
     [41](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py:41) # the refine template does not account for size of previous answer.
     [42](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py:42) new_texts = self._make_compact_text_chunks(query_str, text_chunks)
---> [43](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py:43) return super().get_response(
     [44](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py:44)     query_str=query_str,
     [45](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py:45)     text_chunks=new_texts,
     [46](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py:46)     prev_response=prev_response,
     [47](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py:47)     **response_kwargs,
     [48](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/compact_and_refine.py:48) )

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274, in Dispatcher.span.<locals>.wrapper(func, instance, args, kwargs)
    [270](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:270) self.span_enter(
    [271](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:271)     id_=id_, bound_args=bound_args, instance=instance, parent_id=parent_id
    [272](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:272) )
    [273](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:273) try:
--> [274](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274)     result = func(*args, **kwargs)
    [275](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:275) except BaseException as e:
    [276](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:276)     self.event(SpanDropEvent(span_id=id_, err_str=str(e)))

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:185, in Refine.get_response(self, query_str, text_chunks, prev_response, **response_kwargs)
    [181](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:181) for text_chunk in text_chunks:
    [182](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:182)     if prev_response is None:
    [183](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:183)         # if this is the first chunk, and text chunk already
    [184](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:184)         # is an answer, then return it
--> [185](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:185)         response = self._give_response_single(
    [186](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:186)             query_str, text_chunk, **response_kwargs
    [187](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:187)         )
    [188](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:188)     else:
    [189](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:189)         # refine response if possible
    [190](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:190)         response = self._refine_response_single(
    [191](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:191)             prev_response, query_str, text_chunk, **response_kwargs
    [192](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:192)         )

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:240, in Refine._give_response_single(self, query_str, text_chunk, **response_kwargs)
    [236](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:236) if response is None and not self._streaming:
    [237](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:237)     try:
    [238](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:238)         structured_response = cast(
    [239](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:239)             StructuredRefineResponse,
--> [240](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:240)             program(
    [241](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:241)                 context_str=cur_text_chunk,
    [242](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:242)                 **response_kwargs,
    [243](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:243)             ),
    [244](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:244)         )
    [245](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:245)         query_satisfied = structured_response.query_satisfied
    [246](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:246)         if query_satisfied:

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:84, in DefaultRefineProgram.__call__(self, *args, **kwds)
     [82](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:82)     answer = answer.json()
     [83](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:83) else:
---> [84](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:84)     answer = self._llm.predict(
     [85](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:85)         self._prompt,
     [86](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:86)         **kwds,
     [87](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:87)     )
     [88](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/response_synthesizers/refine.py:88) return StructuredRefineResponse(answer=answer, query_satisfied=True)

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274, in Dispatcher.span.<locals>.wrapper(func, instance, args, kwargs)
    [270](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:270) self.span_enter(
    [271](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:271)     id_=id_, bound_args=bound_args, instance=instance, parent_id=parent_id
    [272](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:272) )
    [273](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:273) try:
--> [274](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274)     result = func(*args, **kwargs)
    [275](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:275) except BaseException as e:
    [276](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:276)     self.event(SpanDropEvent(span_id=id_, err_str=str(e)))

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/llm.py:442, in LLM.predict(self, prompt, **prompt_args)
    [440](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/llm.py:440) else:
    [441](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/llm.py:441)     formatted_prompt = self._get_prompt(prompt, **prompt_args)
--> [442](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/llm.py:442)     response = self.complete(formatted_prompt, formatted=True)
    [443](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/llm.py:443)     output = response.text
    [444](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/llm.py:444) parsed_output = self._parse_output(output)

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274, in Dispatcher.span.<locals>.wrapper(func, instance, args, kwargs)
    [270](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:270) self.span_enter(
    [271](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:271)     id_=id_, bound_args=bound_args, instance=instance, parent_id=parent_id
    [272](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:272) )
    [273](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:273) try:
--> [274](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:274)     result = func(*args, **kwargs)
    [275](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:275) except BaseException as e:
    [276](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py:276)     self.event(SpanDropEvent(span_id=id_, err_str=str(e)))

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:359, in llm_completion_callback.<locals>.wrap.<locals>.wrapped_llm_predict(_self, *args, **kwargs)
    [342](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:342) dispatcher.event(
    [343](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:343)     LLMCompletionStartEvent(
    [344](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:344)         model_dict=model_dict,
   (...)
    [348](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:348)     )
    [349](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:349) )
    [350](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:350) event_id = callback_manager.on_event_start(
    [351](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:351)     CBEventType.LLM,
    [352](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:352)     payload={
   (...)
    [356](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:356)     },
    [357](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:357) )
--> [359](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:359) f_return_val = f(_self, *args, **kwargs)
    [360](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:360) if isinstance(f_return_val, Generator):
    [361](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:361)     # intercept the generator and add a callback to the end
    [362](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/core/llms/callbacks.py:362)     def wrapped_gen() -> CompletionResponseGen:

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/llms/huggingface/base.py:358, in HuggingFaceLLM.complete(self, prompt, formatted, **kwargs)
    [355](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/llms/huggingface/base.py:355)     if key in inputs:
    [356](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/llms/huggingface/base.py:356)         inputs.pop(key, None)
--> [358](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/llms/huggingface/base.py:358) tokens = self._model.generate(
    [359](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/llms/huggingface/base.py:359)     **inputs,
    [360](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/llms/huggingface/base.py:360)     max_new_tokens=self.max_new_tokens,
    [361](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/llms/huggingface/base.py:361)     stopping_criteria=self._stopping_criteria,
    [362](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/llms/huggingface/base.py:362)     **self.generate_kwargs,
    [363](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/llms/huggingface/base.py:363) )
    [364](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/llms/huggingface/base.py:364) completion_tokens = tokens[0][inputs["input_ids"].size(1) :]
    [365](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/llama_index/llms/huggingface/base.py:365) completion = self._tokenizer.decode(completion_tokens, skip_special_tokens=True)

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/torch/utils/_contextlib.py:115, in context_decorator.<locals>.decorate_context(*args, **kwargs)
    [112](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/torch/utils/_contextlib.py:112) @functools.wraps(func)
    [113](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/torch/utils/_contextlib.py:113) def decorate_context(*args, **kwargs):
    [114](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/torch/utils/_contextlib.py:114)     with ctx_factory():
--> [115](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/torch/utils/_contextlib.py:115)         return func(*args, **kwargs)

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1542, in GenerationMixin.generate(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, negative_prompt_ids, negative_prompt_attention_mask, **kwargs)
   [1540](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1540) tokenizer = kwargs.pop("tokenizer", None)  # Pull this out first, we only use it for stopping criteria
   [1541](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1541) generation_config, model_kwargs = self._prepare_generation_config(generation_config, **kwargs)
-> [1542](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1542) self._validate_model_kwargs(model_kwargs.copy())
   [1544](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1544) # 2. Set generation parameters if not already defined
   [1545](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1545) if synced_gpus is None:

File ~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1157, in GenerationMixin._validate_model_kwargs(self, model_kwargs)
   [1154](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1154)         unused_model_args.append(key)
   [1156](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1156) if unused_model_args:
-> [1157](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1157)     raise ValueError(
   [1158](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1158)         f"The following `model_kwargs` are not used by the model: {unused_model_args} (note: typos in the"
   [1159](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1159)         " generate arguments will also show up in this list)"
   [1160](https://file+.vscode-resource.vscode-cdn.net/Users/zhaiwenjia/Library/CloudStorage/GoogleDrive-vincent507cpu%40gmail.com/%E6%88%91%E7%9A%84%E4%BA%91%E7%AB%AF%E7%A1%AC%E7%9B%98/mmRAG/notebook/~/miniconda3/envs/mmRAG/lib/python3.11/site-packages/transformers/generation/utils.py:1160)     )

ValueError: The following `model_kwargs` are not used by the model: ['token_type_ids'] (note: typos in the generate arguments will also show up in this list)

Version

0.10.38

Steps to Reproduce

following Llama3 Cookbook.

import pandas as pd
import os
from llama_index.core.schema import TextNode
from llama_index.core import Document
import faiss
from llama_index.vector_stores.faiss import FaissVectorStore
from llama_index.core import load_index_from_storage, VectorStoreIndex, StorageContext
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core import Settings
from llama_index.llms.huggingface import HuggingFaceLLM

Settings.embed_model = HuggingFaceEmbedding(
    model_name="BAAI/bge-base-en-v1.5"
)

df = pd.read_csv('./train_texts.csv')

nodes = []

for _, row in df.iterrows():
    node = Document(text=row['text'],
                    doc_id=row['id'],
                    metadata={'title':row['title']})
    nodes.append(node)

llm = HuggingFaceLLM(
    model_name='meta-llama/Meta-Llama-3-8B-Instruct',
    # model_kwargs={
    #     'token':hf_token,
    #     # 'torch_type':torch.bfloat16,
    # },
    generate_kwargs={
        'do_sample':True,
        'temperature':0.6,
        'top_p':0.9,
    },
    tokenizer_name='BAAI/bge-base-en-v1.5',
    tokenizer_kwargs={'token':hf_token},
    # stopping_ids=stopping_ids
)

query_engine = index.as_query_engine(similarity_top_k=3)

response = query_engine.query('What year was Elizabeth Matory the opponent of Charles Albert Ruppersberger?')

Then the error occurs.

Relevant Logs/Tracbacks

No response

May 24 '24 02:05 vincent507cpu

llama_index llama_index copied to clipboard

[Bug]: ValueError: The following `model_kwargs` are not used by the model: ['token_type_ids'] (note: typos in the generate arguments will also show up in this list)

Bug Description

Version

Steps to Reproduce

Relevant Logs/Tracbacks

llama_index
llama_index copied to clipboard