graphrag icon indicating copy to clipboard operation
graphrag copied to clipboard

[Issue]: <title> Local query error---OSError: LanceError(IO)

Open wangruidedie opened this issue 1 year ago • 4 comments

Do you need to file an issue?

  • [ ] I have searched the existing issues and this bug is not already filed.
  • [ ] My model is hosted on OpenAI or Azure. If not, please look at the "model providers" issue and don't file a new one here.
  • [ ] I believe this is a legitimate bug, not just a question. If this is a question, please use the Discussions area.

Describe the issue

When I execute the command 'python - m graphrag. query -- root./ragtest -- method local "..."', the error is as follows:

Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/site-packages/graphrag/query/main.py", line 86, in run_local_search( File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/site-packages/graphrag/query/cli.py", line 98, in run_local_search return asyncio.run( ^^^^^^^^^^^^ File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/asyncio/runners.py", line 190, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/site-packages/graphrag/query/api.py", line 172, in local_search description_embedding_store = __get_embedding_description_store( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/site-packages/graphrag/query/api.py", line 62, in __get_embedding_description_store store_entity_semantic_embeddings( File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/site-packages/graphrag/query/input/loaders/dfs.py", line 91, in store_entity_semantic_embeddings vectorstore.load_documents(documents=documents) File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/site-packages/graphrag/vector_stores/lancedb.py", line 55, in load_documents self.document_collection = self.db_connection.create_table( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/site-packages/lancedb/db.py", line 417, in create_table tbl = LanceTable.create( ^^^^^^^^^^^^^^^^^^ File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/site-packages/lancedb/table.py", line 1578, in create lance.write_dataset(empty, tbl._dataset_uri, schema=schema, mode=mode) File "/CSTemp/fishfield/Anaconda3/envs/ragtest/lib/python3.11/site-packages/lance/dataset.py", line 2644, in write_dataset inner_ds = _write_dataset(reader, uri, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ OSError: LanceError(IO): Generic LocalFileSystem error: Unable to copy file from /CSTemp/zk/GraphragTest-main/lancedb/entity_description_embeddings.lance/_versions/1.manifest-ea6604dc-cbb6-4d09-acd6-bf517decb9ee to /CSTemp/zk/GraphragTest-main/lancedb/entity_description_embeddings.lance/_latest.manifest: Device or resource busy (os error 16), /home/runner/work/lance/lance/rust/lance-table/src/io/commit.rs:240:5

Steps to reproduce

No response

GraphRAG Config Used

# Paste your config here

Logs and screenshots

No response

Additional Information

  • GraphRAG Version:
  • Operating System:
  • Python Version:
  • Related Issues:

wangruidedie avatar Aug 30 '24 06:08 wangruidedie

你能为这个issue提供更多的描述吗? @wangruidedie

ka2048 avatar Aug 30 '24 11:08 ka2048

你能为这个issue提供更多的描述吗? @wangruidedie

已经提供全部信息了,我感觉是我们服务器环境的问题,我在我的本地和另外一台服务器上并没有这个问题

wangruidedie avatar Aug 30 '24 13:08 wangruidedie

according to https://github.com/lancedb/lancedb/issues/1630

It sounds like you are using a filesystem that doesn't support hard link. That is required for LanceDB to function. You should > either mount a fully-featured filesystem, or connect to an object storage service.

I encountered this issue while running graphrag in WSL environment and I stored the lancedb windows volumes (/mnt/e/graphrag). After moving the folder to Linux directory (~/graphrag), I didn't encounter this issue anymore.

jiyzhang avatar Sep 16 '24 02:09 jiyzhang

according to https://github.com/lancedb/lancedb/issues/1630

It sounds like you are using a filesystem that doesn't support hard link. That is required for LanceDB to function. You should > either mount a fully-featured filesystem, or connect to an object storage service.

I encountered this issue while running graphrag in WSL environment and I stored the lancedb windows volumes (/mnt/e/graphrag). After moving the folder to Linux directory (~/graphrag), I didn't encounter this issue anymore.

Me too, it works fine on Linux but not on Windows.

YepJin avatar Sep 30 '24 02:09 YepJin