Scrapegraph-ai
Scrapegraph-ai copied to clipboard
ScrapeGraphAI
OMP: Error #15: Initializing libiomp5md.dll, but found libomp140.x86_64.dll already initialized
Describe the bug OMP: Error #15: Initializing libiomp5md.dll, but found libomp140.x86_64.dll already initialized. OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.
To Reproduce Steps to reproduce the behavior: Targetting in the following script various websites, for example this one: https://github.com/VinciGit00/Scrapegraph-ai/releases
from scrapegraphai.graphs import SmartScraperGraph
from scrapegraphai.utils import prettify_exec_info
graph_config = {
"llm": {
"model": "ollama/mistral",
"temperature": 1,
"format": "json",
"model_tokens": 2000,
"base_url": "http://localhost:11434",
},
"embeddings": {
"model": "ollama/nomic-embed-text",
"temperature": 0,
"base_url": "http://localhost:11434",
}
}
smart_scraper_graph = SmartScraperGraph(
prompt="List me all the news with their description.",
source="https://github.com/VinciGit00/Scrapegraph-ai/releases",
config=graph_config
)
result = smart_scraper_graph.run()
print(result)
Expected behavior A list of news with the json format.
Desktop:
- OS: Windows 10
Done pip install --upgrade scrapegraphai installing successfully scrapegraphai-0.10.1
However, the problem persists
Yes, using as source https://perinim.github.io/projects it works (and it worked even with scrapegraphai-0.10.0).
The problem is using different sources, like https://github.com/VinciGit00/Scrapegraph-ai/releases
You can't replicate my result using that link as source?
@baggiponte might this be related to the dependency clashes caused by Poetry you mentioned in issue #198?
@baggiponte might this be related to the dependency clashes caused by Poetry you mentioned in issue #198?
Not sure. I am afraid it's more to do with Intel/Windows. @elmoBG8: how are you installing scrapegraph? Do you use conda (maybe you don't even build scrapegraph on conda though)? What's happening is that there are two libomp binaries.
@baggiponte might this be related to the dependency clashes caused by Poetry you mentioned in issue #198?
Not sure. I am afraid it's more to do with Intel/Windows. @elmoBG8: how are you installing scrapegraph? Do you use conda (maybe you don't even build scrapegraph on conda though)? What's happening is that there are two libomp binaries.
Hello,
inside a virtual environment in anaconda I go with pip install scrapegraphai and then playwright install.
However, I remember that I have no problem for example using as source https://perinim.github.io/projects
I have problems for example with https://github.com/VinciGit00/Scrapegraph-ai/releases as source. Do you get something with it?
@baggiponte might this be related to the dependency clashes caused by Poetry you mentioned in issue #198?
Not sure. I am afraid it's more to do with Intel/Windows. @elmoBG8: how are you installing scrapegraph? Do you use conda (maybe you don't even build scrapegraph on conda though)? What's happening is that there are two libomp binaries.
Hello, inside a virtual environment in anaconda I go with
pip install scrapegraphaiand thenplaywright install.
I am not proficient enough when using windows + conda. Is the venv you create a new one or a "recycled" one where there are already previous libraries? What might happen is that you have different versions of libomp on your machine because you have 2/3 conda envs that each have one. I am not an expert in windows at all, but this new env might pick up the two different omp versions even if they are in separate envs. maybe you can try deleting the conda cache (just googled this) but I haven't used it in years so I might be wrong.
Yes, now I don't see that error, even if I have another problem now... I open another issue, thank you
