Cannot find/open tsne_multicore shared library

[SumNeuron]

I installed this. It works inside the directory. When I move it too dist-packages, it breaks. It is on sys.path

[guenteru]

I just tried to replicate this and it works fine for me (python 3.6.2). Which python version are you running?

[SumNeuron]

3.6

On Nov 12, 2017, at 4:46 PM, guenteru [email protected] wrote:

I just tried to replicate this and it works fine for me (python 3.6.2). Which python version are you running?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DmitryUlyanov/Multicore-TSNE/issues/32#issuecomment-343746149, or mute the thread https://github.com/notifications/unsubscribe-auth/AVzyadLeNCHVr7fo5wI7GobHEjFZbsoKks5s1xLxgaJpZM4QWGoT.

[guenteru]

which command did you use to install?

[SumNeuron]

git clone https://github.com/DmitryUlyanov/Multicore-TSNE.git
cd Multicore-TSNE/
pip install --no-cache-dir .

[guenteru]

try without --no-cache-dir

[SumNeuron]

I did, then it fails

On Nov 14, 2017, at 11:25 AM, guenteru [email protected] wrote:

try without --no-cache-dir

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/DmitryUlyanov/Multicore-TSNE/issues/32#issuecomment-344213179, or mute the thread https://github.com/notifications/unsubscribe-auth/AVzyafNq2YCIuROE7WXfAjAT_Yi2YJZZks5s2WqDgaJpZM4QWGoT.

[AndrewCurran]

I am having the same problem on 3.6.2.

[heurainbow]

same problem

[overdrivek]

any updates regarding this? i have the same issue.

[DmitryUlyanov]

Please try to install with python setup.py install

[akupemula]

I solved the problem by compiling the shared .so file manually using g++

[mpenagar]

I got the same problem with python 3.4 when running the example inside the installation directory "Multicore-TSNE"

Worked if I run python outside the installation directory.

[LukasMosser]

Python 2.7 Compiled using windows build tools 2017 Installed via pip install. and python setup.py install Neither work. Any idea why?

[arianhosseini]

I had the same problem. This is how I fixed it:

• go to the parent directory
• run pip install Multicore-TSNE/ then the egg is added to /usr/local/lib/python2.7/dist-packages/. Same can be done on python3.x with pip3.

[darthdeus]

Tried both python setup.py install, running pip install . and running pip install Multicore-TSNE from the parent dir, neither works on Windows 10 with 2017 devtools.

[abrahamnunes]

I was experiencing this on one of my machines and so I just updated numpy in my Anaconda distribution. That seemed to fix it.

[jroakes]

The problem I am having is that the file tsne_multicore.dll has been compiled with i386 architecture (32-bit) so it will not work on 64-bit systems, thus the "error 0xc1". I am not sure if this is the correct answer for everyone, but I was able to add:

    if 0 != execute(['cmake',
		 '-GMinGW Makefiles',  <----------This
                     '-DCMAKE_BUILD_TYPE={}'.format(build_type),
                     '-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

to the setup.py file to get it to compile with MinGW for WIndows which works and produces the right files for the architecture.

[haltuf]

I have same problem on Win 10, tried to install using all methods mentioned.

[RichardMeyes]

@jroakes THX! That worked perfectly!

[fingoldo]

@DmitryUlyanov could you kindly take a look, there is still a problem on Windows... UPD After server reboot somehow it started working.. )

[adrisede]

Hi, I also have the same problem on Win 10, and also tried with all the methods mentioned and couldn't make it to work.

[RebekkaWegmann]

I also have the same issue on Win 10.

[IrisStark]

Same issue on Win 10. Tried everything above.

[rajbhatti]

Did anyone find any solution? I am getting the same error

[worm6206]

If you are on Windows,

1. Install MinGW 64. I had it already, but you can install from http://mingw-w64.org/doku.php.
2. Then add MinGW flag as https://github.com/DmitryUlyanov/Multicore-TSNE/issues/32#issuecomment-383385516 describes.
3. Next, build with python setup.py install
4. and install with pip install .

[bscully27]

I spent an hour trying different installs with cmd, mingw, pip, setup.py. I'm using python 2.7.10 on windows7. Nothing worked.

I then tried on python3.6, worked immediately.

[mosynaq]

I built the lib using MSVS 2017 and it worked. I can share the file. But there is an easier way: use pypi wheel package (pip regular installation process)! It simply works!

[eardic]

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.

if 0 != execute(['cmake', '-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS '-DCMAKE_BUILD_TYPE={}'.format(build_type), '-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

[ZippoML]

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.

if 0 != execute(['cmake', '-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS '-DCMAKE_BUILD_TYPE={}'.format(build_type), '-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

thx, solved my problem. win 10

[Po-Shin]

Hi, I used pip install MulticoreTSNE and got the same error while running my code. Anyone got the solution to fix it?

thanks!

[ghayth82]

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4. if 0 != execute(['cmake', '-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS '-DCMAKE_BUILD_TYPE={}'.format(build_type), '-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

thx, solved my problem. win 10

where I can find the setup.py file, please?

[ghayth82]

If you are on Windows,

1. Install MinGW 64. I had it already, but you can install from http://mingw-w64.org/doku.php.
2. Then add MinGW flag as #32 (comment) describes.
3. Next, build with python setup.py install
4. and install with pip install .

hi can you help me , what do you mean by "build with python setup.py install" then install with pip install .` ? can you provide me with full commands please

[ZippoML]

If you are on Windows,

1. Install MinGW 64. I had it already, but you can install from http://mingw-w64.org/doku.php.
2. Then add MinGW flag as #32 (comment) describes.
3. Next, build with python setup.py install
4. and install with pip install .

hi can you help me , what do you mean by "build with python setup.py install" then install with pip install .` ? can you provide me with full commands please

I alrdy forgot what i did to solve the problem. But i think the steps are:

1. download package from here https://github.com/DmitryUlyanov/Multicore-TSNE
2. open setup.py install with notepad or jupyter
3. change the code as was written above
4. install the pakage by command in cmd. python setup.py install

[arikanev]

Also having this issue on OSX, @akupemula, can you expand on how to compile the .so with g++ manually? Thanks!

[ntr34g]

i've done everything mentioned here https://github.com/DmitryUlyanov/Multicore-TSNE/issues/32#issuecomment-465035116 compiled with cmake x64 and mingw-w64\x86_64-8.1.0-posix-sjlj-rt_v6-rev0

but still getting "RuntimeError: Cannot find/open tsne_multicore shared library"

[bscully27]

I used to have a working multicoreTSNE on python3.6 in windows7 but uninstalled then tried re-installing, same problem "RuntimeError: Cannot find/open tsne_multicore shared library".

tried both pip and clone installs.

On Wed, Mar 27, 2019 at 3:38 AM P.S. [email protected] wrote:

i've done everything mentioned here #32 (comment) https://github.com/DmitryUlyanov/Multicore-TSNE/issues/32#issuecomment-464067679 compiled withcmake x64 and mingw-w64\x86_64-8.1.0-posix-sjlj-rt_v6-rev0

but still getting "RuntimeError: Cannot find/open tsne_multicore shared library"

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DmitryUlyanov/Multicore-TSNE/issues/32#issuecomment-477017482, or mute the thread https://github.com/notifications/unsubscribe-auth/ApcfAVaaYWKDCLpxmg45CVbaj2CnaclFks5vax_5gaJpZM4QWGoT .

[ntr34g]

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.

if 0 != execute(['cmake', '-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS '-DCMAKE_BUILD_TYPE={}'.format(build_type), '-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

this helped me on win 10 x64 python 3.6.8

[Applenice]

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.

if 0 != execute(['cmake', '-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS '-DCMAKE_BUILD_TYPE={}'.format(build_type), '-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

Thank you very much, this solved my problem，this helped me on windows 10 x64 python 3.6.5

[mukesh5]

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4.

if 0 != execute(['cmake', '-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS '-DCMAKE_BUILD_TYPE={}'.format(build_type), '-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

Thanx this solved my problem on win10 & python 3.5.4

[jonimatix]

I added '-DCMAKE_GENERATOR_PLATFORM=x64' flag to the setup.py and the problem solved in Win64 (Win10) and Python 3.5.4. if 0 != execute(['cmake', '-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS '-DCMAKE_BUILD_TYPE={}'.format(build_type), '-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

Thanx this solved my problem on win10 & python 3.5.4

This works for me as well. If you had already installed the package, just type pip uninstall MulticoreTSNE and then reinstall it..

[michaelthwan]

The above solution works for me. Let me summarize the solution:

1. If you installed package before, pip uninstall MulticoreTSNE
2. Clone the source git clone https://github.com/DmitryUlyanov/Multicore-TSNE.git
3. Edit /Multicore-TSNE/setup.py,

Find these piece of code and add one line

if 0 != execute(['cmake',
'-DCMAKE_GENERATOR_PLATFORM=x64', <----- THIS
'-DCMAKE_BUILD_TYPE={}'.format(build_type),
'-DCMAKE_VERBOSE_MAKEFILE={}'.format(int(self.verbose)),

4. install by source

cd /Multicore-TSNE
pip install .

[xDarkLemon]

Does anyone know hot to fix it on Mac OS?

[zfhxi]

Problem：

demo：

from sklearn.datasets import load_digits
from MulticoreTSNE import MulticoreTSNE as TSNE
from matplotlib import pyplot as plt

digits = load_digits()
embeddings = TSNE(n_jobs=4).fit_transform(digits.data)
vis_x = embeddings[:, 0]
vis_y = embeddings[:, 1]
plt.scatter(vis_x, vis_y, c=digits.target, cmap=plt.cm.get_cmap("jet", 10), marker='.')
plt.colorbar(ticks=range(10))
plt.clim(-0.5, 9.5)
plt.show()

If you run this demo, the error may occurr：

Traceback (most recent call last):
  File "/xxxx/miniconda3/xxx/test_MultiCore_TSNE.py", line 10, in <module>
    embeddings = TSNE(n_jobs=4).fit_transform(digits.data)
  File "/xxxx/miniconda3/xxx/site-packages/MulticoreTSNE/__init__.py", line 103, in __init__
    raise RuntimeError('Cannot find/open tsne_multicore shared library')
RuntimeError: Cannot find/open tsne_multicore shared library

Explore：

Open /xxx//miniconda3/xxx/site-packages/MulticoreTSNE/__init__.py to check what makes it happen，and locate to around line 103：

path = os.path.dirname(os.path.realpath(__file__))
try:
    sofile = (glob(os.path.join(path, "libtsne*.so")) + glob(os.path.join(path, "*tsne*.dll")))[0]
    self.C = self.ffi.dlopen(os.path.join(path, sofile))
except (IndexError, OSError):
    raise RuntimeError('Cannot find/open tsne_multicore shared library')

To get more concrete information about this error, we can modified this source code as follow：

path = os.path.dirname(os.path.realpath(__file__))
try:
    sofile = (glob(os.path.join(path, "libtsne*.so")) + glob(os.path.join(path, "*tsne*.dll")))[0]
    self.C = self.ffi.dlopen(os.path.join(path, sofile))
# except (IndexError, OSError):
# raise RuntimeError('Cannot find/open tsne_multicore shared library')

except IndexError as e:
    print(e)
    exit(-1)

except OSError as e1:
    print(e1)
    exit(-1)

Then run the demo again, errors may occurr as follow：

cannot load library '/xxxx/miniconda3/xxx/site-packages/MulticoreTSNE/libtsne_multicore.so':
 /xxxx/miniconda3/xxx/site-packages/scipy/_lib/_uarray/../../../../../libstdc++.so.6: version `GLIBCXX_3.4.29' not found
 (required by /xxxx/miniconda3/xxx/site-packages/MulticoreTSNE/libtsne_multicore.so)

From this information, we can see that there's no GLIBCXX_3.4.29 in libstdc++.so.6. Test it as follow:

$ real path /xxxxx/miniconda3/xxx/scipy/_lib/_uarray/../../../../../libstdc++.so.6
$ /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26
$ strings /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26  | grep GLIBCXX_3.4.29
$ # return nothing

Solution

Reference：https://blog.csdn.net/qq_39295044/article/details/86685729

We can find /usr/lib/libstdc++.so.6.0.29 contains GLIBCXX_3.4.29, test it:

$ strings /usr/lib/libstdc++.so.6 | grep GLIBCXX_3.4.29
GLIBCXX_3.4.29
GLIBCXX_3.4.29

This lib file satisfies the requirement, so we can replace /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26 with /usr/lib/libstdc++.so.6.0.29:

$ mv /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26 /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26_bak
$ ln -s /usr/lib/libstdc++.so.6.0.29 /xxxx/miniconda3/xxx/lib/libstdc++.so.6.0.26

Run the demo again, good lucky!