macOS env

[lokinfey]

I try to use Numpy.NET in my macos

but it give me this error

Unable to load shared library 'python36' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: dlopen(libpython36, 1): image not found at Python.Runtime.Runtime.Py_IsInitialized() at Python.Runtime.Runtime.Initialize(Boolean initSigs) at Python.Runtime.PythonEngine.Initialize(IEnumerable1 args, Boolean setSysArgv, Boolean initSigs) at Numpy.NumPy.InstallAndImport(Boolean force) at Numpy.NumPy.<>c.<.cctor>b__650_0() at System.Lazy1.ViaFactory(LazyThreadSafetyMode mode) at System.Lazy1.ExecutionAndPublication(LazyHelper executionAndPublication, Boolean useDefaultConstructor) at System.Lazy1.CreateValue() at Numpy.np.array[T](T[,] object, Dtype dtype, Nullable1 copy, String order, Nullable1 subok, Nullable1 ndmin) at Submission#4.<>d__0.MoveNext() in :line 1 --- End of stack trace from previous location where exception was thrown --- at Microsoft.CodeAnalysis.Scripting.ScriptExecutionState.RunSubmissionsAsync[TResult](ImmutableArray1 precedingExecutors, Func2 currentExecutor, StrongBox1 exceptionHolderOpt, Func2 catchExceptionOpt, CancellationToken cancellationToken) at Microsoft.CodeAnalysis.Scripting.Script1.RunSubmissionsAsync(ScriptExecutionState executionState, ImmutableArray1 precedingExecutors, Func2 currentExecutor, Func`2 catchExceptionOpt, CancellationToken cancellationToken) at ICSharpCore.Script.InteractiveScriptEngine.ExecuteAsync(String statement)

[henon]

I don't own a mac so it is not easy for me to test with your scenario. Are you using Numpy.Bare? My guess is that pythonnet is compiled against a windows binary? Could you try to run the simple example from the pythonnet readme on macOS to check if you get the same error message?

If you are willing to work on macOS support I will support you with my knowledge.

[henon]

One more thing: If you use Numpy.Bare.dll it doesn't reference Python.Included, just pythonnet_netstandard. If you use Numpy.dll it uses Python.Included which in turn references pythonnet_netstandard.

Step one is to find out if the problem is at pythonnet_netstandard or at Python.Included.

[lokinfey]

I change Numpy.Bare.dll is the same problem

[henon]

@lokinfey Are you willing to help with this issue?

[Nucs]

The issue is that operating system is trying to find the installed python 3.6 on the machine. As the exception tells you, "consider setting the DYLD_PRINT_LIBRARIES environment variable" to the Python\Python36\Lib directory.
More about the DYLD_PRINT_LIBRARIES. A similar issue in pythonnet #767 - he has a detailed solution. Let me know if it helped you as I can't test it myself.

[deepakkumar1984]

I have MAC, i will try out Nucs suggestion. But I think we need to build Numpy with Pythonnet with OSX flag which will pick the dylib files. Other test I can is using dll.config file as per this https://answers.unity.com/questions/359991/how-to-use-mono-dllnamedllconfig-with-unity.html

[henon]

@deepakkumar1984 thanks man. Make sure you use my fork of pythonnet because it has some additions that were necessary for Numpy. By the way, for Keras.NET it is also the correct dependency, not Python.Included, like you wrote.

[Oceania2018]

Related issue https://github.com/pythonnet/pythonnet/issues/892

@deepakkumar1984 Does this help? http://curtis.schlak.com/2012/01/17/howto-run-pythonnet-on-osx.html

[deepakkumar1984]

The Numpy project is built with pythonnet with WIN parameters, which will look for Python37.dll. After building with OSX parameters, it's failing at method "PyUnicode_FromKindAndData" in runtime.cs:

public static IntPtr PyString_FromString(string value)
        {
#if PYTHON3
            return PyUnicode_FromKindAndData(_UCS, value, value.Length);
#elif PYTHON2
            return PyString_FromStringAndSize(value, value.Length);
#endif
        }

Had to make the following change in the runtime.cs:

#if MONO_LINUX // Linux/macOS use dotted version string
        internal const string dllBase = "python" + _pyversion;
#elif MONO_OSX // Linux/macOS use dotted version string
        internal const string dllBase = "libpython"; //The change to load python in MAC
#else // Windows
        internal const string dllBase = "python" + _pyver;
#endif

In Initialize() method:

//In OSX python PyObject_GetAttrString(op, "__dict__") chashes. Had to add condition to use Python2 method
if (IsPython3 && Environment.OSVersion.Platform.ToString() != "Unix")
            {
                op = PyImport_ImportModule("builtins");
                dict = PyObject_GetAttrString(op, "__dict__");
            }
            else// Python2
            {
                dict = PyImport_GetModuleDict();
                op = PyDict_GetItemString(dict, "__builtin__");
            }

I am still looking into the error with Unicode method if anyone knows please comment.

[Nucs]

That's weird, are you absolutely sure you are using python 3+?

PY3 - PyUnicode_FromKindAndData PY2 - PyString_FromStringAndSize

Edit: If it is possible in osx to monitor process's file accessing, you should verify it using it.

[deepakkumar1984]

Yes I have Python 3.7 installed. It doesn't work either "PyUnicode_FromKindAndData" and "PyString_FromStringAndSize". So I am not sure if there is 3rd function specific for OSX

[Nucs]

Can you post here the exact error? If it fails both methods, it seems to have failed loading the library dlls

[deepakkumar1984]

I will post the full stack after getting back home (no mac at workplace), It seems to load the Python dlls and invoked few Initialization methods in the Runtime.Initialize method and failed at line 337: op = PyString_FromString("string");

[deepakkumar1984]

Stack Trace: System.EntryPointNotFoundException: "Unable to find an entry point named 'PyUnicode_FromKindAndData' in shared library 'libpython'." at Python.Runtime.Runtime.PyUnicode_FromKindAndData(Int32 kind, String s, IntPtr size)\n at Python.Runtime.Runtime.PyUnicode_FromKindAndData(Int32 kind, String s, Int64 size) in /Users/deepakbattini/work/git/pythonnet/src/runtime/runtime.cs:line 1464\n at Python.Runtime.Runtime.PyString_FromString(String value) in /Users/deepakbattini/work/git/pythonnet/src/runtime/runtime.cs:line 1392\n at Python.Runtime.Runtime.Initialize(Boolean initSigs) in /Users/deepakbattini/work/git/pythonnet/src/runtime/runtime.cs:line 341\n at Python.Runtime.PythonEngine.Initialize(IEnumerable`1 args, Boolean setSysArgv, Boolean initSigs) in /Users/deepakbattini/work/git/pythonnet/src/runtime/pythonengine.cs:line 168\n at Python.Runtime.PythonEngine.Initialize(Boolean setSysArgv, Boolean initSigs) in /Users/deepakbattini/work/git/pythonnet/src/runtime/pythonengine.cs:line 145\n at Python.Runtime.PythonEngine.Initialize() in /Users/deepakbattini/work/git/pythonnet/src/runtime/pythonengine.cs:line 140\n at Python.Runtime.PythonConsole.Main(String[] args) in /Users/deepakbattini/work/git/pythonnet/src/console/pythonconsole.cs:35

[Nucs]

Give this a shot, the other answers are are too relevant.

[deepakkumar1984]

I configured the Console app with DYLD_PRINT_LIBRARIES to print all the loaded binaries. Although Python3.7 is installed, but the console shows python2.7 dyld: loaded: /System/Library/Frameworks/Python.framework/Versions/2.7/Python dyld: loaded: /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload/_locale.so

By default python3 shows up by running the command. And I cannot uninstall Python2.7 because it comes with OSX and will corrupt the OS. I am looking for options but looks like this is the main issue.

[henon]

if it is something fundamental like you suggest others must have similar problems with python3

[lokinfey]

The Numpy project is built with pythonnet with WIN parameters, which will look for Python37.dll. After building with OSX parameters, it's failing at method "PyUnicode_FromKindAndData" in runtime.cs:

public static IntPtr PyString_FromString(string value)
        {
#if PYTHON3
            return PyUnicode_FromKindAndData(_UCS, value, value.Length);
#elif PYTHON2
            return PyString_FromStringAndSize(value, value.Length);
#endif
        }

Had to make the following change in the runtime.cs:

#if MONO_LINUX // Linux/macOS use dotted version string
        internal const string dllBase = "python" + _pyversion;
#elif MONO_OSX // Linux/macOS use dotted version string
        internal const string dllBase = "libpython"; //The change to load python in MAC
#else // Windows
        internal const string dllBase = "python" + _pyver;
#endif

In Initialize() method:

//In OSX python PyObject_GetAttrString(op, "__dict__") chashes. Had to add condition to use Python2 method
if (IsPython3 && Environment.OSVersion.Platform.ToString() != "Unix")
            {
                op = PyImport_ImportModule("builtins");
                dict = PyObject_GetAttrString(op, "__dict__");
            }
            else// Python2
            {
                dict = PyImport_GetModuleDict();
                op = PyDict_GetItemString(dict, "__builtin__");
            }

I am still looking into the error with Unicode method if anyone knows please comment.

But when I followed all the step in wiki ,and set the Python.Runtime.dll.config , I can run but no results and no some errors . I think libpython3.7m.dylib is the problem about that ........ Can you give me some suggestions?

[henon]

I suggest creating an issue on the pythonnet repo. There the probability that someone can help with the problem is highest. But make sure you post a pure example without any mention of Numpy.NET in order to avoid confusion.

[Oceania2018]

@henon FYI https://github.com/pythonnet/pythonnet/issues/903

[Esther2013]

@henon I tried it on my macOS, it's passed when I set the compile condition as MONO_OSX.

Here is the define symbols: TRACE;RELEASE;NETSTANDARD;NETSTANDARD2_0;MONO_OSX.

Test run for /Users/estherhu/Projects/pythonnet/src/embed_tests/bin/Debug/netcoreapp2.2/Python.EmbeddingTest.dll(.NETCoreApp,Version=v2.2)
Microsoft (R) Test Execution Command Line Tool Version 15.9.0
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
Failed   TestCastGlobalVar
Error Message:
 Python.Runtime.PythonException : ImportError : No module named PyImportTest.cast_global_var
Stack Trace:
   at Python.Runtime.Runtime.CheckExceptionOccurred() in /Users/estherhu/Projects/pythonnet/src/runtime/runtime.cs:line 569
   at Python.Runtime.PythonEngine.ImportModule(String name) in /Users/estherhu/Projects/pythonnet/src/runtime/pythonengine.cs:line 469
   at Python.Runtime.Py.Import(String name) in /Users/estherhu/Projects/pythonnet/src/runtime/pythonengine.cs:line 693
   at Python.EmbeddingTest.PyImportTest.TestCastGlobalVar() in /Users/estherhu/Projects/pythonnet/src/embed_tests/pyimport.cs:line 78
Failed   TestDottedName
Error Message:
 Python.Runtime.PythonException : ImportError : No module named PyImportTest.test.one
Stack Trace:
   at Python.Runtime.Runtime.CheckExceptionOccurred() in /Users/estherhu/Projects/pythonnet/src/runtime/runtime.cs:line 569
   at Python.Runtime.PythonEngine.ImportModule(String name) in /Users/estherhu/Projects/pythonnet/src/runtime/pythonengine.cs:line 469
   at Python.EmbeddingTest.PyImportTest.TestDottedName() in /Users/estherhu/Projects/pythonnet/src/embed_tests/pyimport.cs:line 58
Failed   TestSysArgsImportException
Error Message:
 Python.Runtime.PythonException : ImportError : No module named PyImportTest.sysargv
Stack Trace:
   at Python.Runtime.Runtime.CheckExceptionOccurred() in /Users/estherhu/Projects/pythonnet/src/runtime/runtime.cs:line 569
   at Python.Runtime.PythonEngine.ImportModule(String name) in /Users/estherhu/Projects/pythonnet/src/runtime/pythonengine.cs:line 469
   at Python.EmbeddingTest.PyImportTest.TestSysArgsImportException() in /Users/estherhu/Projects/pythonnet/src/embed_tests/pyimport.cs:line 68
Skipped  ReInitialize
Skipped  SimpleTestMemory
Skipped  TestUnicode
Skipped  IsStringTrue
Skipped  TestPyObjectCtor
Enter
Enter
Exit

Total tests: 182. Passed: 174. Failed: 3. Skipped: 5.
Test Run Failed.
Test execution time: 3.4660 Seconds

[henon]

awesome @Esther2013 ! I'll release a MacOS version of Numpy.Bare

[lokinfey]

@Esther2013 I have set MONO_OSX , I use this to compile my project

Python.Runtime.csproj.zip

I have followed you to add TRACE;RELEASE;NETSTANDARD;NETSTANDARD2_0;MONO_OSX in the define symbols.

And build it in command line dotnet build -f netstandard2.0 Python.Runtime.csproj

I add Python.Runtime.dll to my project

but I dont't know why it always give me

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 python3.7m.dylib 0x0000000109de8a37 PyUnicode_InternInPlace + 116 1 python3.7m.dylib 0x0000000109e010b4 PyUnicode_InternFromString + 32 2 python3.7m.dylib 0x0000000109e4c781 PyImport_Import + 87 3 python3.7m.dylib 0x0000000109e4b228 PyImport_ImportModule + 28