gempy
gempy copied to clipboard
gempy-project
GemPy Blender Integration
As stated in the 2019 Paper, it should be possible to run GemPy within Blenders integrated Python. I am trying to get this running for quite some time now and ran into numerous issues, one of which i just cannot overcome.
My setup is:
Windows 10 64 Bit Blender 3.0.1 with Python 3.9.7 Miniconda3 with Python 3.9.7
MSYS2 with g++ compiler
base 2020.12-1
base-devel 2022.01-1
bash 5.1.016-1
bash-completion 2.11-1
binutils 2.37-5
bison 3.8.2-2
brotli 1.0.9-2
bsdtar 3.5.3-1
bzip2 1.0.8-3
ca-certificates 20210119-3
coreutils 8.32-2
curl 7.80.0-3
dash 0.5.11.5-1
db 5.3.28-3
diffstat 1.64-1
diffutils 3.8-2
dos2unix 7.4.2-1
file 5.41-2
filesystem 2022.01-3
findutils 4.8.0-1
flex 2.6.4-2
gawk 5.1.0-2
gcc-libs 11.2.0-3
gdbm 1.22-2
getent 2.18.90-3
gettext 0.21-1
glib2 2.68.4-1
gmp 6.2.1-1
gnupg 2.2.32-2
gperf 3.1-3
grep 1~3.0-3
groff 1.22.4-2
gzip 1.11-1
heimdal-libs 7.7.0-3
icu 70.1-1
inetutils 1.9.4-3
info 6.8-3
less 590-1
libargp 20110921-3
libasprintf 0.21-1
libassuan 2.5.5-1
libbz2 1.0.8-3
libcrypt 2.1-3
libcurl 7.80.0-3
libdb 5.3.28-3
libedit 20210910_3.1-1
libexpat 2.4.6-1
libffi 3.3-1
libgcrypt 1.9.4-1
libgdbm 1.22-2
libgettextpo 0.21-1
libgnutls 3.7.3-1
libgpg-error 1.44-1
libgpgme 1.17.0-1
libhogweed 3.7.3-1
libiconv 1.16-2
libidn2 2.3.2-1
libintl 0.21-1
libksba 1.6.0-1
liblz4 1.9.3-1
liblzma 5.2.5-1
libnettle 3.7.3-1
libnghttp2 1.46.0-1
libnpth 1.6-1
libopenssl 1.1.1.m-1
libp11-kit 0.24.1-1
libpcre 8.45-1
libpcre2_8 10.37-1
libpsl 0.21.1-2
libreadline 8.1.001-1
libsqlite 3.36.0-3
libssh2 1.10.0-1
libtasn1 4.18.0-2
libunistring 0.9.10-1
libutil-linux 2.35.2-1
libxml2 2.9.12-3
libxslt 1.1.34-4
libzstd 1.5.2-1
m4 1.4.19-2
make 4.3-3
mingw-w64-x86_64-binutils 2.38-1
mingw-w64-x86_64-bzip2 1.0.8-2
mingw-w64-x86_64-ca-certificates 20210119-1
mingw-w64-x86_64-crt-git 9.0.0.6428.e1b6a2ed7-1
mingw-w64-x86_64-expat 2.4.6-1
mingw-w64-x86_64-gcc 11.2.0-9
mingw-w64-x86_64-gcc-ada 11.2.0-9
mingw-w64-x86_64-gcc-fortran 11.2.0-9
mingw-w64-x86_64-gcc-libgfortran 11.2.0-9
mingw-w64-x86_64-gcc-libs 11.2.0-9
mingw-w64-x86_64-gcc-objc 11.2.0-9
mingw-w64-x86_64-gdb 11.2-2
mingw-w64-x86_64-gdb-multiarch 11.2-2
mingw-w64-x86_64-gettext 0.21-3
mingw-w64-x86_64-gmp 6.2.1-3
mingw-w64-x86_64-headers-git 9.0.0.6428.e1b6a2ed7-1
mingw-w64-x86_64-isl 0.24-1
mingw-w64-x86_64-libffi 3.3-4
mingw-w64-x86_64-libgccjit 11.2.0-9
mingw-w64-x86_64-libiconv 1.16-2
mingw-w64-x86_64-libmangle-git 9.0.0.6428.e1b6a2ed7-1
mingw-w64-x86_64-libsystre 1.0.1-4
mingw-w64-x86_64-libtasn1 4.18.0-1
mingw-w64-x86_64-libtre-git r128.6fb7206-2
mingw-w64-x86_64-libwinpthread-git 9.0.0.6428.e1b6a2ed7-1
mingw-w64-x86_64-make 4.3-1
mingw-w64-x86_64-mpc 1.2.1-1
mingw-w64-x86_64-mpdecimal 2.5.1-1
mingw-w64-x86_64-mpfr 4.1.0.p13-1
mingw-w64-x86_64-ncurses 6.3-3
mingw-w64-x86_64-openssl 1.1.1.m-1
mingw-w64-x86_64-p11-kit 0.24.1-1
mingw-w64-x86_64-pkgconf 1.8.0-2
mingw-w64-x86_64-python 3.9.10-2
mingw-w64-x86_64-readline 8.1.001-1
mingw-w64-x86_64-sqlite3 3.37.2-1
mingw-w64-x86_64-tcl 8.6.11-5
mingw-w64-x86_64-termcap 1.3.1-6
mingw-w64-x86_64-tk 8.6.11.1-2
mingw-w64-x86_64-tools-git 9.0.0.6428.e1b6a2ed7-1
mingw-w64-x86_64-tzdata 2021e-1
mingw-w64-x86_64-windows-default-manifest 6.4-4
mingw-w64-x86_64-winpthreads-git 9.0.0.6428.e1b6a2ed7-1
mingw-w64-x86_64-winstorecompat-git 9.0.0.6428.e1b6a2ed7-1
mingw-w64-x86_64-xxhash 0.8.1-1
mingw-w64-x86_64-xz 5.2.5-2
mingw-w64-x86_64-zlib 1.2.11-9
mingw-w64-x86_64-zstd 1.5.2-1
mintty 1~3.5.3-1
mpdecimal 2.5.0-1
mpfr 4.1.0-1
msys2-keyring 1~20211228-1
msys2-launcher 1.4-1
msys2-runtime 3.3.4-2
nano 6.2-2
ncurses 6.2-2
nettle 3.7.3-1
openssl 1.1.1.m-1
p11-kit 0.24.1-1
pacman 6.0.1-10
pacman-contrib 1.4.0-2
pacman-mirrors 20220205-1
patch 2.7.6-1
perl 5.32.1-2
pinentry 1.2.0-1
pkgconf 1.8.0-1
python 3.9.9-2
python-devel 3.9.9-2
rebase 4.5.0-1
sed 4.8-2
tar 1.34-2
tcl 8.6.10-1
texinfo 6.8-3
texinfo-tex 6.8-3
tftp-hpa 5.2-4
time 1.9-2
tzcode 2021e-1
util-linux 2.35.2-1
wget 1.21.2-1
which 2.21-3
xz 5.2.5-1
zlib 1.2.11-1
zstd 1.5.2-1
I have added the following to path environmental variable
C:\DATA\Coding\blender\3.0\python\bin
C:\DATA\Coding\blender\3.0\python\scripts
C:\DATA\Coding\msys2\mingw64\bin
I am installing the following packages, either to my Conda environment or directly to Blenders Python. I tried base and a separate env for all of this, it didn't make a difference. I also tried to use conda install theano instead of pip. I tried via cmd, AnacondaPrompt and Blenders inbuilt Python interpreter. I have to use specific versions of Pandas and NumPy, because latest pandas has problems with GemPy and latest NumPy has Problems with Theano.
cd c:\condapath-or-blenderpath\python\
python -m pip install --upgrade pip
python -m pip install theano
python -m pip install gempy
python -m pip install pandas==1.3.4
python -m pip install numpy==1.20.3
I can run my test.py from within the AnacondaPrompt, getting the expected results and the following output
test.py
# A most basic model to test the functionality
# https://www.youtube.com/watch?v=n0btC5Zilyc&t=568s
import numpy # Just
import pandas # for
import theano # testing
import gempy as gp
geo_model = gp.create_model('testmodel')
gp.init_data(geo_model, extent=[0, 791, 0, 200, -582, 0],
resolution=[50, 50, 50])
gp.set_interpolator(geo_model, output=['geology'],
theano_optimizer='fast_compile')
p2d = gp.plot_2d(geo_model, section_names=None, direction=None,
cell_number=None)
ax = p2d.add_section(cell_number=1, directionm='y')
p3d = gp.plot_3d(geo_model, plotter_type='background', notebook=False)
geo_model.set_default_surfaces()
geo_model.add_surface_points(X=223, Y=0.01, Z=-94, surface='surface1', idx=0)
geo_model.add_surface_points(X=458, Y=0, Z=-107, surface='surface1', idx=1)
geo_model.add_surface_points(X=612, Y=0, Z=-14, surface='surface1', idx=2)
geo_model.add_orientations(X=350, Y=0, Z=-300, surface='surface1',
pole_vector=(0, 0, 1))
p2d.plot_data(ax, cell_number=11)
p3d.plot_data()
gp.compute_model(geo_model)
p2d.plot_contacts(ax, cell_number=5)
p3d.plot_surfaces()
p3d.plot_structured_grid()
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.
No module named 'osgeo'
Not subsurface compatibility available
Active grids: ['regular']
Setting kriging parameters to their default values.
Compiling theano function...
Level of Optimization: fast_compile
Device: cpu
Precision: float64
Number of faults: 0
Compilation Done!
Kriging values:
values
range 1002.20008
$C_o$ 23914.404762
drift equations [3]
<...both plots opening succesfully...>
I tried to versions of connecting Blender to GemPy.
Attempt 1: Using a Conda environment Overriding the Blender Python with a Conda Python environment. To do so, i renamed Blenders Python Folder and created a hardlink to my conda environment, and there i create a directory bin, in which i place another hardlink, to the Python executable
cd c:\data\coding\blender\3.0\
ren python _python
mklink /j python c:\users\<username>\miniconda3
cd C:\Users\<username>\miniconda3
mkdir bin
cd bin
mklink /h python.exe c:\users\<username>\miniconda3\python.exe
Blender recognizes Python, i can import the packages, however on executing the test.py script i get the following
ImportError
Traceback (most recent call last):
File "c:\DATA\Coding\Blender\test.py", line 8, in <module>
gp.set_interpolator(geo_model, output=['geology'],
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\gempy\api_modules\setters.py", line 108, in set_interpolator
geo_model._interpolator.compile_th_fn_geo(inplace=True, grid=grid)
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\gempy\core\interpolator.py", line 1124, in compile_th_fn_geo
self.theano_graph.theano_output(),
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\gempy\core\theano_modules\theano_graph_pro.py", line 631, in theano_output
solutions[:9] = self.compute_series()
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\gempy\core\theano_modules\theano_graph_pro.py", line 465, in compute_series
series, self.updates1 = theano.scan(
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\scan_module\scan.py", line 774, in scan
condition, outputs, updates = scan_utils.get_updates_and_outputs(fn(*args))
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\gempy\core\theano_modules\theano_graph_pro.py", line 2141, in compute_a_series
self.compute_scalar_field(weights, grid,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\gempy\core\theano_modules\theano_graph_pro.py", line 434, in compute_scalar_field
return self.scalar_field_at_all(weights, grid_val, fault_matrix)
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\gempy\core\theano_modules\theano_graph_pro.py", line 1725, in scalar_field_at_all
Z_x_loop, updates3 = theano.scan(
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\scan_module\scan.py", line 843, in scan
dummy_f = function(dummy_args,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\compile\function.py", line 306, in function
fn = pfunc(params=inputs,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\compile\pfunc.py", line 483, in pfunc
return orig_function(inputs, cloned_outputs, mode,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\compile\function_module.py", line 1841, in orig_function
fn = m.create(defaults)
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\compile\function_module.py", line 1714, in create
_fn, _i, _o = self.linker.make_thunk(
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\link.py", line 697, in make_thunk
return self.make_all(input_storage=input_storage,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\link.py", line 826, in make_all
thunks += [node.op.make_thunk(node,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\scan_module\scan_op.py", line 861, in make_thunk
self.fn = function(wrapped_inputs,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\compile\function.py", line 306, in function
fn = pfunc(params=inputs,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\compile\pfunc.py", line 483, in pfunc
return orig_function(inputs, cloned_outputs, mode,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\compile\function_module.py", line 1841, in orig_function
fn = m.create(defaults)
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\compile\function_module.py", line 1714, in create
_fn, _i, _o = self.linker.make_thunk(
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\link.py", line 697, in make_thunk
return self.make_all(input_storage=input_storage,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\vm.py", line 1087, in make_all
thunks.append(node.op.make_thunk(node,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\op.py", line 954, in make_thunk
return self.make_c_thunk(node, storage_map, compute_map,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\op.py", line 857, in make_c_thunk
outputs = cl.make_thunk(input_storage=node_input_storage,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\cc.py", line 1215, in make_thunk
cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\cc.py", line 1153, in __compile__
thunk, module = self.cthunk_factory(error_storage,
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\cc.py", line 1623, in cthunk_factory
module = get_module_cache().module_from_key(
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\cmodule.py", line 1189, in module_from_key
module = lnk.compile_cmodule(location)
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\cc.py", line 1520, in compile_cmodule
module = c_compiler.compile_str(
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\cmodule.py", line 2420, in compile_str
return dlimport(lib_filename)
File "C:\DATA\Coding\Blender\3.0\python\lib\site-packages\theano\gof\cmodule.py", line 317, in dlimport
rval = __import__(module_name, {}, {}, [module_name])
ImportError: DLL load failed while importing m7b9aaee4c7e7a84c9349e61bca7c6a325d685aa6ac844a5dba2a836e7ea148e3: Das angegebene Modul wurde nicht gefunden.
2. Attempt: Using Blenders Python Since Blender is not shipping a "true" Python distribution, but some lightweight one, i need to copy a few files from Miniconda3, to get rid of DLLLoadErrors
xcopy C:\Users\<username>\miniconda3\include c:\data\coding\blender\3.0\python\include /E /H /I
xcopy C:\Users\<username>\miniconda3\libs c:\data\coding\blender\3.0\python\libs /E /H /I
after that, theano compiles its stuff untill it gets the same ImportError as stated above. A few times, there has been a different filecode, but it was mostly the m7b9... - which i think is already weird: If a file is missing, it should be the same every time?
I did check within C:\Users\<username>\AppData\Local\Theano\compiledir_Windows-10-10.0.19044-SP0-Intel64_Family_6_Model_142_Stepping_12_GenuineIntel-3.9.7-64 and there are a lot of folders/filecodes, however, the specified is actually missing. And i just can't know how to get it generating...
3. Attempt: I also tried copy-paste a complete working environment and overwrite Blenders Python. Didn't work either.
So i got a few questions: Can you confirm that GemPy 2.x actually works with Blender 3.0? Can you confirm other combinations working? What is the recommended way of getting those 2 together? Am i following the right direction or am i missing something? While googling i found out, that many users are having problems when developing with Blender and Python, if they have to compile something to c or so. Also with Theano being discontinued, there are not really any tutorials on how to get Blender and Theano working. Actually i've read of many Problems with Theano and Win10 64Bit? Most professional users seem to use Linux systems?
I would really like to use GemPy directly from Blender, using it as a potential frontend for GemPy. I am aware of the option to create numpy.ndarrays outside of it with GemPy and just import them, but that solution is not viable for me at the moment.
I'd appreciate any help/hints what path to follow next! If more information is needed please tell. Thanks in advance!
Hi @stoaschaedl,
Interesting project! We currently take a very similar path at the moment and develop a Blender Add on for GemPy. Quite experimental at the moment, so not out in the public, yet... but if you are interested, we can invite you for one of our Blender GemPy sessions. Feel free to send me an Email if you are interested ([email protected]).
