gempy icon indicating copy to clipboard operation
gempy copied to clipboard

Theano error when setting interpolator/compiling

Open shawnboltz opened this issue 2 years ago • 6 comments

Hello,

Describe the bug I recently discovered GemPy and tried to go through the tutorial this afternoon. When I ran gp.set_interpolator(geo_model, compile_theano=True, theano_optimizer="fast_compile") I got the following error:

Exception: ('The following error happened while compiling the node', Elemwise{lt,no_inplace}(Elemwise{sqrt,no_inplace}.0, InplaceDimShuffle{x,x}.0), '\n', 'Compilation failed (return status=1): C:\\Users\\ynt2\\AppData\\Local\\Temp\\2\\ccHRV7LU.s: Assembler messages:\r. C:\\Users\\ynt2\\AppData\\Local\\Temp\\2\\ccHRV7LU.s:2304: Error: invalid register for .seh_savexmm\r. ', '[Elemwise{lt,no_inplace}(<TensorType(float64, matrix)>, <TensorType(float64, (True, True))>)]')

To Reproduce

  1. Install gempy in a new environment (and activate the new environment)
  2. Create a new model using the test dataset associated with Tutorial 1.1
  3. Call gp.set_interpolator(geo_model, compile_theano=True, theano_optimizer="fast_compile")

I tried both installing from pip and from the latest master and got the same result.

Expected behavior An exception shouldn't be raised.

Desktop (please complete the following information):

  • OS: Windows 10
  • GemPy Version: 2.2.11 (alternatively, commit 4a2afb28)
  • Jupyter Version (if applicable): 1.0.0

Additional context I also tried reinstalling using the environment provided by @Japhiolite here: https://github.com/cgre-aachen/gempy/issues/680#issuecomment-1075350431 and got the same result.

I've checked to make sure all of the dependencies for theano are installed, and I've attached a dump of my conda environment.

I would appreciate any help!

conda_list.txt

shawnboltz avatar Apr 05 '22 00:04 shawnboltz

Hi @shawnboltz

thanks for that excellent issue. Really helps with trying to reproduce the error.

Before trying with the exact versions in your conda list, maybe, could you export your conda environment to a file using - in your environment - conda list --export > package-list.txt? That would facilitate to rebuild your environment.

I set up a clean environment, and tried to compile theano for gempy...successfully. So there I could run a model.

To reproduce:

  • Create a new environment: conda create -n theano_test python=3.8
  • activate environment: conda activate theano_test
  • Install conda packages for theano: `conda install -c conda-forge jupyter numpy scipy mkl-service libpython m2w64-toolchain pytest
  • Install gempy: pip install gempy
  • Try importing gempy in a jupyter-notebook and compile theano with gp.set_interpolator(geo_model, compile_theano=True, theano_optimizer="fast_compile")

I currently don't know where to start hunting that theano bug down...

Japhiolite avatar Apr 05 '22 12:04 Japhiolite

Hey @Japhiolite,

Same error, unfortunately. I've attached the export following your instructions... maybe there could be some weird difference in package versions? Another option is that it's a problem with my system, as I am running on a pretty locked down machine.

Have you ever considered swapping over to Aesara? I don't know how involved that would actually be, but I wonder if it would help mitigate Theano-related errors.

package-list.txt

shawnboltz avatar Apr 05 '22 22:04 shawnboltz

Update: Out of curiosity I tried naively swapping theano for aesara just to see if it would work and I'm still getting the same "invalid register .seh_savexmm". Doing a little more digging, it appears to be a problem with the m2w64-toolchain?

See https://github.com/bincrafters/community/issues/1099#issue-548680682

shawnboltz avatar Apr 06 '22 00:04 shawnboltz

Hi @shawnboltz , can you try install MinGW-w64 following this solution and see if it works?

GeorgeLiang3 avatar May 18 '22 09:05 GeorgeLiang3

@GeorgeLiang3 Unfortunately, there's no way that I will be able to follow that solution on the system that I need to use the software on. Thank you for the suggestion, though.

shawnboltz avatar May 19 '22 22:05 shawnboltz

Dear @shawnboltz

another user reported the same problem Error: invalid register for .seh_savexmm\r. ', ' via a different channel and solved the cxxflags.append('-fno-asynchronous-unwind-tables') to ~/theano/gof/cmodule.py file (it may be in a different path for you).

Maybe this helps you to get theano working?

Japhiolite avatar Jun 14 '22 15:06 Japhiolite

Closing due to inactivity, feel free to reopen if the issue still persists.

AlexanderJuestel avatar Mar 17 '23 13:03 AlexanderJuestel