Issue executing a kernel - dev mode

[graceebc9]

Hello! as per a previous issue I cant download parcels through conda so using the dev version. Im having an issue when trying to execute a paticle move / kernel.

I'm trying to run

pset.execute(kernels,                 # the kernel (which defines how particles move)
             runtime=86400*24,        # the total length of the run in seconds
             dt=300,                  # the timestep of the kernel in seconds
             output_file=output_file)

from https://nbviewer.org/github/OceanParcels/parcels/blob/master/parcels/examples/tutorial_parcels_structure.ipynb#2.-ParticleSet

but get the error

-INFO: Compiled ArrayAgeParticleAgeNorthVelAdvectionRK4 ==> /var/folders/n5/_2d3c5sd5j3810327mknn0sh0000gn/T/parcels-501/lib7c3719442f0d622f710c8078b20b017b_0.so
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
Input In [10], in <cell line: 1>()
----> 1 pset.execute(kernels,                 # the kernel (which defines how particles move)
      2              runtime=86400*24,        # the total length of the run in seconds
      3              dt=300,                  # the timestep of the kernel in seconds
      4              output_file=output_file)

File ~/miniconda3/envs/parcels2/lib/python3.10/site-packages/parcels/particleset/baseparticleset.py:341, in BaseParticleSet.execute(self, pyfunc, pyfunc_inter, endtime, runtime, dt, moviedt, recovery, output_file, movie_background_field, verbose_progress, postIterationCallbacks, callbackdt)
    339         cppargs = ['-DDOUBLE_COORD_VARIABLES'] if self.collection.lonlatdepth_dtype else None
    340         self.kernel.compile(compiler=GNUCompiler(cppargs=cppargs, incdirs=[path.join(get_package_dir(), 'include'), "."]))
--> 341         self.kernel.load_lib()
    343 # Set up the interaction kernel(s) if not set and given.
    344 if self.interaction_kernel is None and pyfunc_inter is not None:

File ~/miniconda3/envs/parcels2/lib/python3.10/site-packages/parcels/kernel/basekernel.py:260, in BaseKernel.load_lib(self)
    259 def load_lib(self):
--> 260     self._lib = npct.load_library(self.lib_file, '.')
    261     self._function = self._lib.particle_loop

File ~/miniconda3/envs/parcels2/lib/python3.10/site-packages/numpy/ctypeslib.py:158, in load_library(libname, loader_path)
    156 if os.path.exists(libpath):
    157     try:
--> 158         return ctypes.cdll[libpath]
    159     except OSError:
    160         ## defective lib file
    161         raise

File ~/miniconda3/envs/parcels2/lib/python3.10/ctypes/__init__.py:449, in LibraryLoader.__getitem__(self, name)
    448 def __getitem__(self, name):
--> 449     return getattr(self, name)

File ~/miniconda3/envs/parcels2/lib/python3.10/ctypes/__init__.py:444, in LibraryLoader.__getattr__(self, name)
    442 if name[0] == '_':
    443     raise AttributeError(name)
--> 444 dll = self._dlltype(name)
    445 setattr(self, name, dll)
    446 return dll

File ~/miniconda3/envs/parcels2/lib/python3.10/ctypes/__init__.py:374, in CDLL.__init__(self, name, mode, handle, use_errno, use_last_error, winmode)
    371 self._FuncPtr = _FuncPtr
    373 if handle is None:
--> 374     self._handle = _dlopen(self._name, mode)
    375 else:
    376     self._handle = handle

OSError: dlopen(/var/folders/n5/_2d3c5sd5j3810327mknn0sh0000gn/T/parcels-501/lib7c3719442f0d622f710c8078b20b017b_0.so, 0x0006): tried: '/var/folders/n5/_2d3c5sd5j3810327mknn0sh0000gn/T/parcels-501/lib7c3719442f0d622f710c8078b20b017b_0.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/private/var/folders/n5/_2d3c5sd5j3810327mknn0sh0000gn/T/parcels-501/lib7c3719442f0d622f710c8078b20b017b_0.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))

From digging around it looks like another issue with the mac M1, and I think it's the same issue here https://github.com/nmwsharp/robust-laplacians-py/issues/7

Thanks

[erikvansebille]

Thanks for the update, @graceebc9. Could you try to install the parcels development again, but adapt line 8 in the environment_py3_osx.yml file from

  - clang_osx-64

to

  - clang_osx-arm64

I don't have an ARM-based mac myself, so can't test this

[erikvansebille]

Or, perhaps what you could try first (before reinstalling with clang_osx-arm64) is to issue

export CC=gcc 

Can you let me know if that worked?

[graceebc9]

Thanks @erikvansebille. I tried running 'export CC=gcc' in the command line whilst having the conda env active, and also tried running within the jupyter notebook I'm using as !export CC=gcc. Still getting the same error when running the kernel. Not sure if you meant for me to run that command elsewhere? THanks

[erikvansebille]

Thanks for trying @graceebc9. Just to be clear, the steps should be

1. activate your condo environment
2. type export CC=gcc
3. start Jupyter

If it still doesn't work with these three steps, could you try to reinstall the environment after replacing clang_osx-64 for clang_osx-arm64 (see also https://github.com/OceanParcels/parcels/issues/1220#issuecomment-1217542583 above)?

[philippemiron]

I'm currently out of the office and don't have access to my arm computer, but those are the notes I wrote a few months back.

First I used Miniforge (instead of miniconda), see more info on here.

wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
bash Miniforge3-MacOSX-arm64.sh

Then, I manually installed parcels using the following, I did not have to set any environment variables related to GCC.

conda env create -f environment_py3_osx_arm.yml
conda activate parcels
git clone https://github.com/OceanParcels/parcels
cd parcels/
python setup.py install

With the following environment file environment_py3_osx_arm.yml:

name: parcels
channels:
  - conda-forge

dependencies:
  - python>=3.4
  - cachetools>=1.0.0
  - cgen
  - clang_osx-arm64
  - coverage
  - ffmpeg>=3.2.3
  - flake8>=2.1.0
  - git
  - mpi4py>=3.0.1
  - mpich>=3.2.1
  - jupyter
  - matplotlib>=2.0.2
  - netcdf4>=1.1.9
  - numpy>=1.9.1
  - progressbar2
  - py>=1.4.27
  - pymbolic
  - python-dateutil
  - scipy>=0.16.0
  - six>=1.10.0
  - xarray>=0.10.8
  - dask>=2.0
  - cftime>=1.3.1
  - pytest
  - nbval
  - scikit-learn
  - pykdtree

[erikvansebille]

Has this been solved, @graceebc9? Perhaps try the new version v2.4.0 on https://anaconda.org/conda-forge/parcels, which also includes support for osx-arm64

[JamiePringle]

@erikvansebille I had the same problem with the latest version; fixed as you suggested with export CC=gcc

[ammedd]

I had similar problems installing and running parcels on my new MacBook with M2 chip. The only solution that works for me is the one philippemiron suggested, however I did use miniconda. And one can/should add pyproj.

https://github.com/OceanParcels/parcels/issues/1220#issuecomment-1221328527