library architecture mismatch libleveldb

[alva-seal]

Has this bug already been reported? Please check if there is a bug report already open that matches this.

Describe the bug bundled llibleveldb.dynlib is only x86 and won't work on arm (apple silicon M1)

To Reproduce Steps to reproduce the behavior: Installing amulet-map editor via python3 -m pip install amulet-map-editor starting it via python3 -m amulet_map_editor

Expected behavior starting of the editor

comandline

fabi@Fabians-Mac-mini Amulet-Map-Editor % python3 -m amulet_map_editor
Failed to import requirements. Check that you extracted correctly.
dlopen(/opt/homebrew/lib/python3.9/site-packages/amulet/libs/leveldb/libleveldb.dylib, 0x0006): tried: '/opt/homebrew/lib/python3.9/site-packages/amulet/libs/leveldb/libleveldb.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/libleveldb.dylib' (no such file), '/usr/lib/libleveldb.dylib' (no such file)
Traceback (most recent call last):
  File "/Users/fabi/amulet/Amulet-Map-Editor/amulet_map_editor/__main__.py", line 14, in <module>
    from amulet_map_editor.api.framework import AmuletApp
  File "/Users/fabi/amulet/Amulet-Map-Editor/amulet_map_editor/api/framework/__init__.py", line 1, in <module>
    from .amulet_ui import AmuletUI
  File "/Users/fabi/amulet/Amulet-Map-Editor/amulet_map_editor/api/framework/amulet_ui.py", line 6, in <module>
    from amulet.api.errors import LoaderNoneMatched
  File "/opt/homebrew/lib/python3.9/site-packages/amulet/__init__.py", line 14, in <module>
    from amulet.level.load import load_level, load_format
  File "/opt/homebrew/lib/python3.9/site-packages/amulet/level/__init__.py", line 1, in <module>
    from .load import load_format, load_level
  File "/opt/homebrew/lib/python3.9/site-packages/amulet/level/load.py", line 6, in <module>
    from amulet.api.level import World, Structure
  File "/opt/homebrew/lib/python3.9/site-packages/amulet/api/level/__init__.py", line 1, in <module>
    from .base_level import BaseLevel
  File "/opt/homebrew/lib/python3.9/site-packages/amulet/api/level/base_level/__init__.py", line 1, in <module>
    from .base_level import BaseLevel
  File "/opt/homebrew/lib/python3.9/site-packages/amulet/api/level/base_level/base_level.py", line 28, in <module>
    from .chunk_manager import ChunkManager
  File "/opt/homebrew/lib/python3.9/site-packages/amulet/api/level/base_level/chunk_manager.py", line 12, in <module>
    from amulet.api.cache import get_cache_db
  File "/opt/homebrew/lib/python3.9/site-packages/amulet/api/cache.py", line 7, in <module>
    from amulet.libs.leveldb import LevelDB
  File "/opt/homebrew/lib/python3.9/site-packages/amulet/libs/leveldb/__init__.py", line 1, in <module>
    from .leveldb import LevelDB, LevelDBException
  File "/opt/homebrew/lib/python3.9/site-packages/amulet/libs/leveldb/leveldb.py", line 22, in <module>
    ldb = ctypes.cdll.LoadLibrary(lib_path)
  File "/opt/homebrew/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 452, in LoadLibrary
    return self._dlltype(name)
  File "/opt/homebrew/Cellar/[email protected]/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ctypes/__init__.py", line 374, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/opt/homebrew/lib/python3.9/site-packages/amulet/libs/leveldb/libleveldb.dylib, 0x0006): tried: '/opt/homebrew/lib/python3.9/site-packages/amulet/libs/leveldb/libleveldb.dylib' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/libleveldb.dylib' (no such file), '/usr/lib/libleveldb.dylib' (no such file)

Desktop (please complete the following information):

• OS: macOS 12.0.1
• Version amulet-map-editor 0.8.15, python 3.9.7

Additional context as python is running natively on a apple silicon Mac (M1 processor) it seems to not be able to handle intel librarys. Is there a reason to not use the python package leveldb?

[gentlegiantJGC]

Minecraft uses a custom leveldb with a different compression algorithm so we have to compile it ourselves.

[gentlegiantJGC]

The code is here https://github.com/Amulet-Team/leveldb-mcpe

[alva-seal]

Thank you for pointing that out.

If someone has the same problem I attached the libleveldb.dynlib (as zip because GitHub would not let me do it otherwise) I compiled on the Apple Silicon M1. It can be placed instead of the bundled library.

I don't know if it is possible to put this one in the package for everyone and selecting the correct one automatically. libleveldb.dylib.zip

[gentlegiantJGC]

It looks like most apple computers have intel CPUs but the M1 has an Arm CPU and that is what is causing the issue.

[gentlegiantJGC]

I would like to work out how to compile a universal library that can work on intel and arm processors. Apparently that is possible. https://developer.apple.com/documentation/apple-silicon/building-a-universal-macos-binary

I have had a go but I can't work out how to do it from the command line through make. I don't have access to a macos device and github only has intel runners currently. I couldn't event get it to build just an arm version.

If anyone wants to have a go the code that builds them is here. https://github.com/Amulet-Team/leveldb-mcpe/blob/master/.github/workflows/build.yml#L61

[alva-seal]

Yes at the moment most Macs are still Intel but they will move to all M1 in the future.

The build I created this morning was done by just build the leveldb-mcpe library on my M1 Mac I use.

I copied the intel lib and the arm lib together with fabi@Fabians-Mac-mini leveldb-mcpe % lipo -create -output libleveldb.dylib libleveldb_arm.dylib leveldb_mcpe_macosx_10_9_x86_64.dylib But this leads to a codesiged error for me when executing. My guess is because the two librarys in that file are from different signatures. Unfortunatly I don't have acces so a intel Mac right now so I can't try that further right now.

[gentlegiantJGC]

Looking further into that document I linked earlier it isn't something we can just do from the command line. It will require modifying the makefile which isn't a format that I have used before.

[Jason-Silla]

Thank you for pointing that out.

If someone has the same problem I attached the libleveldb.dynlib (as zip because GitHub would not let me do it otherwise) I compiled on the Apple Silicon M1. It can be placed instead of the bundled library.

I don't know if it is possible to put this one in the package for everyone and selecting the correct one automatically. libleveldb.dylib.zip

What file is this. I don't know which one is the bundled library. If you could please help me, I would be very grateful.

[gentlegiantJGC]

I can't verify if the file linked is safe so only do this if you trust the user that created it. The file that you need to replace should be in amulet/libs/leveldb within your python site packages directory

[Jason-Silla]

I can't verify if the file linked is safe so only do this if you trust the user that created it. The file that you need to replace should be in amulet/libs/leveldb within your python site packages directory

IT WORKED!!! Thank you so much!

[matsgrotterud]

Can anyone give exact procedure to fix this issue? I did not understand what to replace here

[ViRb3]

Exhaustive building notes and pre-compiled app can be found in the issue linked above.

[gentlegiantJGC]

This issue should now be fixed with the latest leveldb library. If a pre-compiled version for your platform does not exist it will be compiled on your computer.