Building for M1 Mac

[ViRb3]

Currently, Amulet fails to build on an M1 Mac due to some minor issues. Here are my exhaustive notes on how to build it successfully, and pre-compiled binaries at the bottom. Hopefully this won't be necessary eventually :)

1. Create virtual environment (venv)

# ensure you have Python 3.9, as Python 3.10 and above crash on world load [1]
brew install [email protected]
# install venv manager of choice
pip install poetry
# create venv using Python 3.9
mkdir amulet && cd amulet
poetry init --python 3.9.x -n
# store venv in the same directory for this project
poetry config virtualenvs.in-project true --local
# make sure Python 3.9 is enabled for this venv
poetry env use /opt/homebrew/Cellar/[email protected]/3.9.13_1/bin/python3

2. Install wxpython

• Method A - from homebrew

  # install wxpython
  brew install wxpython
  # link brew's wxpython into the venv
  cd .venv/lib/python3.9/site-packages
  ln -s /opt/homebrew/Cellar/wxpython/4.1.1_2/lib/python3.9/site-packages/wx
  ln -s /opt/homebrew/Cellar/wxpython/4.1.1_2/lib/python3.9/site-packages/wxPython-4.1.1-py3.9.egg-info
  cd -
  

• Method B - from source

  # install wxpython dependency
  brew install libtiff
  # prepare for build
  export LDFLAGS=-L/opt/homebrew/lib
  export CPPFLAGS=-I/opt/homebrew/include
  # install wxpython
  poetry add wxpython==4.1.1
  

3. Install Amulet Editor

poetry add amulet-map-editor==0.9.4

4. Fix leveldb-mcpe [2]

# clone project
git clone https://github.com/Amulet-Team/leveldb-mcpe
cd leveldb-mcpe
# disable unsupported SSE extensions
sed -i '' 's/PLATFORM_SSEFLAGS="-msse4.2"/PLATFORM_SSEFLAGS=""/' build_detect_platform
# build
make -j8
# install by replacing the original file from venv
cp out-shared/libleveldb.dylib.1.20 ../.venv/lib/python3.9/site-packages/amulet/libs/leveldb/leveldb_mcpe_macosx_10_9_x86_64.dylib
cd -

5. Run Amulet Editor

poetry run python3 -m amulet_map_editor

BONUS - Create self-contained app

Save the following bootstrap code as main.py. It is necessary because we will use PyInstaller in GUI mode, and we have to override the default working directory from root (/) to something Amulet can write in, otherwise there will be instant crash.

from pathlib import Path
import os

appdata = Path.home() / "Library/Application Support" / "Amulet Editor"
appdata.mkdir(0o755, True, True)
os.chdir(appdata)

from amulet_map_editor.api.framework import AmuletApp

app = AmuletApp(0)
app.MainLoop()

Once you have this file, simply:

# install pyinstaller as dev dependency
poetry add -D pyinstaller
# generate app, preserving unrecognized resources from amulet_map_editor
poetry run pyinstaller main.py --collect-all amulet_map_editor --collect-all minecraft_model_reader --windowed --name "Amulet Editor"

Viola - you have your final product in ./dist/main.app!

BONUS 2 - Pre-compiled download

Amulet Editor-0.9.4.tar.xz.zip

References

[1] - https://github.com/Amulet-Team/Amulet-Map-Editor/issues/465

[2] - https://github.com/Amulet-Team/Amulet-Map-Editor/issues/455

[nicwn]

Following along but using pyenv to install framework build Python:

$ env PYTHON_CONFIGURE_OPTS="--enable-framework" pyenv install 3.9.6

[robichaudc]

Just wanted to say thanks, Method A worked for me on a M1 mac running Monterey 12.2.1 with amulet 0.8..20

I tried updating to use the latest amulet version (0.9.4) but couldn't figure out how to get that working.

[ViRb3]

The exact same instructions work for 0.9.4, just need:

poetry add amulet-map-editor==0.9.4

instead of:

poetry add amulet-map-editor==0.8.20

Updated binaries: Amulet Editor-0.9.4.tar.xz.zip

[Andripao11]

I have a problem, when I run mkdir amulet && cd amulet poetry init --python 3.9.x -n I get mkdir: amulet: File exists zsh: command not found: poetry

[ViRb3]

You need to install poetry as written in the guide.

[Andripao11]

Yes I did brew install python and pip install poetry but I stil get the error

[ViRb3]

Hmm, does /opt/homebrew/bin/poetry exist? If yes, add /opt/homebrew/bin to your path. Otherwise something went wrong with your installation.

[Andripao11]

I found out that when I do python3 --version it tells me Python 3.8.9 Is it normal?

[Andripao11]

I upgraded python to 3.9 and doing all the commands again it worked, thanks for the guide!

[gentlegiantJGC]

I think this issue should be fixed in 0.10.2b3 now that I have integrated the new LevelDB library. If your platform does not have a pre-built version it will compile all of the LevelDB code on your computer.

[stale[bot]]

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[ViRb3]

Summary for Amulet 0.10.5 on macOS 13 M1: it is completely broken :(

Here's some notes:

• wxPython still contains no pre-built wheels in pip, and it fails to build automatically due to missing dependencies
• wxPython in brew is now at 4.2.0 while amulet still uses 4.1.1, so symlinking this no longer works
• wxPython can still be built manually using my "Method B" in my first post
• amulet now lists pyopengl-accelerate~=3.0 as a hard dependency, but it has not been updated in years and does not support M1. Therefore, installing amulet now straight up fails on M1 using any package manager. I had to remove the dependency and build from source, then it worked: https://github.com/Amulet-Team/Amulet-Map-Editor/blob/0.10/setup.cfg#L21. @gentlegiantJGC you should be able to use a conditional statement to exclude this dependency for unsupported platforms like the M1 mac. The map editor still works, it's just a bit slower, not noticeable on M1 mac.
• leveldb-mcpe now builds without any extra changes, nice!

With these changes, the rest of my building guide is the same. And finally, amulet works! However, when you open a world, it does not render at all, it just remains blank:

The error seems to be shader compilation due to unsupported OpenGL extension: https://paste.gg/p/anonymous/3830daa23f0243dca1554a2bb4ae388c

The same world renders just fine with Amulet 0.9.4.

Thanks!

[gentlegiantJGC]

Thanks. I will look into this. I have bought an old Macbook so debugging these issues should be possible. Not sure when I will get around to it though. I have started working on the Qt port which should solve some of the other Mac and Linux issues.

[ViRb3]

Feel free to ask if you want anything tested in the meantime, I would be more than happy to help. I think the only blocking issue at the moment is the opengl shader issue, apart from that amulet would be fully usable, even if slightly harder to build. Do you have any ideas about the shader? Thanks

[ViRb3]

Worth noting - macOS only supports up to OpenGL 4.1/OpenCL 1.2, and it's now even deprecated. If you are targeting any version later than that, it's probably the cause.

[gentlegiantJGC]

I experienced the same error when trying to run it on the macbook but I haven't had a chance to debug it. We don't define a minimum OpenGL version. It uses the GPU driver defaults. Wx Wiki: If no attributes are defined at all, then those provided by the GPU driver defaults will be used.

[gentlegiantJGC]

OpenGL is a compatibility nightmare.

[Max-RM]

Thanks. I will look into this. I have bought an old Macbook so debugging these issues should be possible. Not sure when I will get around to it though. I have started working on the Qt port which should solve some of the other Mac and Linux issues.

Cool! looking forward!

[gentlegiantJGC]

I can't access the paste url from a few messages back. The error was along these lines though

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/amulet_map_editor/programs/edit/plugins/tools/select.py", line 329, in _on_draw
    self.canvas.renderer.draw_level()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/amulet_map_editor/programs/edit/api/renderer.py", line 228, in draw_level
    self.render_world.draw(self.canvas.camera.transformation_matrix)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/amulet_map_editor/api/opengl/mesh/level/level.py", line 273, in draw
    self._chunk_manager.draw(camera_matrix, self.camera_location)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/amulet_map_editor/api/opengl/mesh/level/region.py", line 96, in draw
    region.draw(camera_matrix, cam_cx, cam_cz)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/amulet_map_editor/api/opengl/mesh/level/region.py", line 272, in draw
    self._create_geometry()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/amulet_map_editor/api/opengl/mesh/level/region.py", line 252, in _create_geometry
    self._setup()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/amulet_map_editor/api/opengl/mesh/tri_mesh.py", line 90, in _setup
    self._shader = get_shader(self.context_identifier, self.shader_name)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/amulet_map_editor/api/opengl/shaders/__init__.py", line 60, in get_shader
    shader = compile_shader()
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/amulet_map_editor/api/opengl/shaders/__init__.py", line 49, in compile_shader
    _load_shader(
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/amulet_map_editor/api/opengl/shaders/__init__.py", line 73, in _load_shader
    return OpenGL.GL.shaders.compileShader(shader, shader_type)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/OpenGL/GL/shaders.py", line 235, in compileShader
    raise ShaderCompilationError(
OpenGL.GL.shaders.ShaderCompilationError: ('Shader compile failure (0): b"WARNING: 0:2: extension \'GL_ARB_explicit_attrib_location\' is not supported\\nERROR: 0:4: \'layout\' : syntax error: syntax error\\n"', [b'#version 120\r\n#extension GL_ARB_explicit_attrib_location : enable\r\n\r\nlayout(location = 0) in vec3 positions;\r\nlayout(location = 1) in vec2 vTexCoord;\r\nlayout(location = 2) in vec4 vTexOffset;\r\nlayout(location = 3) in vec3 vTint;\r\n\r\nvarying vec2 fTexCoord;\r\nvarying vec4 fTexOffset;\r\nvarying vec3 fTint;\r\n\r\nuniform mat4 transformation_matrix;\r\n\r\nvoid main(){\r\n    gl_Position = transformation_matrix * vec4(positions, 1.0);\r\n    fTexCoord = vTexCoord;\r\n    fTexOffset = vTexOffset;\r\n    fTint = vTint;\r\n}'], GL_VERTEX_SHADER)

Nicely formatted error

WARNING: 0:2: extension 'GL_ARB_explicit_attrib_location' is not supported
ERROR: 0:4: 'layout' : syntax error: syntax error
 
#version 120
#extension GL_ARB_explicit_attrib_location : enable
layout(location = 0) in vec3 positions;
layout(location = 1) in vec2 vTexCoord;
layout(location = 2) in vec4 vTexOffset;
layout(location = 3) in vec3 vTint;
varying vec2 fTexCoord;
varying vec4 fTexOffset;
varying vec3 fTint;
uniform mat4 transformation_matrix;
void main(){
    gl_Position = transformation_matrix * vec4(positions, 1.0);
    fTexCoord = vTexCoord;
    fTexOffset = vTexOffset;
    fTint = vTint;
}

@ViRb3 @DBarned do you get the same issue in previous versions of Amulet? I don't know why this issue has popped up now when it has been working before. We haven't changed any of the opengl code in a long while.

This issue does happen on my test machine so I will be able to debug the issue but I don't know when that will be.

[DBarned]

Version 0.9.6 still works fine for me with older worlds, but any 0.10.x newer versions have the issue for me.

[ViRb3]

Confirming the above. Maybe you updated a dependency which pulled support for additional OpenGL extensions? Maybe some defaults changed? It should be possible to specify which extensions you want to enable in pyopengl. Also if you're referring to my paste, it works just fine here, could you have a DNS issue?

OpenGL.GL.shaders.ShaderCompilationError: ('Shader compile failure (0): b"WARNING: 0:2: extension \'GL_ARB_explicit_attrib_location\' is not supported\\nERROR: 0:4: \'layout\' : syntax error: syntax error\\n"', [b'#version 120\n#extension GL_ARB_explicit_attrib_location : enable\n\nlayout(location = 0) in vec3 positions;\nlayout(location = 1) in vec2 vTexCoord;\nlayout(location = 2) in vec4 vTexOffset;\nlayout(location = 3) in vec3 vTint;\n\nvarying vec2 fTexCoord;\nvarying vec4 fTexOffset;\nvarying vec3 fTint;\n\nuniform mat4 transformation_matrix;\n\nvoid main(){\n    gl_Position = transformation_matrix * vec4(positions, 1.0);\n    fTexCoord = vTexCoord;\n    fTexOffset = vTexOffset;\n    fTint = vTint;\n}'], GL_VERTEX_SHADER)
Traceback (most recent call last):
  File "/Users/user/Desktop/amulet/.venv/lib/python3.9/site-packages/amulet_map_editor/programs/edit/plugins/tools/select.py", line 330, in _on_draw
    self.canvas.renderer.draw_sky_box()
  File "/Users/user/Desktop/amulet/.venv/lib/python3.9/site-packages/amulet_map_editor/programs/edit/api/renderer.py", line 224, in draw_sky_box
    self.sky_box.draw(self.canvas.camera.transformation_matrix)
  File "/Users/user/Desktop/amulet/.venv/lib/python3.9/site-packages/amulet_map_editor/api/opengl/mesh/sky_box.py", line 63, in draw
    super().draw(
  File "/Users/user/Desktop/amulet/.venv/lib/python3.9/site-packages/amulet_map_editor/api/opengl/mesh/tri_mesh.py", line 161, in draw
    self._setup()
  File "/Users/user/Desktop/amulet/.venv/lib/python3.9/site-packages/amulet_map_editor/api/opengl/mesh/tri_mesh.py", line 90, in _setup
    self._shader = get_shader(self.context_identifier, self.shader_name)
  File "/Users/user/Desktop/amulet/.venv/lib/python3.9/site-packages/amulet_map_editor/api/opengl/shaders/__init__.py", line 60, in get_shader
    shader = compile_shader()
  File "/Users/user/Desktop/amulet/.venv/lib/python3.9/site-packages/amulet_map_editor/api/opengl/shaders/__init__.py", line 49, in compile_shader
    _load_shader(
  File "/Users/user/Desktop/amulet/.venv/lib/python3.9/site-packages/amulet_map_editor/api/opengl/shaders/__init__.py", line 73, in _load_shader
    return OpenGL.GL.shaders.compileShader(shader, shader_type)
  File "/Users/user/Desktop/amulet/.venv/lib/python3.9/site-packages/OpenGL/GL/shaders.py", line 235, in compileShader
    raise ShaderCompilationError(

[gentlegiantJGC]

I think that pull request should fix the opengl-accelerate and the shader compilation errors. It fixes the shader compilation error on my old macbook. I reverted part of the code back to what was in 0.9.6. I must have assumed it was not needed.

I think the only remaining issue is the installer step. I have refactored FBS to remove the runtime dependency so perhaps I could look into using it for this version of Amulet. It was originally intended for the Qt port.

[smertz001]

I'm having this similar issue with my Intel Mac Pro, as I mentioned in #793

If there are instructions on how to test this, I will be more than willing to help test it.

[gentlegiantJGC]

This command should install it I think. pip install https://github.com/Amulet-Team/Amulet-Map-Editor/archive/fix-macos.zip

You may need to prefix python3 -m

[smertz001]

I gave this a try, and my first run through I had to install the wheel module. So that might be something that needs to be added to the requirements for installing in the future so that it installs automatically as needed.

My second time through I got: (Amulet) steve@Steves-Mac-Pro Amulet % python3 -m pip install https://github.com/Amulet-Team/Amulet-Map-Editor/archive/fix-macos.zip Collecting https://github.com/Amulet-Team/Amulet-Map-Editor/archive/fix-macos.zip Using cached https://github.com/Amulet-Team/Amulet-Map-Editor/archive/fix-macos.zip Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... done Requirement already satisfied: pyopengl-accelerate~=3.0 in ./lib/python3.9/site-packages (from amulet-map-editor===refs-pull-847-head) (3.1.5) Requirement already satisfied: amulet-nbt~=2.0 in ./lib/python3.9/site-packages (from amulet-map-editor===refs-pull-847-head) (2.0.3) Requirement already satisfied: numpy~=1.17 in ./lib/python3.9/site-packages (from amulet-map-editor===refs-pull-847-head) (1.23.5) Requirement already satisfied: packaging in ./lib/python3.9/site-packages (from amulet-map-editor===refs-pull-847-head) (22.0) Requirement already satisfied: pymctranslate~=1.2 in ./lib/python3.9/site-packages (from amulet-map-editor===refs-pull-847-head) (1.2.9) Requirement already satisfied: pyopengl~=3.0 in ./lib/python3.9/site-packages (from amulet-map-editor===refs-pull-847-head) (3.1.6) Requirement already satisfied: amulet-core~=1.9 in ./lib/python3.9/site-packages (from amulet-map-editor===refs-pull-847-head) (1.9.11) Requirement already satisfied: wxPython==4.1.1 in ./lib/python3.9/site-packages (from amulet-map-editor===refs-pull-847-head) (4.1.1) Requirement already satisfied: minecraft-resource-pack~=1.3 in ./lib/python3.9/site-packages (from amulet-map-editor===refs-pull-847-head) (1.3.3) Requirement already satisfied: six in ./lib/python3.9/site-packages (from wxPython==4.1.1->amulet-map-editor===refs-pull-847-head) (1.16.0) Requirement already satisfied: pillow in ./lib/python3.9/site-packages (from wxPython==4.1.1->amulet-map-editor===refs-pull-847-head) (9.3.0) Requirement already satisfied: amulet-leveldb~=1.0b0 in ./lib/python3.9/site-packages (from amulet-core~=1.9->amulet-map-editor===refs-pull-847-head) (1.0.0b5) Requirement already satisfied: portalocker~=2.4 in ./lib/python3.9/site-packages (from amulet-core~=1.9->amulet-map-editor===refs-pull-847-head) (2.6.0) Requirement already satisfied: mutf8~=1.0 in ./lib/python3.9/site-packages (from amulet-nbt~=2.0->amulet-map-editor===refs-pull-847-head) (1.0.6) Building wheels for collected packages: amulet-map-editor Building wheel for amulet-map-editor (pyproject.toml) ... done Created wheel for amulet-map-editor: filename=amulet_map_editor-refs_pull_847_head-cp39-cp39-macosx_10_9_x86_64.whl size=2061984 sha256=d0821769e8ed2fcdcb8984e67b746aa1b790b2d7535a706b3e95376f46806b99 Stored in directory: /Users/steve/Library/Caches/pip/wheels/75/a0/b8/a266cd32bd56cdd3da4205c3782d79f4cab497b494afa19eed WARNING: Built wheel for amulet-map-editor is invalid: Metadata 1.2 mandates PEP 440 version, but 'refs-pull-847-head' is not Failed to build amulet-map-editor ERROR: Could not build wheels for amulet-map-editor, which is required to install pyproject.toml-based projects

[gentlegiantJGC]

Hmm. Wheel is already a pre-install dependency so I don't know what is going on there. https://github.com/Amulet-Team/Amulet-Map-Editor/blob/0.10/pyproject.toml#L4

Try downloading the zip from the url, extract and then pip install <path> where is the path to the unzipped directory.

[gentlegiantJGC]

If that fails I can make a beta release with the changes.

[smertz001]

Yup, extracting the zip and installing it that way fails the same.

[gentlegiantJGC]

Try this python -m pip install amulet-map-editor==0.10.7b0 --upgrade

[smertz001]

It installed cleanly and the world loaded up correctly! So far so good! Thanks !