[FEATURE] Add arm support (e.g. for new Apple M1 processors) for Docker image

[spoonerWeb]

trafficstars

Is your feature request related to a problem? Please describe. Currently it's not possible to run the docker container (e.g. using ddev)

Describe the solution you'd like Support for arm architecture

Additional context Using typo3solr/solr image in ddev context.

docker-compose.solr.yaml

version: '3.6'
services:
  solr:
    container_name: ddev-${DDEV_SITENAME}-solr
    image: typo3solr/ext-solr:11.0.3
    restart: "no"
    expose:
      - 8983
    labels:
      com.ddev.site-name: ${DDEV_SITENAME}
      com.ddev.approot: $DDEV_APPROOT
    environment:
      - VIRTUAL_HOST=$DDEV_HOSTNAME
      - HTTP_EXPOSE=8983
    volumes:
      - "./solr:/opt/solr/server/solr"
  web:
    links:
      - solr

Error when starting on M1 processor:

Starting Solr 
OpenJDK 64-Bit Server VM warning: Failed to reserve shared memory. (error = 1)
OpenJDK 64-Bit Server VM warning: Failed to reserve shared memory. (error = 1)
OpenJDK 64-Bit Server VM warning: Failed to reserve shared memory. (error = 1)
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0000004017d4a4e0, pid=23, tid=122
#
# JRE version: OpenJDK Runtime Environment 18.9 (11.0.8+10) (build 11.0.8+10)
# Java VM: OpenJDK 64-Bit Server VM 18.9 (11.0.8+10, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# J 97 c1 java.lang.String.isEmpty()Z [email protected] (14 bytes) @ 0x0000004017d4a4e0 [0x0000004017d4a4c0+0x0000000000000020]
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /tmp/hs_err_pid23.log
Compiled method (c1)    2957   97       3       java.lang.String::isEmpty (14 bytes)
 total in heap  [0x0000004017d4a310,0x0000004017d4a688] = 888
 relocation     [0x0000004017d4a488,0x0000004017d4a4b8] = 48
 main code      [0x0000004017d4a4c0,0x0000004017d4a5e0] = 288
 stub code      [0x0000004017d4a5e0,0x0000004017d4a610] = 48
 metadata       [0x0000004017d4a610,0x0000004017d4a618] = 8
 scopes data    [0x0000004017d4a618,0x0000004017d4a630] = 24
 scopes pcs     [0x0000004017d4a630,0x0000004017d4a670] = 64
 dependencies   [0x0000004017d4a670,0x0000004017d4a678] = 8
 nul chk table  [0x0000004017d4a678,0x0000004017d4a688] = 16
Could not load hsdis-amd64.so; library not loadable; PrintAssembly is disabled
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
#
qemu: uncaught target signal 6 (Aborted) - core dumped

Target versions v10 and v11

[dkd-kaehm]

@spoonerWeb Thanks for reporting. We must check how to add platforms on docker hub.

But currently you can build the image locally, just refer to following snipped:

https://github.com/TYPO3-Solr/solr-ddev-site/blob/dd776dad79d3ca405652124bb91e5b5d1fd732be/.ddev/docker-compose.solr.yaml#L5-L8

version: '3.6'

services:
  solr-site:
    build:
      context:
        ../packages/ext-solr/
      dockerfile: Docker/SolrServer/Dockerfile

Note: EXT:solr or contents of EXT:Solr/Resources/Private/Solr must present before ddev start command is executed.

[spoonerWeb]

Thanks, this solution works for now.

[schliesser]

Building the image locally during ddev start solved the issue also for me on M1.

Had always HTTP ERROR 404 javax.servlet.UnavailableException: Error processing the request. CoreContainer is either not initialized or shutting down. errors with the image typo3solr/ext-solr:11.0.3.

As we run composer commands inside ddev and therefore the solr ext may not available, I've copied the necessary files of the extension to .ddev/solr-build to provide a clean and easy boot:

[baschny]

@dkd-kaehm we started building cross-plattform images lately (using Github Actions also) so if you need some help, we could offer some help in that regard.

Starting from solr 7.x, the official solr images (which you use as a base) are already build for arm64/v8 (M1 compatible), so it would be pretty easy to have them build for you too.

I see you are building and testing the docker images via Github Actions already, but then upload the artefacts. How does this image end up in docker hub at the end?

Since you are using the pre-configured actions from buildx, refer to this example on how to "enhance" your build process to produce a multi-arch image (it has to be done in the same step, since the manifest must include both platforms before uploading them to docker-hub):

https://github.com/docker/build-push-action/blob/master/docs/advanced/multi-platform.md

What you need to add is:

1. the qemu action (in order to build for arm even on the amd64 platforms that Github Actions provides us). This can be slow if you need to compile stuff, but since you are just copying files, it should be enough

2. configure the additional platforms in the docker/build-push-action action.

[dkd-kaehm]

@dkd-kaehm we started building cross-plattform images lately (using Github Actions also) so if you need some help, we could offer some help in that regard.

Starting from solr 7.x, the official solr images (which you use as a base) are already build for arm64/v8 (M1 compatible), so it would be pretty easy to have them build for you too.

I see you are building and testing the docker images via Github Actions already, but then upload the artefacts. How does this image end up in docker hub at the end?

Since you are using the pre-configured actions from buildx, refer to this example on how to "enhance" your build process to produce a multi-arch image (it has to be done in the same step, since the manifest must include both platforms before uploading them to docker-hub):

https://github.com/docker/build-push-action/blob/master/docs/advanced/multi-platform.md

What you need to add is:

1. the qemu action (in order to build for arm even on the amd64 platforms that Github Actions provides us). This can be slow if you need to compile stuff, but since you are just copying files, it should be enough

2. configure the additional platforms in the `docker/build-push-action` action.

@baschny Thanks for advice, currently the GitHub-Actions and Docker Hub builds are not related/connected with each other, we are using the Automated Builds of Docker Hub, see https://docs.docker.com/docker-hub/builds/ and https://github.com/docker/roadmap/issues/109.

To provide the ARM images we must switch to the actions, which is currently unfavorable, because it is still possible to build the images locally, but in the future we'll of course do that.

[DenisMir]

Any progress on this topic?

[dkd-kaehm]

Any progress on this topic?

Currently no plans to switch to other docker build stack. All our power is now on TYPO3 12 compatibility and maintaining the supported versions. If somebody did the builds of docker images via GH-Actions already, please let us know and integrate the new build stack together. Of course the pull-reqeuests with POCs for that task and other are welcome.

[sjorek]

… Of course the pull-reqeuests with POCs for that task and other are welcome.

Here you are: #3501. It still lacks a POC, but it's a first step.