trino
trino copied to clipboard
trinodb
Added ppc64le support
Description
Enabled ppc64le (IBM Power architecture) builds by adding an appropriate entry in ci.yml. The complete code was already ready for ppc64le support, so only this minor change was needed.
Is this change a fix, improvement, new feature, refactoring, or other?
New feature (additional architecture support).
Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)
No.
How would you describe this change to a non-technical end user or system administrator?
Support for running Trino on IBM Power servers.
Documentation
() No documentation is needed. (X) Sufficient documentation is included in this PR. ( ) Documentation PR is available with #prnumber. ( ) Documentation issue #issuenumber is filed, and can be handled later.
Release notes
( ) No release notes entries required. (X) Release notes entries required with the following suggested text: Added support for running Trino on IBM Power servers (ppc64le architecture).
Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla
![cla-bot[bot] avatar](https://avatars.githubusercontent.com/in/2404?v=4 "Published by a pub.dev verified publisher"){kind=small}
Thanks for starting the discussion on this, which prompted me to look at Temurin. They seem to produce high-quality Docker images, which look like a good option to replace the current UBI / Zulu images we've been using. I created a PR to switch to Temurin and add support for ppc64le: https://github.com/trinodb/trino/pull/13522
Unfortunately, there is a qemu-user-static bug which prevents running the JDK under emulation with Docker, so it would be great if you could test this change on the real hardware.
Building the docker image on ppc64le hardware works now (I checked out master & simply build using 392 artifacts):
root@p1313-kvm1:~/trino/core/docker# uname -a
Linux p1313-kvm1 4.15.0-171-generic #180-Ubuntu SMP Wed Mar 2 17:25:07 UTC 2022 ppc64le ppc64le ppc64le GNU/Linux
root@p1313-kvm1:~/trino/core/docker#
root@p1313-kvm1:~/trino/core/docker#
root@p1313-kvm1:~/trino/core/docker# ./build.sh -r 392 -a ppc64le
🎣 Downloading server and client artifacts for release version 392
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.apache.maven:standalone-pom >-------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:get (default-cli) @ standalone-pom ---
[WARNING] The artifact xml-apis:xml-apis:jar:2.0.2 has been relocated to xml-apis:xml-apis:jar:1.0.b2
[INFO] Resolving io.trino:trino-server:tar.gz:392
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.736 s
[INFO] Finished at: 2022-08-08T07:54:35-04:00
[INFO] ------------------------------------------------------------------------
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------< org.apache.maven:standalone-pom >-------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:get (default-cli) @ standalone-pom ---
[WARNING] The artifact xml-apis:xml-apis:jar:2.0.2 has been relocated to xml-apis:xml-apis:jar:1.0.b2
[INFO] Resolving io.trino:trino-cli:jar:executable:392
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.619 s
[INFO] Finished at: 2022-08-08T07:54:37-04:00
[INFO] ------------------------------------------------------------------------
🧱 Preparing the image build context directory
🫙 Building the image for ppc64le
[+] Building 29.1s (11/11) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 1.39kB 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/eclipse-temurin:17.0.4_8-jdk 0.8s
=> [auth] library/eclipse-temurin:pull token for registry-1.docker.io 0.0s
=> [1/5] FROM docker.io/library/eclipse-temurin:17.0.4_8-jdk@sha256:6db573b0312c640a2c557173df9e35cb84eb1d66ee564be7b6ed9693bcf71834 8.0s
=> => resolve docker.io/library/eclipse-temurin:17.0.4_8-jdk@sha256:6db573b0312c640a2c557173df9e35cb84eb1d66ee564be7b6ed9693bcf71834 0.1s
=> => sha256:6db573b0312c640a2c557173df9e35cb84eb1d66ee564be7b6ed9693bcf71834 1.70kB / 1.70kB 0.0s
=> => sha256:3f728a7ba069f8153d61adb0bdbb3fa13b0898f86a600efb2d1e2a61f1bf018a 1.16kB / 1.16kB 0.0s
=> => sha256:f5ee2436379970ef4941c4810e34575c931bf2be8f555d3133dbda3a83c4eb25 5.10kB / 5.10kB 0.0s
=> => sha256:6f8c2fc0a4f976c1c0bd1c0e14022b3ed8b7c83cdb247c83016052c3678aaf28 35.72MB / 35.72MB 1.5s
=> => sha256:b415449d41b9dcf3446ca4a7c8cd4460dfe52b1b9da2299c0dee989c007c98fe 17.87MB / 17.87MB 0.4s
=> => sha256:b47a250aca32b54d3293aa909415c63f1b3b45b77986e82c4999d6e0271d262f 191.45MB / 191.45MB 4.1s
=> => sha256:21bf72b35869d648077ace71b46d58a301c242dcf81a98bb99607db967e28c2f 160B / 160B 0.5s
=> => extracting sha256:6f8c2fc0a4f976c1c0bd1c0e14022b3ed8b7c83cdb247c83016052c3678aaf28 1.1s
=> => extracting sha256:b415449d41b9dcf3446ca4a7c8cd4460dfe52b1b9da2299c0dee989c007c98fe 0.6s
=> => extracting sha256:b47a250aca32b54d3293aa909415c63f1b3b45b77986e82c4999d6e0271d262f 2.2s
=> => extracting sha256:21bf72b35869d648077ace71b46d58a301c242dcf81a98bb99607db967e28c2f 0.0s
=> [internal] load build context 4.4s
=> => transferring context: 714.86MB 4.3s
=> [2/5] RUN set -xeu && apt-get update -q && apt-get install -y -q less python3 && rm -rf /var/lib/apt/lists/* && update-alternatives --install /usr/bin/python python 14.3s
=> [3/5] COPY trino-cli-392-executable.jar /usr/bin/trino 0.2s
=> [4/5] COPY --chown=trino:trino trino-server-392 /usr/lib/trino 1.6s
=> [5/5] COPY --chown=trino:trino default/etc /etc/trino 0.2s
=> exporting to image 3.7s
=> => exporting layers 3.7s
=> => writing image sha256:73d21a845469764a3eb0bad5a4d051a9d545ed3fbece55e29c5d7af82d2999c8 0.0s
=> => naming to docker.io/library/trino:392-ppc64le 0.0s
🧹 Cleaning up the build context directory
🏃 Testing built images
🚀 Built [trino:392-ppc64le] sha256:73d21a845469764a3eb0bad5a4d051a9d545ed3fbece55e29c5d7af82d2999c8
root@p1313-kvm1:~/trino/core/docker#
root@p1313-kvm1:~/trino/core/docker#
root@p1313-kvm1:~/trino/core/docker# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
trino 392-ppc64le 73d21a845469 43 seconds ago 1.25GB
Starting looks good as well:
root@p1313-kvm1:~/trino/core/docker# docker run -p 8080:8080 --name trino trino:392-ppc64le
...
2022-08-08T12:06:17.588Z INFO main io.trino.server.Server ======== SERVER STARTED ========
I noticed a minor (documentation) bug - ppc64le needs to be added here now: https://github.com/trinodb/trino/blob/master/core/docker/build.sh#L11
The trino client appears to be buggy for my setup - my shell freezes/looses its prompt exactly here:
root@p1313-kvm1:~# docker exec -it trino trino
trino>
Something like this is possible, though:
root@p1313-kvm1:~# docker exec -it trino /bin/bash
trino@c4c60adf4688:/$
trino@c4c60adf4688:/$ trino --version
Trino CLI 392
But just starting trino results in a freeze, again.
Try running the CLI with JNA disabled:
java -Dorg.jline.terminal.jna=false -jar /usr/bin/trino
Thanks for testing.
The CLI is using JLine. Could you see if their demo works or has the same problem?
On a ppc64le machine, I went into this container:
docker run -it --user root:root --entrypoint=/bin/bash eclipse-temurin:17.0.4_8-jdk
And essentially build JLine:
root@4bdc8a4fe8a9:/jline3# ./build rebuild
...
[INFO] Reactor Summary:
[INFO]
[INFO] JLine Parent 3.21.1-SNAPSHOT ....................... SUCCESS [ 13.288 s]
[INFO] JLine Terminal ..................................... SUCCESS [ 15.950 s]
[INFO] JLine JNA Terminal ................................. SUCCESS [ 5.983 s]
[INFO] JLine JANSI Terminal ............................... SUCCESS [ 4.590 s]
[INFO] JLine Reader ....................................... SUCCESS [ 14.740 s]
[INFO] JLine Style ........................................ SUCCESS [ 4.486 s]
[INFO] JLine Builtins ..................................... SUCCESS [ 10.237 s]
[INFO] JLine Console ...................................... SUCCESS [ 8.618 s]
[INFO] JLine Groovy ....................................... SUCCESS [ 11.138 s]
[INFO] JLine Remote SSH ................................... SUCCESS [ 4.079 s]
[INFO] JLine Remote Telnet ................................ SUCCESS [ 2.949 s]
[INFO] JLine Demo ......................................... SUCCESS [ 7.253 s]
[INFO] JLine Graal Demo ................................... SUCCESS [ 3.500 s]
[INFO] JLine Bundle 3.21.1-SNAPSHOT ....................... SUCCESS [ 19.134 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:06 min
[INFO] Finished at: 2022-08-08T19:58:31Z
[INFO] ------------------------------------------------------------------------
This demo, for example, is working:
root@4bdc8a4fe8a9:/jline3# ./build demo
Launching Gogo JLine...
Classpath: /jline3/demo/target/classes:/jline3/demo/target/lib/jline-remote-telnet-3.21.1-SNAPSHOT.jar:/jline3/demo/target/lib/jline-groovy-3.21.1-SNAPSHOT.jar:/jline3/demo/target/lib/jline-console-3.21.1-SNAPSHOT.jar:/jline3/demo/target/lib/jline-terminal-3.21.1-SNAPSHOT.jar:/jline3/demo/target/lib/jline-terminal-jna-3.21.1-SNAPSHOT.jar:/jline3/demo/target/lib/jline-terminal-jansi-3.21.1-SNAPSHOT.jar:/jline3/demo/target/lib/jline-remote-ssh-3.21.1-SNAPSHOT.jar:/jline3/demo/target/lib/jline-style-3.21.1-SNAPSHOT.jar:/jline3/demo/target/lib/jline-reader-3.21.1-SNAPSHOT.jar:/jline3/demo/target/lib/jline-builtins-3.21.1-SNAPSHOT.jar:/jline3/demo/target/lib/org.apache.felix.gogo.runtime-1.1.4.jar:/jline3/demo/target/lib/org.apache.felix.gogo.jline-1.1.8.jar
____________________________
Welcome to Apache Felix Gogo
g! 19:59:19
g! 19:59:20
g! ls 19:59:21
LICENSE.txt appveyor.yml build.config changelog.md demo groovy jline mvnw.cmd reader remote-telnet terminal terminal-jna
.. README.md build builtins console graal header.txt mvnw pom.xml remote-ssh style terminal-jansi
g!
... the Groovy REPL demos works as well.
Try running the CLI with JNA disabled:
java -Dorg.jline.terminal.jna=false -jar /usr/bin/trinoThanks for testing.
This one works:
trino@4a11426d6a10:/$ java -Dorg.jline.terminal.jna=false -jar /usr/bin/trino
trino>
trino>
trino> help
Supported commands:
QUIT
EXIT
CLEAR
EXPLAIN [ ( option [, ...] ) ] <query>
options: FORMAT { TEXT | GRAPHVIZ | JSON }
TYPE { LOGICAL | DISTRIBUTED | VALIDATE | IO }
DESCRIBE <table>
SHOW COLUMNS FROM <table>
SHOW FUNCTIONS
SHOW CATALOGS [LIKE <pattern>]
SHOW SCHEMAS [FROM <catalog>] [LIKE <pattern>]
SHOW TABLES [FROM <schema>] [LIKE <pattern>]
USE [<catalog>.]<schema>
trino>
@lehrig can you build last released version and test it?
@wendigo what do you mean exactly?
- The trino client in 392 (which I simply reused from the released binaries)?
- The docker image (which I already did)?
- Something from JLine?
@lehrig Jline. There were couple of changes since last release so I'd like to rule them out
@lehrig Jline. There were couple of changes since last release so I'd like to rule them out
Sure, so I proceeded as before but did git checkout jline-parent-3.21.0 before building.
Build was again successful and ./build demo worked as well.
Hmm, I wonder if their demo is using Jansi rather than JNA by default.
Good point - I found this line here: https://github.com/jline/jline3/blob/jline-parent-3.21.0/demo/jline-gogo.sh#L51
So, I rerun the demo using this:
./build demo jna
Result:
Launching Gogo JLine...
Classpath: /jline3/demo/target/classes:/jline3/demo/target/lib/jline-remote-telnet-3.21.0.jar:/jline3/demo/target/lib/jline-terminal-jna-3.21.0.jar:/jline3/demo/target/lib/jline-reader-3.21.0.jar:/jline3/demo/target/lib/jline-builtins-3.21.0.jar:/jline3/demo/target/lib/jline-remote-ssh-3.21.0.jar:/jline3/demo/target/lib/jline-terminal-jansi-3.21.0.jar:/jline3/demo/target/lib/jline-console-3.21.0.jar:/jline3/demo/target/lib/jline-style-3.21.0.jar:/jline3/demo/target/lib/jline-terminal-3.21.0.jar:/jline3/demo/target/lib/jline-groovy-3.21.0.jar:/jline3/demo/target/lib/org.apache.felix.gogo.runtime-1.1.4.jar:/jline3/demo/target/lib/org.apache.felix.gogo.jline-1.1.8.jar:/jline3/demo/target/lib/jna-5.9.0.jar
____________________________
Welcome to Apache Felix Gogo
g!
...and it's frozen from here.
Edit: same behavior on jline-parent-3.21.0 and master branches.
I have seen that there is the new trino 393 release, however, there are no ppc64le images pushed to docker.io. I thought this would happen now with https://github.com/trinodb/trino/pull/13522...
So, what else is missing?
We forgot to include the new architecture in the release scripts: https://github.com/trinodb/release-scripts/pull/5
BTW I think we can close this PR now, even if we want to continue the discussion about the CLI here.
@lehrig how do you test it? By actually running on IBM Power or some sort of virtualization? I could dig into it but I don't have an access to the hardware
BTW I think we can close this PR now, even if we want to continue the discussion about the CLI here.
agreed.
@lehrig how do you test it? By actually running on IBM Power or some sort of virtualization? I could dig into it but I don't have an access to the hardware
I'm using actual IBM Power hardware.
If you need access, I think this is a good overview of options: https://community.ibm.com/community/user/powerdeveloper/blogs/linda-alkire-kinnunen/2022/08/08/accelerate-your-open-source-development-with-acces
The best option in this context might be https://openpower.ic.unicamp.br/minicloud/ where you can apply for free via https://docs.google.com/forms/d/e/1FAIpQLSexcyToQSV34i0aLOX82pYrV30JGYPUj86si1aNcmwtmqRbCw/viewform. Not sure, if that fits for you...