sqlite-jdbc Docker builds with uid/gid 0 (root)

Describe the bug When building native binaries via make native-all, output directories in /target have root user permissions (uid 0, gid 0) resulting in build error Permission denied

To Reproduce

From linux environment, run make native-all
Check directory permissions after build fails

Expected behavior building via dockcross should utilize the user id and group id from the shell of the user executing it such that their output matches that of the user.

e.g.

// Makefile

U_ID:="$(shell id -u)"
G_ID:="$(shell id -g)"

// For all other architectures as shown in attached image
linux64: $(SQLITE_UNPACKED) jni-header
	docker run $(DOCKER_RUN_OPTS) -u "$(U_ID):$(G_ID)" .......

Logs

mkdir: cannot create directory 'target/classes/org/sqlite/native/Linux-Musl/aarch64': Permission denied
make: *** [Makefile:125: src/main/resources/org/sqlite/native/Linux-Musl/aarch64/libsqlitejdbc.so] Error 1
make: *** [Makefile:162: linux-musl-arm64] Error 2

Environment (please complete the following information):

OS:

$ cat /etc/os-release
NAME="Pop!_OS"
VERSION="22.04 LTS"
ID=pop
ID_LIKE="ubuntu debian"
PRETTY_NAME="Pop!_OS 22.04 LTS"
VERSION_ID="22.04"
HOME_URL="https://pop.system76.com"
SUPPORT_URL="https://support.system76.com"
BUG_REPORT_URL="https://github.com/pop-os/pop/issues"
PRIVACY_POLICY_URL="https://system76.com/privacy"
VERSION_CODENAME=jammy
UBUNTU_CODENAME=jammy
LOGO=distributor-logo-pop-os

CPU architecture: x86_64
sqlite-jdbc version: 3.43.0.0

Additional context

Sep 10 '23 02:09 05nelsonm

If you already have a fix feel free to send a PR.

Sep 10 '23 04:09 gotson

freebsd32 and freebsd64 use docker image empterdose/freebsd-cross-build:9.3 and are unable to specify an unprivileged user via option -u $(U_ID):$(G_ID).

From the README it says bumping to 11.4 will allow the ability to specify a user. Thoughts on bumping it to 11.4, or would that be an issue?

These 2 builds are the only ones that are unhappy.

EDIT: empterdose docker images aren't really working b/c of the need to install bash, openjdk8 and perl, which can't be done when specifying a user (locked). Also getting errors env: can't execute bash: No such file or directory

Unsure what to do other than to build and use a different image.

Sep 10 '23 10:09 05nelsonm

@michael-o knows about freebsd much more than I do, and may advise on supporting only v11+.

It seems the empterdose image has been updated and supports aarch64 now, which would allow to drop my own fork.

Sep 10 '23 13:09 gotson

empterdose docker images aren't really working b/c of the need to install bash, openjdk8 and perl, which can't be done when specifying a user (locked). Also getting errors env: can't execute bash: No such file or directory

Those should be installable with sudo, even when running the container with an unprivileged user, no?

Sep 10 '23 13:09 gotson

@gotson Let me try to understand the issue and how I can help here.

Sep 10 '23 13:09 michael-o

First of all note:

https://github.com/MrDOS/freebsd-cross-build/issues/1
https://github.com/fusesource/jansi/issues/257

I'd first update and then retry and see results. 9.3 is very old now (2016-12-31).

Sep 10 '23 13:09 michael-o

@05nelsonm are you able to clarify why the original issue of having directories / files owned by root is a problem?

It's been like this for some time, and never has been a problem for anyone. Is that because your build system differs in some ways?

We even build all native libs on github actions without any issues.

Sep 10 '23 13:09 gotson

@05nelsonm are you able to clarify why the original issue of having directories / files owned by root is a problem?

It's been like this for some time, and never has been a problem for anyone. Is that because your build system differs in some ways?

Yes I am building on laptop from non-root user (user id 1000) and kept running into build errors related to permissions resulting in the inability to copy files >> Permission denied.

Have everything building now with the proper user id:group id, except the freebsd* binaries, and the build is successful.

We even build all native libs on github actions without any issues.

Yes the container for GH actions is fine b/c it's user id:group id is 0:0 (i.e. root)

In general, running docker as root is ill-advised.

First of all note:

Update to 12.4 with clang MrDOS/freebsd-cross-build#1

Upgrade FreeBSD cross image fusesource/jansi#257

I'd first update and then retry and see results. 9.3 is very old now (2016-12-31).

Looking into it now

Sep 10 '23 13:09 05nelsonm

Briefly looking at the issue and the output (don't attach screenshots, they are horrible) it does not specificly apply to FreeBSD, but on how the actual container is run. I still would try to upgrade to 11.4 first and see if the error exist. Doesn't make sense to analyse an old image's behavior, no?

Sep 10 '23 13:09 michael-o

Briefly looking at the issue and the output (don't attach screenshots, they are horrible) it does not specificly apply to FreeBSD, but on how the actual container is run. I still would try to upgrade to 11.4 first and see if the error exist. Doesn't make sense to analyse an old image's behavior, no?

Yes I've tried updating to new version and adding --user $(U_ID):$(G_ID), but freebsd complains because it is unable to install dependencies (non-root user). Will try some other images.

Sep 10 '23 13:09 05nelsonm

Briefly looking at the issue and the output (don't attach screenshots, they are horrible) it does not specificly apply to FreeBSD, but on how the actual container is run. I still would try to upgrade to 11.4 first and see if the error exist. Doesn't make sense to analyse an old image's behavior, no?

Yes I've tried updating to new version and adding --user $(U_ID):$(G_ID), but freebsd complains. Will try some other images.

What is the exact output? I bet that the source user does not exist in the container, how can it?

Sep 10 '23 13:09 michael-o

What is the exact output? I bet that the source user does not exist in the container, how can it?

The --user ... command is a docker thing, it will automatically run using that uid:gid. The issue is with alpine linux which the freebsd containers are based on, and the inability to execute apk add <lib>. So, it exits with error as it is unable to find bash or perl from that user.

Sep 10 '23 13:09 05nelsonm

What is the exact output? I bet that the source user does not exist in the container, how can it?

The --user ... command is a docker thing, it will automatically run using that uid:gid. The issue is with alpine linux which the freebsd containers are based on, and the inability to execute apk add <lib>. So, it exits with error as it is unable to find bash or perl from that user.

So it is not a suitable replacement for the old one at the moment or you need a layer on top with OpenJDK 8?

Sep 10 '23 13:09 michael-o

So it is not a suitable replacement for the old one at the moment or you need a layer on top with OpenJDK 8?

Here's the output from upgrading to 11.4

with uid:gid

docker run --rm -u 1000:1000 -v $PWD:/workdir empterdose/freebsd-cross-build:11.4 sh -c 'apk add bash; apk add openjdk8; apk add perl; make clean-native native OS_NAME=FreeBSD OS_ARCH=x86 CROSS_PREFIX=i386-freebsd11-'
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied
Makefile.common:4: Set JAVA_HOME environment variable
Will build using target: FreeBSD-x86 (detected os: FreeBSD arch: x86)
env: can't execute 'bash': No such file or directory
env: can't execute 'bash': No such file or directory
env: can't execute 'bash': No such file or directory
env: can't execute 'bash': No such file or directory
env: can't execute 'bash': No such file or directory
rm -rf target/sqlite-3.43.0-FreeBSD-x86
rm: can't remove 'target/sqlite-3.43.0-FreeBSD-x86/sqlite3ext.h': Permission denied
rm: can't remove 'target/sqlite-3.43.0-FreeBSD-x86/sqlite3.c': Permission denied
rm: can't remove 'target/sqlite-3.43.0-FreeBSD-x86/sqlite3.c.tmp': Permission denied
make: *** [Makefile:218: clean-native] Error 1
make: *** [Makefile:150: freebsd32] Error 2

with out uid:gid

make: i386-freebsd11-gcc: No such file or directory
make: *** [Makefile:72: target/sqlite-3.43.0-FreeBSD-x86/sqlite3.o] Error 127
i386-freebsd11-gcc -o target/sqlite-3.43.0-FreeBSD-x86/sqlite3.o -c -Itarget/sqlite-3.43.0-FreeBSD-x86 -Itarget/sqlite-amalgamation-3430000 -I/include -Ilib/inc_linux -Os -fPIC -fvisibility=hidden   \
    -DSQLITE_ENABLE_LOAD_EXTENSION=1 \
    -DSQLITE_HAVE_ISNAN \
    -DHAVE_USLEEP=1 \
    -DSQLITE_ENABLE_COLUMN_METADATA \
    -DSQLITE_CORE \
    -DSQLITE_ENABLE_FTS3 \
    -DSQLITE_ENABLE_FTS3_PARENTHESIS \
    -DSQLITE_ENABLE_FTS5 \
    -DSQLITE_ENABLE_RTREE \
    -DSQLITE_ENABLE_STAT4 \
    -DSQLITE_ENABLE_DBSTAT_VTAB \
    -DSQLITE_ENABLE_MATH_FUNCTIONS \
    -DSQLITE_THREADSAFE=1 \
    -DSQLITE_DEFAULT_MEMSTATUS=0 \
    -DSQLITE_DEFAULT_FILE_PERMISSIONS=0666 \
    -DSQLITE_MAX_VARIABLE_NUMBER=250000 \
    -DSQLITE_MAX_MMAP_SIZE=1099511627776 \
    -DSQLITE_MAX_LENGTH=2147483647 \
    -DSQLITE_MAX_COLUMN=32767 \
    -DSQLITE_MAX_SQL_LENGTH=1073741824 \
    -DSQLITE_MAX_FUNCTION_ARG=127 \
    -DSQLITE_MAX_ATTACHED=125 \
    -DSQLITE_MAX_PAGE_COUNT=4294967294 \
     \
    target/sqlite-3.43.0-FreeBSD-x86/sqlite3.c
make: *** [Makefile:150: freebsd32] Error 2

EDIT: Yes I think a layer on top with bash, openjdk8, perl installed would do the trick.

Sep 10 '23 13:09 05nelsonm

Let's start without uid first: https://github.com/xerial/sqlite-jdbc/blob/3eaabec2475ca0577f025b3606a992a99f1cd318/Makefile.common#L154C1-L156 You need to override the first two to the actual command names, then JAVA_HOME isn't set, so one needs the OpenJDK inside of the container.

Sep 10 '23 13:09 michael-o

I don't quite understand how root on your laptop could not have the permissions to write to your own user folder.

Sep 10 '23 13:09 gotson

I don't quite understand how root on your laptop could not have the permissions to write to your own user folder.

It's the base files for linux x86_64. Docker builds them and outputs with root user to /target, then from my shell when preparing the next build target, am unable to copy b/c no permissions.

Sep 10 '23 14:09 05nelsonm

Let's start without uid first: https://github.com/xerial/sqlite-jdbc/blob/3eaabec2475ca0577f025b3606a992a99f1cd318/Makefile.common#L154C1-L156 You need to override the first two to the actual command names, then JAVA_HOME isn't set, so one needs the OpenJDK inside of the container.

Not 100% certain on what those variables in Makefile.common need to be overridden to. Sorry, am a bit out of my depth when it comes to the native side of the house.

Sep 10 '23 14:09 05nelsonm

Installed a AL8 VM here and trying this myself. Needs a bit of time.

Sep 11 '23 07:09 michael-o

Installed a AL8 VM here and trying this myself. Needs a bit of time.

K.

My commit for adding a user to other builds is https://github.com/05nelsonm/sqlite-jdbc/commit/56b45e3fd01b1256336452e24a686fab7363f393

Sep 11 '23 09:09 05nelsonm

Got the OP semi-resolved on freebsd32 and freebsd64 via addgroup and adduser and then using sudo (because su - doesn't work due to home directory change).

It does not work though if the U_ID or G_ID are 0 (root) b/c it will fail to add the user/group.

freebsd32: $(SQLITE_UNPACKED) jni-header
	docker run $(DOCKER_RUN_OPTS) -v $$PWD:/workdir empterdose/freebsd-cross-build:9.3 sh -c 'addgroup --gid $(G_ID) UBUILD; adduser --ingroup UBUILD --uid $(U_ID) --disabled-password --no-create-home --gecos "" UBUILD; apk add sudo; apk add bash; apk add openjdk8; apk add perl; sudo -u UBUILD make clean-native native OS_NAME=FreeBSD OS_ARCH=x86 CROSS_PREFIX=i386-freebsd9-'

freebsd64: $(SQLITE_UNPACKED) jni-header
	docker run $(DOCKER_RUN_OPTS) -v $$PWD:/workdir empterdose/freebsd-cross-build:9.3 sh -c 'addgroup --gid $(G_ID) UBUILD; adduser --ingroup UBUILD --uid $(U_ID) --disabled-password --no-create-home --gecos "" UBUILD; apk add sudo; apk add bash; apk add openjdk8; apk add perl; sudo -u UBUILD make clean-native native OS_NAME=FreeBSD OS_ARCH=x86_64 CROSS_PREFIX=x86_64-freebsd9-'

Sep 11 '23 09:09 05nelsonm

@05nelsonm what's your local setup ?

I am myself running this locally on a mac, and i have no problem. My mac user is not root.

Sep 11 '23 09:09 gotson

@05nelsonm what's your local setup ?

I am myself running this locally on a mac, and i have no problem. My mac user is not root.

Yeah mac is a little different than linux.

$ cat /etc/os-release
NAME="Pop!_OS"
VERSION="22.04 LTS"
ID=pop
ID_LIKE="ubuntu debian"
PRETTY_NAME="Pop!_OS 22.04 LTS"
VERSION_ID="22.04"
HOME_URL="https://pop.system76.com"
SUPPORT_URL="https://support.system76.com"
BUG_REPORT_URL="https://github.com/pop-os/pop/issues"
PRIVACY_POLICY_URL="https://system76.com/privacy"
VERSION_CODENAME=jammy
UBUNTU_CODENAME=jammy
LOGO=distributor-logo-pop-os

$ id -un
administrator
$ id -u
1000
$ id -g
1000
$ cat /etc/group | grep "administrator"
sudo:x:27:administrator
administrator:x:1000:
libvirt:x:131:administrator
docker:x:998:administrator

Cloned the repository into my project's directory and ran make native-all which initially failed b/c output was permissioned for root (0:0).

Sep 11 '23 09:09 05nelsonm

So, with the exception of freebsd* target permissions, everything builds for me now via make native-all package test without error and produces a runnable .jar file.

Makefile modifications >> https://github.com/05nelsonm/sqlite-jdbc/commit/56b45e3fd01b1256336452e24a686fab7363f393

FreeBSD permissions

Would it be advisable to submit a PR and then handle the freebsd complications at a later date in time?

Sep 11 '23 13:09 05nelsonm

I cannot properly link with the 11.4 image because of:

[osipovmi@al8 sqlite-jdbc]$ docker run empterdose/freebsd-cross-build:11.4 which i386-freebsd11-strip

[osipovmi@al8 sqlite-jdbc]$
[osipovmi@al8 sqlite-jdbc]$ docker run empterdose/freebsd-cross-build:9.3 which i386-freebsd9-strip
/usr/local/bin/i386-freebsd9-strip

The prefixes strip command is missing.

Sep 11 '23 14:09 michael-o

See also: https://github.com/MrDOS/freebsd-cross-build/issues/1#issuecomment-1714001366

Sep 11 '23 14:09 michael-o

sqlite-jdbc sqlite-jdbc copied to clipboard

Docker builds with uid/gid 0 (root)

sqlite-jdbc
sqlite-jdbc copied to clipboard