nimble
nimble copied to clipboard
nim-lang
".nimble" File Detection broken on `linux/arm/v7` via QEMU on `amd64` Host
- Clone this project. https://github.com/theAkito/userdef
- Put this in
test.Dockerfile.FROM akito13/nim:ubuntu AS build COPY . /userdef WORKDIR /userdef - Build
arm64image.docker buildx build \ --force-rm \ --platform linux/arm64/v8 \ --tag "local/akito13/userdef:debug-arm64-libc" \ --file test.Dockerfile \ --load \ . - Build
armimage.docker buildx build \ --force-rm \ --platform linux/arm/v7 \ --tag "local/akito13/userdef:debug-arm-libc" \ --file test.Dockerfile \ --load \ . - Run the
arm64image &nimble installproject dependencies & notice, that it works.docker run -it --rm --user root -v "$PWD:/cwd" local/akito13/userdef:debug-arm64-libc /bin/bash nimble install --accept --depsOnly --verbose - Run the
armimage &nimble installproject dependencies & notice, that it does not work.docker run -it --rm --user root -v "$PWD:/cwd" local/akito13/userdef:debug-arm-libc /bin/bash nimble install --accept --depsOnly --verbose
Output from nimble install on arm.
root@8b7deace220a:/userdef# nimble install --accept --depsOnly --verbose
Warning: Nimble data file "/root/.nimble/nimbledata2.json" is not found.
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /userdef
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
root@8b7deace220a:/userdef# nimble install --accept --depsOnly --verbose
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been loaded.
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /userdef
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
root@8b7deace220a:/userdef#
Running chown root:root -R . works around the local problem, but then the dependency cannot be installed, due to the same issue.
root@8b7deace220a:/userdef# chown root:root -R .
root@8b7deace220a:/userdef# nimble install --accept --depsOnly --verbose
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been loaded.
Verifying dependencies for [email protected]
Reading official package list
Prompt: No local packages.json found, download it from internet? -> [forced yes]
Downloading Official package list
Trying https://raw.githubusercontent.com/nim-lang/packages/master/packages.json
Success Package list downloaded.
Checking for useradd@>= 0.4.0
Installing useradd@>= 0.4.0
Reading official package list
Downloading https://github.com/theAkito/nim-useradd using git
Cloning latest tagged version: 0.4.0
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /tmp/nimble_34/githubcom_theAkitonimuseradd_0.4.0
Warning: Couldn't remove Nimble's temp dir
Details: Directory not empty
... Additional info: /tmp/nimble_34
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
root@8b7deace220a:/userdef#
All this works without an issue on arm64. No errors, there.
This is a showstopper for arm builds.
This is really weird. I don't have a lot of Docker experience. Perhaps Docker is doing something weird? My project which has a docker container works fine according to the person who implemented it, but perhaps something's changed in one of the latest commits? Can you run ls and show me the output?
I don't have a lot of Docker experience.
No problem. I am a very advanced Docker user. 😁
My project which has a docker container works fine according to the person who implemented it, but perhaps something's changed in one of the latest commits?
Does this include the arm distribution? 🤔
Can you run
lsand show me the output?
Where exactly? What do you need to know? What are you looking for? 🙂
Ah, I guess I didn't word this correctly. 😅
I'm curious as to why Nimble can't find the file with the ARM target but it can with ARM64. Execute this program:
import std/os
writeFile("test", "hello!")
doAssert fileExists("test"), "File doesn't exist"
what does it output?
If the assertion fails and the file is indeed there, it's an upstream bug in the Nim std/os library with the ARM target.
@xTrayambak
# COMMAND 0
root@1c71f6139f2a:/userdef# nimble install --accept --depsOnly --verbose
Warning: Nimble data file "/root/.nimble/nimbledata2.json" is not found.
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /userdef
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
# COMMAND 1
root@1c71f6139f2a:/userdef# apt update ; apt install -y file nano
Get:1 http://ports.ubuntu.com/ubuntu-ports jammy InRelease [270 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports jammy-updates InRelease [119 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports jammy-backports InRelease [109 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports jammy-security InRelease [110 kB]
Get:5 http://ports.ubuntu.com/ubuntu-ports jammy/universe armhf Packages [16.8 MB]
Get:6 http://ports.ubuntu.com/ubuntu-ports jammy/restricted armhf Packages [13.3 kB]
Get:7 http://ports.ubuntu.com/ubuntu-ports jammy/multiverse armhf Packages [199 kB]
Get:8 http://ports.ubuntu.com/ubuntu-ports jammy/main armhf Packages [1701 kB]
Get:9 http://ports.ubuntu.com/ubuntu-ports jammy-updates/universe armhf Packages [938 kB]
Get:10 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main armhf Packages [863 kB]
Get:11 http://ports.ubuntu.com/ubuntu-ports jammy-updates/multiverse armhf Packages [3280 B]
Get:12 http://ports.ubuntu.com/ubuntu-ports jammy-updates/restricted armhf Packages [13.6 kB]
Get:13 http://ports.ubuntu.com/ubuntu-ports jammy-backports/universe armhf Packages [30.7 kB]
Get:14 http://ports.ubuntu.com/ubuntu-ports jammy-backports/main armhf Packages [77.8 kB]
Get:15 http://ports.ubuntu.com/ubuntu-ports jammy-security/restricted armhf Packages [12.6 kB]
Get:16 http://ports.ubuntu.com/ubuntu-ports jammy-security/main armhf Packages [604 kB]
Get:17 http://ports.ubuntu.com/ubuntu-ports jammy-security/universe armhf Packages [681 kB]
Get:18 http://ports.ubuntu.com/ubuntu-ports jammy-security/multiverse armhf Packages [592 B]
Fetched 22.5 MB in 9s (2551 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
4 packages can be upgraded. Run 'apt list --upgradable' to see them.
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
libmagic-mgc libmagic1
Suggested packages:
hunspell
The following NEW packages will be installed:
file libmagic-mgc libmagic1 nano
0 upgraded, 4 newly installed, 0 to remove and 4 not upgraded.
Need to get 626 kB of archives.
After this operation, 8347 kB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main armhf libmagic-mgc armhf 1:5.41-3ubuntu0.1 [257 kB]
Get:2 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main armhf libmagic1 armhf 1:5.41-3ubuntu0.1 [80.2 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports jammy-updates/main armhf file armhf 1:5.41-3ubuntu0.1 [20.6 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports jammy/main armhf nano armhf 6.2-1 [268 kB]
Fetched 626 kB in 0s (1622 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libmagic-mgc.
(Reading database ... 13091 files and directories currently installed.)
Preparing to unpack .../libmagic-mgc_1%3a5.41-3ubuntu0.1_armhf.deb ...
Unpacking libmagic-mgc (1:5.41-3ubuntu0.1) ...
Selecting previously unselected package libmagic1:armhf.
Preparing to unpack .../libmagic1_1%3a5.41-3ubuntu0.1_armhf.deb ...
Unpacking libmagic1:armhf (1:5.41-3ubuntu0.1) ...
Selecting previously unselected package file.
Preparing to unpack .../file_1%3a5.41-3ubuntu0.1_armhf.deb ...
Unpacking file (1:5.41-3ubuntu0.1) ...
Selecting previously unselected package nano.
Preparing to unpack .../archives/nano_6.2-1_armhf.deb ...
Unpacking nano (6.2-1) ...
Setting up libmagic-mgc (1:5.41-3ubuntu0.1) ...
Setting up libmagic1:armhf (1:5.41-3ubuntu0.1) ...
Setting up file (1:5.41-3ubuntu0.1) ...
Setting up nano (6.2-1) ...
update-alternatives: using /bin/nano to provide /usr/bin/editor (editor) in auto mode
update-alternatives: warning: skip creation of /usr/share/man/man1/editor.1.gz because associated file /usr/share/man/man1/nano.1.gz (of link group editor) doesn't exist
update-alternatives: using /bin/nano to provide /usr/bin/pico (pico) in auto mode
update-alternatives: warning: skip creation of /usr/share/man/man1/pico.1.gz because associated file /usr/share/man/man1/nano.1.gz (of link group pico) doesn't exist
Processing triggers for libc-bin (2.35-0ubuntu3.3) ...
# COMMAND 2
root@1c71f6139f2a:/userdef# ls
Dockerfile README.md docker-build-local.sh helpers libc.debug.Dockerfile src tests userdef.nimble
LICENSE debug.Dockerfile docker-run-local.sh libc.Dockerfile oil.nims tasks userdef userdef_debug
# COMMAND 3
root@1c71f6139f2a:/userdef# file userdef.nimble
userdef.nimble: ASCII text
# COMMAND 4
root@1c71f6139f2a:/userdef# nano t.nim
# COMMAND 5
root@1c71f6139f2a:/userdef# nim c -r t.nim
Hint: used config file '/etc/nim/nim.cfg' [Conf]
Hint: used config file '/etc/nim/config.nims' [Conf]
..................................................................................................................
CC: ../usr/lib/nim/system/exceptions.nim
CC: ../usr/lib/nim/std/private/digitsutils.nim
CC: ../usr/lib/nim/std/assertions.nim
CC: ../usr/lib/nim/system/dollars.nim
CC: ../usr/lib/nim/std/syncio.nim
CC: ../usr/lib/nim/system.nim
CC: ../usr/lib/nim/std/private/oscommon.nim
CC: ../usr/lib/nim/pure/times.nim
CC: ../usr/lib/nim/std/envvars.nim
CC: ../usr/lib/nim/std/cmdline.nim
CC: t.nim
Hint: [Link]
Hint: mm: orc; threads: on; opt: none (DEBUG BUILD, `-d:release` generates faster code)
52070 lines; 13.368s; 49.207MiB peakmem; proj: /userdef/t.nim; out: /userdef/t [SuccessX]
Hint: /userdef/t [Exec]
# COMMAND 6
root@1c71f6139f2a:/userdef# nimble install --accept --depsOnly --verbose
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been loaded.
Verifying dependencies for [email protected]
Reading official package list
Prompt: No local packages.json found, download it from internet? -> [forced yes]
Downloading Official package list
Trying https://raw.githubusercontent.com/nim-lang/packages/master/packages.json
Success Package list downloaded.
Checking for useradd@>= 0.4.0
Installing useradd@>= 0.4.0
Reading official package list
Downloading https://github.com/theAkito/nim-useradd using git
Cloning latest tagged version: 0.4.0
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /tmp/nimble_906/githubcom_theAkitonimuseradd_0.4.0
Warning: Couldn't remove Nimble's temp dir
Details: Directory not empty
... Additional info: /tmp/nimble_906
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
# COMMAND 7
root@1c71f6139f2a:/userdef# ls -alh
total 3.3M
drwxr-xr-x 1 root root 4.0K Oct 25 11:26 .
drwxr-xr-x 1 root root 4.0K Oct 25 11:24 ..
drwxr-xr-x 9 root root 4.0K Oct 25 11:24 .git
-rw-r--r-- 1 root root 248 Apr 1 2022 .gitattributes
drwxr-xr-x 3 root root 4.0K Oct 14 15:19 .github
-rw-r--r-- 1 root root 322 Apr 7 2022 .gitignore
-rw-r--r-- 1 root root 85 Apr 8 2022 .gitmodules
drwxr-xr-x 6 root root 20K Oct 25 11:24 .history
drwxr-xr-x 3 root root 4.0K Apr 8 2022 .vscode
-rw-r--r-- 1 root root 1.1K Oct 14 22:56 Dockerfile
-rw-r--r-- 1 root root 35K Mar 29 2022 LICENSE
-rw-r--r-- 1 root root 8.9K Apr 1 2023 README.md
-rw-r--r-- 1 root root 1.1K Oct 14 22:25 debug.Dockerfile
-rw-r--r-- 1 root root 1.4K Oct 25 11:20 docker-build-local.sh
-rw-r--r-- 1 root root 1.3K Oct 14 15:05 docker-run-local.sh
drwxr-xr-x 2 root root 4.0K Apr 7 2022 helpers
-rw-r--r-- 1 root root 1.2K Oct 14 22:46 libc.Dockerfile
-rw-r--r-- 1 root root 1.2K Oct 25 11:24 libc.debug.Dockerfile
-rw-r--r-- 1 root root 362 Apr 7 2022 oil.nims
drwxr-xr-x 3 root root 4.0K Mar 29 2022 src
-rwxr-xr-x 1 root root 85K Oct 25 11:26 t
-rw-r--r-- 1 root root 94 Oct 25 11:26 t.nim
drwxr-xr-x 2 root root 4.0K Apr 1 2023 tasks
-rw-r--r-- 1 root root 6 Oct 25 11:26 test
drwxr-xr-x 2 root root 4.0K Apr 1 2023 tests
-rwxr-xr-x 1 root root 144K Oct 14 14:33 userdef
-rw-r--r-- 1 root root 3.0K Oct 14 22:56 userdef.nimble
-rwxr-xr-x 1 root root 2.9M Oct 14 14:33 userdef_debug
root@1c71f6139f2a:/userdef#
The Nim program with the assertion works.
Notice, how it first does not find the local Nimble file. Then, it does not find the external one.
This likely isn't related to file detection per sé, but to some permission madness with Nim on ARM...
@xTrayambak
# Files in folder...
root@8d48ddcbd85c:/userdef# ls -alh
total 3.2M
drwxr-xr-x 10 root root 4.0K Oct 25 11:46 .
drwxr-xr-x 1 root root 4.0K Oct 25 11:46 ..
drwxr-xr-x 9 root root 4.0K Oct 25 11:46 .git
-rw-r--r-- 1 root root 248 Apr 1 2022 .gitattributes
drwxr-xr-x 3 root root 4.0K Oct 14 15:19 .github
-rw-r--r-- 1 root root 322 Apr 7 2022 .gitignore
-rw-r--r-- 1 root root 85 Apr 8 2022 .gitmodules
drwxr-xr-x 6 root root 20K Oct 25 11:46 .history
drwxr-xr-x 3 root root 4.0K Apr 8 2022 .vscode
-rw-r--r-- 1 root root 1.1K Oct 14 22:56 Dockerfile
-rw-r--r-- 1 root root 35K Mar 29 2022 LICENSE
-rw-r--r-- 1 root root 8.9K Apr 1 2023 README.md
-rw-r--r-- 1 root root 1.1K Oct 14 22:25 debug.Dockerfile
-rw-r--r-- 1 root root 1.4K Oct 25 11:45 docker-build-local.sh
-rw-r--r-- 1 root root 1.3K Oct 14 15:05 docker-run-local.sh
drwxr-xr-x 2 root root 4.0K Apr 7 2022 helpers
-rw-r--r-- 1 root root 1.2K Oct 14 22:46 libc.Dockerfile
-rw-r--r-- 1 root root 1.2K Oct 25 11:24 libc.debug.Dockerfile
-rw-r--r-- 1 root root 362 Apr 7 2022 oil.nims
drwxr-xr-x 3 root root 4.0K Mar 29 2022 src
drwxr-xr-x 2 root root 4.0K Apr 1 2023 tasks
drwxr-xr-x 2 root root 4.0K Apr 1 2023 tests
-rwxr-xr-x 1 root root 144K Oct 14 14:33 userdef
-rw-r--r-- 1 root root 3.0K Oct 14 22:56 userdef.nimble
-rwxr-xr-x 1 root root 2.9M Oct 14 14:33 userdef_debug
# Spamming `nimble install` to no avail...
root@8d48ddcbd85c:/userdef# nimble install --accept --depsOnly --verbose
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been loaded.
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /userdef
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
root@8d48ddcbd85c:/userdef# nimble install --accept --depsOnly --verbose
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been loaded.
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /userdef
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
root@8d48ddcbd85c:/userdef# nimble install --accept --depsOnly --verbose
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been loaded.
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /userdef
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
root@8d48ddcbd85c:/userdef# nimble install --accept --depsOnly --verbose
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been loaded.
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /userdef
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
# Writing one random file into folder...
root@8d48ddcbd85c:/userdef# cat > t.nim << EOF
import std/os
writeFile("test", "hello!")
doAssert fileExists("test"), "File doesn't exist"
EOF
# Nimble file is now found.
# But now the external one is not found.
root@8d48ddcbd85c:/userdef# nimble install --accept --depsOnly --verbose
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been loaded.
Verifying dependencies for [email protected]
Reading official package list
Prompt: No local packages.json found, download it from internet? -> [forced yes]
Downloading Official package list
Trying https://raw.githubusercontent.com/nim-lang/packages/master/packages.json
Success Package list downloaded.
Checking for useradd@>= 0.4.0
Installing useradd@>= 0.4.0
Reading official package list
Downloading https://github.com/theAkito/nim-useradd using git
Cloning latest tagged version: 0.4.0
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /tmp/nimble_61/githubcom_theAkitonimuseradd_0.4.0
Warning: Couldn't remove Nimble's temp dir
Details: Directory not empty
... Additional info: /tmp/nimble_61
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
root@8d48ddcbd85c:/userdef# nimble install --accept --depsOnly --verbose
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been loaded.
Verifying dependencies for [email protected]
Reading official package list
Checking for useradd@>= 0.4.0
Installing useradd@>= 0.4.0
Reading official package list
Downloading https://github.com/theAkito/nim-useradd using git
Cloning latest tagged version: 0.4.0
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /tmp/nimble_287/githubcom_theAkitonimuseradd_0.4.0
Warning: Couldn't remove Nimble's temp dir
Details: Directory not empty
... Additional info: /tmp/nimble_287
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
root@8d48ddcbd85c:/userdef# nimble install --accept --depsOnly --verbose
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been loaded.
Verifying dependencies for [email protected]
Reading official package list
Checking for useradd@>= 0.4.0
Installing useradd@>= 0.4.0
Reading official package list
Downloading https://github.com/theAkito/nim-useradd using git
Cloning latest tagged version: 0.4.0
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /tmp/nimble_507/githubcom_theAkitonimuseradd_0.4.0
Warning: Couldn't remove Nimble's temp dir
Details: Directory not empty
... Additional info: /tmp/nimble_507
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
root@8d48ddcbd85c:/userdef#
Nimble generally doesn't like being run under root. That being said, it shouldn't do what you're seeing either.
Nimble generally doesn't like being run under root. That being said, it shouldn't do what you're seeing either.
Well, for other developers, which I have seen sometimes, it's usually easy to blame the environment, which is why I specifically have proof, that precisely the same setup works on arm64, as well as amd64 etc., but not on arm. So, considering this inconsistency, there must be something wrong with Nim or Nimble or both. If there is an issue like this, it must be at least consistent, to argue against the environment. 😄
Yeah, sorry if that came off as an excuse. I didn't mean for it to seem like one. I believe this is an issue with os.fileExists on ARM builds. I'll check a bit more and see if you should file a bug report on the Nim GitHub repo
I just checked the code for that function and there is no way it's doing something wrong. Can Nimble/Nim run on your host OS? If it can, does it still do the same behaviour there as well?
Yeah, sorry if that came off as an excuse. I didn't mean for it to seem like one.
No, not at all, it did not come off as one. I actually was saying the opposite. Usually, I see others blaming the environment, but you do not do that. So, thanks! 😄
I believe this is an issue with
os.fileExistson ARM builds. I'll check a bit more and see if you should file a bug report on the Nim GitHub repo
I am not sure about that. It starts to "work" when I add a file to the folder. How are those two actions even related? I thought the whole time, it's a permission thing, but now I think, it's some low-level freaky stuff, where the "untouched" directory is accesible, but as soon as you change one single bit in the directory, it suddenly gets "available".
I am sure, if I would have a possibility to find out what random folder inside /tmp Nimble chooses to write its dependency, I could make it "work", by touching a file into it & the Nimble file would be found!
Can Nimble/Nim run on your host OS? If it can, does it still do the same behaviour there as well?
nimble install --accept --depsOnly --verbose runs fine on my host, but it's amd64 & files are "touched" all the time there, because it's a Git repository, etc.
Do you have read permissions in the directory? @theAkito
I initialised a fresh container from a freshly built image & the directory in it looks like this.
root@52007a98d333:/userdef# ls -alh
total 3.2M
drwxr-xr-x 10 root root 4.0K Oct 26 09:51 .
drwxr-xr-x 1 root root 4.0K Oct 26 09:51 ..
drwxr-xr-x 9 root root 4.0K Oct 26 09:51 .git
-rw-r--r-- 1 root root 248 Apr 1 2022 .gitattributes
drwxr-xr-x 3 root root 4.0K Oct 14 15:19 .github
-rw-r--r-- 1 root root 322 Apr 7 2022 .gitignore
-rw-r--r-- 1 root root 85 Apr 8 2022 .gitmodules
drwxr-xr-x 6 root root 20K Oct 26 09:51 .history
drwxr-xr-x 3 root root 4.0K Apr 8 2022 .vscode
-rw-r--r-- 1 root root 1.1K Oct 14 22:56 Dockerfile
-rw-r--r-- 1 root root 35K Mar 29 2022 LICENSE
-rw-r--r-- 1 root root 8.9K Apr 1 2023 README.md
-rw-r--r-- 1 root root 1.1K Oct 14 22:25 debug.Dockerfile
-rw-r--r-- 1 root root 1.4K Oct 26 09:50 docker-build-local.sh
-rw-r--r-- 1 root root 1.3K Oct 14 15:05 docker-run-local.sh
drwxr-xr-x 2 root root 4.0K Apr 7 2022 helpers
-rw-r--r-- 1 root root 1.2K Oct 14 22:46 libc.Dockerfile
-rw-r--r-- 1 root root 1.2K Oct 25 11:24 libc.debug.Dockerfile
-rw-r--r-- 1 root root 362 Apr 7 2022 oil.nims
drwxr-xr-x 3 root root 4.0K Mar 29 2022 src
drwxr-xr-x 2 root root 4.0K Apr 1 2023 tasks
drwxr-xr-x 2 root root 4.0K Apr 1 2023 tests
-rwxr-xr-x 1 root root 144K Oct 14 14:33 userdef
-rw-r--r-- 1 root root 3.0K Oct 14 22:56 userdef.nimble
-rwxr-xr-x 1 root root 2.9M Oct 14 14:33 userdef_debug
root@52007a98d333:/userdef#
However, I'm running as root, anyway, so it shouldn't really matter.
As an additional FYI, I can tell you, that I have quite a couple of Nim projects inside Docker images & containers & all build fine. This is however, the first project, which tries to run Nim 2.0.0 on arm.
Here, is an example of a project building fine on the official Nimage with Nim 1.6.14 & arm.
https://github.com/theAkito/zoominvitr
Ah, I meant if the non-root account had read access to the file. This doesn't seem to be an issue with Nim, probably Nimble doing something weird. I'll check that out once I get home.
Hey. I've thoroughly checked Nimble and there is no way that it's either Nimble or Nim's fault. This is most likely an issue upstream with the POSIX library. Perhaps, Docker is doing something silly as well?
Hey. I've thoroughly checked Nimble and there is no way that it's either Nimble or Nim's fault. This is most likely an issue upstream with the POSIX library. Perhaps, Docker is doing something silly as well?
I see. I saw this is a possibility, as well.
What API would I need to call to be able to reproduce it in, for example, C? Just something with listing the contents of a directory? I'm not that familiar with the POSIX API.
If I'm gonna open an issue on their side with Nim code, there's no way they will take it as an error with Docker or POSIX or whatever. They will say, it's because of Nim....
Nim calls stat() on POSIX systems. Perhaps, try that?
stt.c
#include <sys/stat.h>
#include <errno.h>
int main (){
char *file = "userdef/userdef.nimble";
long size;
struct stat *buf = malloc(sizeof(struct stat));
errno = 0;
if(stat(file, buf) == 0)
{
size = buf->st_size;
printf("Size of \"%s\" is %lld bytes.\n", file, size);
}
else
{
perror(file); //if stat fails, print a diagnostic.
}
}
Procedure
docker buildx build \
--force-rm \
--platform linux/arm/v7 \
--tag "local/akito13/userdef:debug-arm-libc" \
--file test.Dockerfile \
--load \
.
docker run -it --rm --user root -v "$PWD:/cwd" local/akito13/userdef:debug-arm-libc /bin/bash
apt update
apt install -y nano
nano stt.c
# Paste content of `stt.c` & save.
gcc stt.c -o stt
chown 777 stt
./stt
# Output ---> `Size of "userdef/userdef.nimble" is 4611693629109439415 bytes.`
cd userdef
nimble install --accept --depsOnly --verbose
# Output...
root@89265fe314c9:/userdef# nimble install --accept --depsOnly --verbose
Warning: Nimble data file "/root/.nimble/nimbledata2.json" is not found.
packageinfo.nim(274) findNimbleFile
Error: Could not find a file with a .nimble extension inside the specified directory: /userdef
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
root@89265fe314c9:/userdef#
That's so weird. I'll have to look further, but as far as I know, there shouldn't be any issues. This is really peculiar.
Well, well, well....
So, I set up an entirely new Raspberry Pi with an armv7l from scratch, just to test this. Took a very long while due to installation issues, but it was worth it.
Here is the result...
github-issue-nimble-1151.sh
#!/bin/bash
## https://github.com/nim-lang/nimble/issues/1151
## Requires installed Docker.
echo "Running on..."
uname -om
echo
echo "Cloning source code..."
# mkdir src || true
# cd src || exit 1
git clone https://github.com/theAkito/userdef.git
echo
echo "Running Nim via Docker on this source code..."
cd userdef || exit 1
docker run -itv "$PWD:/cwd" --rm akito13/nim:2.0.0 bash -c "cd /cwd ; nimble install --accept --depsOnly --verbose"
echo
echo "Finished!"
Output
root@test:~# bash !:1
bash github-issue-nimble-1151.sh
Running on...
armv7l GNU/Linux
Cloning source code...
mkdir: cannot create directory ‘src’: File exists
Cloning into 'userdef'...
remote: Enumerating objects: 409, done.
remote: Counting objects: 100% (54/54), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 409 (delta 26), reused 37 (delta 16), pack-reused 355
Receiving objects: 100% (409/409), 79.90 KiB | 1.74 MiB/s, done.
Resolving deltas: 100% (228/228), done.
Running Nim via Docker on this source code...
Unable to find image 'akito13/nim:2.0.0' locally
2.0.0: Pulling from akito13/nim
1adfa548dcf4: Pull complete
a6cc3d089be1: Pull complete
b56c05f622e7: Pull complete
247a12f9b469: Pull complete
d7d2ce8ff264: Pull complete
Digest: sha256:3816cd9064ff2e543d4c33af78511dd28ff0596b8e6b0bab969a79678113c90c
Status: Downloaded newer image for akito13/nim:2.0.0
Warning: Nimble data file "/root/.nimble/nimbledata2.json" is not found.
Verifying dependencies for [email protected]
Reading official package list
Prompt: No local packages.json found, download it from internet? -> [forced yes]
Downloading Official package list
Trying https://raw.githubusercontent.com/nim-lang/packages/master/packages.json
Success Package list downloaded.
Checking for useradd@>= 0.3.0
Installing useradd@>= 0.3.0
Reading official package list
Downloading https://github.com/theAkito/nim-useradd using git
Cloning latest tagged version: 0.4.0
Verifying dependencies for [email protected]
Installing [email protected]
Copying file /tmp/nimble_1/githubcom_theAkitonimuseradd_0.3.0/useradd.nim to /root/.nimble/pkgs2/useradd-0.4.0-4c012e30fd76f54b0c7ce27969f0f08232d4046a/useradd.nim
Copying file /tmp/nimble_1/githubcom_theAkitonimuseradd_0.3.0/useradd/utils.nim to /root/.nimble/pkgs2/useradd-0.4.0-4c012e30fd76f54b0c7ce27969f0f08232d4046a/useradd/utils.nim
Copying file /tmp/nimble_1/githubcom_theAkitonimuseradd_0.3.0/useradd/crypt.nim to /root/.nimble/pkgs2/useradd-0.4.0-4c012e30fd76f54b0c7ce27969f0f08232d4046a/useradd/crypt.nim
Copying file /tmp/nimble_1/githubcom_theAkitonimuseradd_0.3.0/useradd/meta.nim to /root/.nimble/pkgs2/useradd-0.4.0-4c012e30fd76f54b0c7ce27969f0f08232d4046a/useradd/meta.nim
Copying file /tmp/nimble_1/githubcom_theAkitonimuseradd_0.3.0/useradd.nimble to /root/.nimble/pkgs2/useradd-0.4.0-4c012e30fd76f54b0c7ce27969f0f08232d4046a/useradd.nimble
Success: useradd installed successfully.
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
Finished!
root@test:~#
I guess, this must be a bug with Nim on QEMU, then....
Is it possible, that Nim does some fishy stuff in a QEMU environment or do you think, this is solely QEMU's fault, @xTrayambak?
I will check out, where I would open a bug report on QEMU about this issue.
Afaik, Nim has nothing in place for VMs. It only has stuff for architectures, not VMs.
Afaik, Nim has nothing in place for VMs. It only has stuff for architectures, not VMs.
I meant, maybe there are instructions that specifically do not work on certain virtual environments. So, if you do not account for those environments, you probably wouldn't catch the error, because it would always work on your system. However, that again sounds very unlikely, considering this only happens on ARM.
Tested with precisely the same test, as before using the Raspberry Pi, just to make sure. Still, same result.
root@d4215a06fa30:/userdef# nimble install --accept --depsOnly --verbose
Warning: Nimble data file "/root/.nimble/nimbledata2.json" is not found.
Verifying dependencies for [email protected]
Reading official package list
Prompt: No local packages.json found, download it from internet? -> [forced yes]
Downloading Official package list
Trying https://raw.githubusercontent.com/nim-lang/packages/master/packages.json
Success Package list downloaded.
Checking for useradd@>= 0.3.0
Installing useradd@>= 0.3.0
Reading official package list
Downloading https://github.com/theAkito/nim-useradd using git
Cloning latest tagged version: 0.4.0
Verifying dependencies for [email protected]
Installing [email protected]
Copying file /tmp/nimble_13/githubcom_theAkitonimuseradd_0.3.0/useradd.nim to /root/.nimble/pkgs2/useradd-0.4.0-4c012e30fd76f54b0c7ce27969f0f08232d4046a/useradd.nim
Copying file /tmp/nimble_13/githubcom_theAkitonimuseradd_0.3.0/useradd/utils.nim to /root/.nimble/pkgs2/useradd-0.4.0-4c012e30fd76f54b0c7ce27969f0f08232d4046a/useradd/utils.nim
Copying file /tmp/nimble_13/githubcom_theAkitonimuseradd_0.3.0/useradd/crypt.nim to /root/.nimble/pkgs2/useradd-0.4.0-4c012e30fd76f54b0c7ce27969f0f08232d4046a/useradd/crypt.nim
Copying file /tmp/nimble_13/githubcom_theAkitonimuseradd_0.3.0/useradd/meta.nim to /root/.nimble/pkgs2/useradd-0.4.0-4c012e30fd76f54b0c7ce27969f0f08232d4046a/useradd/meta.nim
Copying file /tmp/nimble_13/githubcom_theAkitonimuseradd_0.3.0/useradd.nimble to /root/.nimble/pkgs2/useradd-0.4.0-4c012e30fd76f54b0c7ce27969f0f08232d4046a/useradd.nimble
Success: useradd installed successfully.
Info: Nimble data file "/root/.nimble/nimbledata2.json" has been saved.
root@d4215a06fa30:/userdef#
So, I guess this must be a QEMU issue, then. Though, then I would need to do all this in a plain QEMU environment first, to make sure it happens there, as well. Otherwise, it still might be connected with Docker...
Still plenty of work ahead!
@xTrayambak Thanks for your help! Sorry, for making you put this much effort & time into this.
It seems like, there is nothing wrong with Nimble in this case, although it all still smells fishy, because this error started happening with Nim 2.0.0 ....
So, am not sure, if there is some new feature or whatever instruction, which did change to such behaviour.
For now, I will pursue this on the QEMU end & see what happens then.
Since this issue will not be solved, until it is fixed, wherever it ought to be fixed in, I think it's best to keep this issue open, because the original description is still true. Nimble does not fully work on ARM, from the perspective of a Nimble user.
Probably should be tested on most recent Nim version, however I certainly won't be able to do that within the next two weeks. Perhaps, after that - I'll note that.