dataall
dataall copied to clipboard
data-dot-all
Having an issue with message 'aws-dataall-graphql-1 exited with code 127' when trying to deploy locally
P.S. Don't attach files. Please, prefer add code snippets directly in the message body.
Hi,
I'm running into another issue when trying to deploy locally.
The build seems to work but I am getting the following messages:
aws-dataall-graphql-1 exited with code 127
and
: No such file or directory | /usr/bin/env: bash
I tried googling the issue but don't seem to be getting anywhere.
Could it be due to trying to deploy on Windows?
The StackOverflow question here seems to suggest something similar.
It leads to the following errors when opening the page on http://localhost:8080/:
I've added a more detailed log below.
Edit to add: I've experienced the same problem on a separate WIndows 10 machine with a fresh clone.
If you'd like any other info let me know. Any help greatly appreciated :) Thanks.
[+] Running 6/6
✔ Network aws-dataall_default Created 0.2s
✔ Container aws-dataall-frontend-1 Created 0.7s
✔ Container aws-dataall-db-1 Created 0.7s
✔ Container aws-dataall-elasticsearch-1 Created 0.7s
✔ Container aws-dataall-cdkproxy-1 Created 0.2s
✔ Container aws-dataall-graphql-1 Created 0.1s
Attaching to aws-dataall-cdkproxy-1, aws-dataall-db-1, aws-dataall-elasticsearch-1, aws-dataall-frontend-1, aws-dataall-graphql-1
aws-dataall-db-1 | The files belonging to this database system will be owned by user "postgres".
aws-dataall-db-1 | This user must also own the server process.
aws-dataall-db-1 |
aws-dataall-db-1 | The database cluster will be initialized with locale "en_US.utf8".
aws-dataall-db-1 | The default database encoding has accordingly been set to "UTF8".
aws-dataall-db-1 | The default text search configuration will be set to "english".
aws-dataall-db-1 |
aws-dataall-db-1 | Data page checksums are disabled.
aws-dataall-db-1 |
aws-dataall-db-1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
aws-dataall-db-1 | creating subdirectories ... ok
aws-dataall-db-1 | selecting default max_connections ... 100
aws-dataall-db-1 | selecting default shared_buffers ... 128MB
aws-dataall-db-1 | selecting default timezone ... Etc/UTC
aws-dataall-db-1 | selecting dynamic shared memory implementation ... posix
aws-dataall-db-1 | creating configuration files ... ok
aws-dataall-db-1 | running bootstrap script ... ok
: No such file or directory | /usr/bin/env: bash
aws-dataall-db-1 | performing post-bootstrap initialization ... ok
aws-dataall-graphql-1 exited with code 0
aws-dataall-db-1 | syncing data to disk ... ok
aws-dataall-db-1 |
aws-dataall-db-1 | WARNING: enabling "trust" authentication for local connections
aws-dataall-db-1 | You can change this by editing pg_hba.conf or using the option -A, or
aws-dataall-db-1 | --auth-local and --auth-host, the next time you run initdb.
aws-dataall-db-1 |
aws-dataall-db-1 | Success. You can now start the database server using:
aws-dataall-db-1 |
aws-dataall-db-1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
aws-dataall-db-1 |
aws-dataall-db-1 | waiting for server to start....2023-09-14 21:09:45.023 UTC [47] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
aws-dataall-db-1 | 2023-09-14 21:09:45.199 UTC [48] LOG: database system was shut down at 2023-09-14 21:09:43 UTC
aws-dataall-db-1 | 2023-09-14 21:09:45.211 UTC [47] LOG: database system is ready to accept connections
aws-dataall-db-1 | done
aws-dataall-db-1 | server started
aws-dataall-elasticsearch-1 | Created elasticsearch keystore in /usr/share/elasticsearch/config/elasticsearch.keystore
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
aws-dataall-graphql-1 exited with code 127
aws-dataall-db-1 | CREATE DATABASE
aws-dataall-db-1 |
aws-dataall-db-1 |
aws-dataall-db-1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
aws-dataall-db-1 |
aws-dataall-db-1 | 2023-09-14 21:09:46.167 UTC [47] LOG: received fast shutdown request
aws-dataall-db-1 | waiting for server to shut down....2023-09-14 21:09:46.173 UTC [47] LOG: aborting any active transactions
aws-dataall-db-1 | 2023-09-14 21:09:46.177 UTC [47] LOG: worker process: logical replication launcher (PID 54) exited with exit code 1
aws-dataall-db-1 | 2023-09-14 21:09:46.178 UTC [49] LOG: shutting down
aws-dataall-db-1 | 2023-09-14 21:09:46.317 UTC [47] LOG: database system is shut down
aws-dataall-db-1 | done
aws-dataall-db-1 | server stopped
aws-dataall-db-1 |
aws-dataall-db-1 | PostgreSQL init process complete; ready for start up.
aws-dataall-db-1 |
aws-dataall-db-1 | 2023-09-14 21:09:46.550 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
aws-dataall-db-1 | 2023-09-14 21:09:46.550 UTC [1] LOG: listening on IPv6 address "::", port 5432
aws-dataall-db-1 | 2023-09-14 21:09:46.567 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
aws-dataall-db-1 | 2023-09-14 21:09:46.634 UTC [75] LOG: database system was shut down at 2023-09-14 21:09:46 UTC
aws-dataall-db-1 | 2023-09-14 21:09:46.648 UTC [1] LOG: database system is ready to accept connections
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
aws-dataall-graphql-1 exited with code 127
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
aws-dataall-graphql-1 exited with code 127
aws-dataall-cdkproxy-1 | INFO: Will watch for changes in these directories: ['/code']
aws-dataall-cdkproxy-1 | INFO: Uvicorn running on http://0.0.0.0:2805 (Press CTRL+C to quit)
aws-dataall-cdkproxy-1 | INFO: Started reloader process [277] using statreload
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
aws-dataall-graphql-1 exited with code 127
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
aws-dataall-graphql-1 exited with code 127
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
aws-dataall-graphql-1 exited with code 127
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
: No such file or directory | /usr/bin/env: bash
aws-dataall-graphql-1 exited with code 127
aws-dataall-elasticsearch-1 | {"type": "server", "timestamp": "2023-09-14T21:10:02,990Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-clus
ter", "node.name": "fb4ae5189ae2", "message": "version[7.9.3], pid[7], build[default/docker/c4138e51121ef06a6404866cddc601906fe5c868/2020-10-16T10:36:16.141335Z],
OS[Linux/5.15.90.1-microsoft-standard-WSL2/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/15/15+36-1562]" }
aws-dataall-elasticsearch-1 | {"type": "server", "timestamp": "2023-09-14T21:10:02,999Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-clus
ter", "node.name": "fb4ae5189ae2", "message": "JVM home [/usr/share/elasticsearch/jdk]" }
aws-dataall-elasticsearch-1 | {"type": "server", "timestamp": "2023-09-14T21:10:02,999Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-clus
ter", "node.name": "fb4ae5189ae2", "message": "JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+Alway
sPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio
.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHoo
kEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Xms1g, -Xmx1g, -XX:+UseG1GC, -XX:G1ReservePercent=25, -XX:InitiatingHeapOccupancyPe
rcent=30, -Djava.io.tmpdir=/tmp/elasticsearch-5533190046993222359, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xl
og:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Des.cgroups.hierarchy.override=/, -Xmx256m, -Xms256m, -XX:MaxDirectMemo
rySize=134217728, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type
=docker, -Des.bundled_jdk=true]" }
Hi @Jamesgo1 we saw this issue before for Windows users reported in #502. In our team we do not use Windows, but let me try to find users that run on Windows
Hi @Jamesgo1, I was able to reach out to some colleagues that have documented their issues with Windows-docker-data.all All credits to @marjet26 and his team. I read through it but it does not seem like they faced your issue. In any case I posted it here in case it might be helpful for other users.
I am going to try to re-create your issue, what version of the code are you using?
Data.all on Windows
If you want to start the data.all application locally on a Windows machine, the basic instructions here are still valid: https://awslabs.github.io/aws-dataall/deploy-locally/
- You need to install Docker Desktop for Windows, if you do not have done that already. You might face several problems that you would not see on Linux or Mac:
- If Docker Desktop does not start with an out of memory error, ensure to increase the RAM docker uses in the settings as much as possible. I used 4,5 GB and starting data.all worked.
- If you get “Unable to locate credentials” when trying to link environments in data.all, you might face a series of problems outlined below.
- Although Windows knows the $HOME variable reference in the docker-compose.yaml of data all, it is still not resolved properly. You might see the error time="2023-04-20T09:31:02+02:00" level=warning msg="The "HOME" variable is not set. Defaulting to a blank string.".
- Manual workaround: In this case, the first workaround that did it for me is replacing $HOME in the docker-compose.yaml with the absolute path to your home folder (use forward slashes instead of backslashes). Of course this is not a great solution and you have to be careful to not push these changes.
- Workaround example:
volumes:
- ./backend:/code
- "C:/Users/username/.aws/credentials:/root/.aws/credentials:ro"
- Workaround without modifying docker-compose.yaml: There is a much better workaround for Windows users: Create a new file named .env in the data.all repository root folder, the same directly the docker-compose.yaml file resides in. Be sure the file is git-ignored (which it should be by default), NEVER commit that file to the repository. Set your home variable in .env , for example:
HOME=C:/Users/username
Now the application should startup without issues with an unmodified docker-compose.yaml.
- Note: If docker does not find the path on the host (as in the case above when using $HOME), it mounts a folder named “credentials” as empty directory into the containers, and you get the error “Unable to locate credentials” in data.all.
- You get the following error Error response from daemon: user declined directory sharing. Then on Windows, you have to explicitly configure file resource sharing to allow docker to mount the folders from the host into containers:
- Note: Also tried Ubuntu in WSL on Windows, however, this poses yet a different set of problems.
Hi @dlpzx.
Thanks for the info.
The last commit I have is 3680586 on the main branch I forked.
And the version in the docker-compose file is 3.2
Those instructions were useful as I was getting the error described when first building:
warning msg="The "HOME" variable is not set. Defaulting to a blank string.".
And adding that has gone when I added the .env file and home path.
However, I am still getting the same error messages even when hard coding the home path as described above.
Hi @Jamesgo1, sorry for the delay.
What is the exact error you are getting now?
@Jamesgo1 I had the same problem before, for me the issue was in the line ending between CRLF and LF, change those two files 'backend/docker/dev/Dockerfile' and 'backend/docker/dev/wait-for-it.sh' from CRLF to LF and rerun the docker-compose build --no-cache to force the recreation, but you can overall manage the CRLF and LF in your git configuration,
Hey @AmineIzanami thanks a lot for the info. I've not had a chance to look at this for a while but it'll be really helpful when I do.
Hi @Jamesgo1, was the help provided by @AmineIzanami helpful? If the issue is no longer needed we will soon close it. Let us know if you need any help
Hi, Since the issue has been inactive, we will close it soon, please comment if you still need any help
Closing this issue due to inactivity, feel free to reopen if you have questions or still need any help on this issue.