HaxeFoundation
Trouble running integration tests locally
Not really sure what could have changed, but the tests stopped working for me (even at older commits that worked previously).
Firstly, there is this warning at the beginning:
Warning: Buildkit version (v0.6.20) is different from Earthly version (0.6.20)
Secondly, the local website isn't loading, and instead displays this error:
Uncaught exception - mysql.c(509) : Failed to connect to mysql server : Failed to resolve host 'dbHost'
Called from ? line 1
Called from website/Server.hx line 46
Called from website/Server.hx line 52
Called from haxelib/server/SiteDb.hx line 127
Called from /usr/local/share/haxe/std/neko/_std/sys/db/Mysql.hx line 203
Finally, the integration tests fail to connect to the server. The haxelib commands return:
Error: Failed with error: Custom(std@socket_recv)
Any help/pointers would be appreciated.
For the version mismatch warning, the latest version of earthly is 0.6.21, try it?
You said the local website wasn't loading - how did you launch it? If it's docker-compose -f test/docker-compose.yml up -d, you can try again without the -d flag (detach mode), such that you can see the log and look for hints. One common cause of issues that prevent the containers from starting is that there are existing programs using the ports that we want to use. The ports are 80 for the web server and 3306 for the MySQL db, see test/docker-compose.yml.
For the integration tests, my local run of earthly --allow-privileged +ci-tests completed without error. I guess it's the same port conflict problem.
Thanks for the tips!
I haven't yet tried 0.6.21, I'll have to give it a go when the repositories for my distro are updated.
docker-compose -f test/docker-compose.yml up -d was how I was starting the server. When I remove the -d flag, I do indeed get the complete log. Looks like there's a mysql issue:
test-dbHost-1 | 2022-08-08 20:01:47+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.36-1debian10 started.
test-haxelib_server-1 | *** Running /etc/my_init.d/00_regen_ssh_host_keys.sh...
test-haxelib_server-1 | *** Running /etc/my_init.d/10_syslog-ng.init...
test-dbHost-1 | 2022-08-08 20:01:47+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
test-dbHost-1 | 2022-08-08 20:01:47+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 5.7.36-1debian10 started.
test-haxelib_server-1 | Aug 8 20:01:47 7ff29bd797f7 syslog-ng[13]: syslog-ng starting up; version='3.25.1'
test-dbHost-1 | 2022-08-08T20:01:47.427756Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
test-dbHost-1 | 2022-08-08T20:01:47.428439Z 0 [Note] mysqld (mysqld 5.7.36) starting as process 1 ...
test-dbHost-1 | 2022-08-08T20:01:47.431219Z 0 [Note] InnoDB: PUNCH HOLE support available
test-dbHost-1 | 2022-08-08T20:01:47.431230Z 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
test-dbHost-1 | 2022-08-08T20:01:47.431233Z 0 [Note] InnoDB: Uses event mutexes
test-dbHost-1 | 2022-08-08T20:01:47.431235Z 0 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
test-dbHost-1 | 2022-08-08T20:01:47.431237Z 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
test-dbHost-1 | 2022-08-08T20:01:47.431239Z 0 [Note] InnoDB: Using Linux native AIO
test-dbHost-1 | 2022-08-08T20:01:47.431690Z 0 [Note] InnoDB: Number of pools: 1
test-dbHost-1 | 2022-08-08T20:01:47.432414Z 0 [Note] InnoDB: Using CPU crc32 instructions
test-dbHost-1 | 2022-08-08T20:01:47.433706Z 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M
test-dbHost-1 | 2022-08-08T20:01:47.439334Z 0 [Note] InnoDB: Completed initialization of buffer pool
test-dbHost-1 | 2022-08-08T20:01:47.441087Z 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
test-dbHost-1 | 20:01:47 UTC - mysqld got signal 11 ;
test-dbHost-1 | This could be because you hit a bug. It is also possible that this binary
test-dbHost-1 | or one of the libraries it was linked against is corrupt, improperly built,
test-dbHost-1 | or misconfigured. This error can also be caused by malfunctioning hardware.
test-dbHost-1 | Attempting to collect some information that could help diagnose the problem.
test-dbHost-1 | As this is a crash and something is definitely wrong, the information
test-dbHost-1 | collection process might fail.
test-dbHost-1 |
test-dbHost-1 | key_buffer_size=8388608
test-dbHost-1 | read_buffer_size=131072
test-dbHost-1 | max_used_connections=0
test-dbHost-1 | max_threads=151
test-dbHost-1 | thread_count=0
test-dbHost-1 | connection_count=0
test-dbHost-1 | It is possible that mysqld could use up to
test-dbHost-1 | key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 68197 K bytes of memory
test-dbHost-1 | Hope that's ok; if not, decrease some variables in the equation.
test-dbHost-1 |
test-dbHost-1 | Thread pointer: 0x0
test-dbHost-1 | Attempting backtrace. You can use the following information to find out
test-dbHost-1 | where mysqld died. If you see no messages after this, something went
test-dbHost-1 | terribly wrong...
test-dbHost-1 | stack_bottom = 0 thread_stack 0x40000
test-dbHost-1 | mysqld(my_print_stacktrace+0x2c)[0x5590a919998c]
test-dbHost-1 | mysqld(handle_fatal_signal+0x501)[0x5590a8a99f71]
test-dbHost-1 | /lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7f3252d21730]
test-dbHost-1 | mysqld(_Z11ut_crc32_hwPKhm+0xd9)[0x5590a94b7299]
test-dbHost-1 | mysqld(_Z22page_zip_calc_checksumPKvm24srv_checksum_algorithm_tb+0x77)[0x5590a93cf6e7]
test-dbHost-1 | mysqld(_Z24page_zip_verify_checksumPKvm+0xa8)[0x5590a93cf808]
test-dbHost-1 | mysqld(_Z21buf_page_is_corruptedbPKhRK11page_size_tb+0x35a)[0x5590a94f8e1a]
test-dbHost-1 | mysqld(_ZN8Datafile19validate_first_pageEPmb+0x431)[0x5590a9580c11]
test-dbHost-1 | mysqld(_ZN13SysTablespace24read_lsn_and_check_flagsEPm+0x78)[0x5590a9587368]
test-dbHost-1 | mysqld(_ZN13SysTablespace14open_or_createEbbPmS0_+0x372)[0x5590a9588f22]
test-dbHost-1 | mysqld(_Z34innobase_start_or_create_for_mysqlv+0x2688)[0x5590a946eeb8]
test-dbHost-1 | mysqld(+0xfa4d20)[0x5590a9341d20]
test-dbHost-1 | mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x55)[0x5590a8aebc35]
test-dbHost-1 | mysqld(+0xbf7126)[0x5590a8f94126]
test-dbHost-1 | mysqld(_Z40plugin_register_builtin_and_init_core_sePiPPc+0x1dc)[0x5590a8f95ffc]
test-dbHost-1 | mysqld(+0x6f773e)[0x5590a8a9473e]
test-dbHost-1 | mysqld(_Z11mysqld_mainiPPc+0x758)[0x5590a8a95ba8]
test-dbHost-1 | /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb)[0x7f32527e909b]
test-dbHost-1 | mysqld(_start+0x2a)[0x5590a8a8abea]
test-dbHost-1 | The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
test-dbHost-1 | information that should help you find out what is causing the crash.
So I'm guessing that I might have to investigate mysql to figure out what's going on exactly. It must have something to do with my system since github actions (and your machine) is unaffected. I've refreshed the cache multiple times only to have the same issue so it probably isn't a one off corruption either.
Maybe the MySQL data is somehow corrupted. You may try to remove the docker volume and let it start clean:
docker-compose -f test/docker-compose.yml down -v
The -v flag means volumes (https://docs.docker.com/engine/reference/commandline/compose_down/).
Thanks! That has fixed the website. Running regular haxelib commands with this server now works, and when I try to run the integration tests normally just by compiling integration_tests.hxml and setting all the required environment variables (the ones set in the Earthfile), everything works fine.
However, when I start to run the tests via earthly --allow-privileged +ci-tests it still fails in the same way, mentioning the Custom(std@socket_recv) error again.
Just a minor correction to what I said, this is what I run to do the tests (locally instead of via earthly):
export HAXELIB_SERVER=localhost && export HAXELIB_SERVER_PORT=80 \
&& export HAXELIB_DB_HOST=localhost && export HAXELIB_DB_PORT=3306 \
&& export HAXELIB_DB_USER=dbUser && export HAXELIB_DB_PASS=dbPass \
&& export HAXELIB_DB_NAME=haxelib
haxe ci.hxml
neko bin/ci.n
This is required so that the dbconfig.json file is set correctly.
Also, I get a different error now when I try to run earthly --allow-privileged +ci-tests:
+ci-tests *failed* | Command: neko [bin/integration_tests.n]
+ci-tests *failed* | Class: tests.integration.TestEmpty Called from ? line 1
+ci-tests *failed* | Called from IntegrationTests.hx line 256
+ci-tests *failed* | Called from haxe/unit/TestRunner.hx line 128
+ci-tests *failed* | Called from haxe/unit/TestRunner.hx line 149
+ci-tests *failed* | Called from IntegrationTests.hx line 185
+ci-tests *failed* | Called from /usr/share/haxe/std/neko/_std/sys/db/Mysql.hx line 202
+ci-tests *failed* | Uncaught exception - mysql.c(509) : Failed to connect to mysql server : Failed to read handshake packet
But the website still works when I open localhost in a browser, and the tests run fine when not running via earthly.