erlmld
erlmld copied to clipboard
Published
20 hours ago •
AdRoll
AdRoll
Add kinesis integration testing
- docker compose.yml creates a testing environment: * Use kinesalite and dynalite (github.com/mhart) to provide kinesis and dynamodb work-alikes * Create ca-certificate and server cert signed with this ca, so that kinesalite and dynalite can pose as the real deals * Insert the ca-cert into trusted certificats for the test container * Create dns aliases for standard aws endpoints
- New consume_SUITE.erl and load_from_dir script enable testing against real AWS or using the docker compose env
- Add Makefile to hold the required commands to start tests under docker
I'm on a running it in my mac and I get:
error: failed switching to "work_user": unable to find user work_user: no matching entries in passwd file
What command did you use exactly?
I highly recommend documenting the setup in the README.
I'll add something once I understand what's going on.. for me the setup was apt install docker-ce then make test; obviously that lacks .. something?
Almost working, but I still got this error while I was running the last commit.
test-tests-1 | ===> Running Common Test suites...
test-tests-1 | %%% consume_SUITE:
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:02.262740 ===
test-tests-1 | erlmld_sup_simple_consume_test listening on 40593
test-tests-1 |
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:02.265542 ===
test-tests-1 | erlmld_runner_simple_consume_test launched "/home/work/copy/_build/test/lib/erlmld/priv/run_kinesis.sh" (pid 1136)
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:02.608564 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: child_terminated
test-tests-1 | reason: {child_exited,1136,{exit_status,256}}
test-tests-1 | offender: [{pid,<0.2132.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:02.609100 ===
test-tests-1 | erlmld_runner_simple_consume_test launched "/home/work/copy/_build/test/lib/erlmld/priv/run_kinesis.sh" (pid 1157)
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:02.966878 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: child_terminated
test-tests-1 | reason: {child_exited,1157,{exit_status,256}}
test-tests-1 | offender: [{pid,<0.2135.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:02.967439 ===
test-tests-1 | erlmld_runner_simple_consume_test launched "/home/work/copy/_build/test/lib/erlmld/priv/run_kinesis.sh" (pid 1178)
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:03.307353 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: child_terminated
test-tests-1 | reason: {child_exited,1178,{exit_status,256}}
test-tests-1 | offender: [{pid,<0.2138.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:03.307840 ===
test-tests-1 | erlmld_runner_simple_consume_test launched "/home/work/copy/_build/test/lib/erlmld/priv/run_kinesis.sh" (pid 1199)
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:03.652304 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: child_terminated
test-tests-1 | reason: {child_exited,1199,{exit_status,256}}
test-tests-1 | offender: [{pid,<0.2141.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:03.652876 ===
test-tests-1 | erlmld_runner_simple_consume_test launched "/home/work/copy/_build/test/lib/erlmld/priv/run_kinesis.sh" (pid 1220)
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:04.028469 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: child_terminated
test-tests-1 | reason: {child_exited,1220,{exit_status,256}}
test-tests-1 | offender: [{pid,<0.2144.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:04.029812 ===
test-tests-1 | erlmld_runner_simple_consume_test launched "/home/work/copy/_build/test/lib/erlmld/priv/run_kinesis.sh" (pid 1241)
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:04.417206 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: child_terminated
test-tests-1 | reason: {child_exited,1241,{exit_status,256}}
test-tests-1 | offender: [{pid,<0.2147.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:04.417784 ===
test-tests-1 | erlmld_runner_simple_consume_test launched "/home/work/copy/_build/test/lib/erlmld/priv/run_kinesis.sh" (pid 1262)
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:04.764537 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: child_terminated
test-tests-1 | reason: {child_exited,1262,{exit_status,256}}
test-tests-1 | offender: [{pid,<0.2150.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:04.765446 ===
test-tests-1 | erlmld_runner_simple_consume_test launched "/home/work/copy/_build/test/lib/erlmld/priv/run_kinesis.sh" (pid 1283)
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:05.152821 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: child_terminated
test-tests-1 | reason: {child_exited,1283,{exit_status,256}}
test-tests-1 | offender: [{pid,<0.2153.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:05.153767 ===
test-tests-1 | erlmld_runner_simple_consume_test launched "/home/work/copy/_build/test/lib/erlmld/priv/run_kinesis.sh" (pid 1304)
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:05.495754 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: child_terminated
test-tests-1 | reason: {child_exited,1304,{exit_status,256}}
test-tests-1 | offender: [{pid,<0.2156.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:05.496381 ===
test-tests-1 | erlmld_runner_simple_consume_test launched "/home/work/copy/_build/test/lib/erlmld/priv/run_kinesis.sh" (pid 1325)
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:05.838265 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: child_terminated
test-tests-1 | reason: {child_exited,1325,{exit_status,256}}
test-tests-1 | offender: [{pid,<0.2159.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 | =INFO REPORT==== 7-Jul-2023::20:59:05.838710 ===
test-tests-1 | erlmld_runner_simple_consume_test launched "/home/work/copy/_build/test/lib/erlmld/priv/run_kinesis.sh" (pid 1346)
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:06.211647 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: child_terminated
test-tests-1 | reason: {child_exited,1346,{exit_status,256}}
test-tests-1 | offender: [{pid,<0.2162.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 | =SUPERVISOR REPORT==== 7-Jul-2023::20:59:06.211692 ===
test-tests-1 | supervisor: {local,erlmld_sup_simple_consume_test}
test-tests-1 | errorContext: shutdown
test-tests-1 | reason: reached_max_restart_intensity
test-tests-1 | offender: [{pid,<0.2162.0>},
test-tests-1 | {id,mld_runner},
test-tests-1 | {mfargs,{erlmld_runner,start_link,
test-tests-1 | [erlmld_runner_simple_consume_test,
test-tests-1 | "/tmp/erlmld/erlmld.simple_consume_test.properties",
test-tests-1 | kinesis]}},
test-tests-1 | {restart_type,permanent},
test-tests-1 | {significant,false},
test-tests-1 | {shutdown,brutal_kill},
test-tests-1 | {child_type,worker}]
test-tests-1 |
test-tests-1 |
test-tests-1 | %%% consume_SUITE ==> simple_consume_test: FAILED
test-tests-1 | %%% consume_SUITE ==> {'EXIT',shutdown}
test-tests-1 |
test-tests-1 | EXPERIMENTAL: Writing retry specification at /home/work/copy/_build/test/logs/retry.spec
test-tests-1 | call rebar3 ct with '--retry' to re-run failing cases.
test-tests-1 | Failed 1 tests. Passed 0 tests.
test-tests-1 | Results written to "/home/work/copy/_build/test/logs/index.html".
test-tests-1 | ===> Failures occurred running tests: 1
Please do not merge this PR before it has been tested in mac arm and mac x86. As soon as we get it working on x86 I'll test it in arm. Thaaanks!
