test
test copied to clipboard
Exit code of 0 after a failed test
Describe the bug in detail.
A test (or tests) fail, but the exit code for the dart test
process is 0.
@gkc first noticed this on a GitHub Action run. (Full logs at logs_24055523517.zip.)
test/enrollment_test.dart: A test to verify send enroll request, deny enrollment and auth by enrollmentId should fail (failed)
However the job still passed.
This was initially thought to be an issue with GitHub Actions, and @gkc opened https://github.com/atsign-foundation/at_libraries/issues/586 but the behavior has since been replicated locally, so it looks like it's a dart test
problem rather than a GitHub Actions problem.
Include the specific command you ran and any relevant details about the
environment, including operating system, Dart SDK version, and the version of
package:test
you are using.
dart test --concurrency=1
Dart SDK version: 3.4.1 (stable) (Tue May 21 15:46:25 2024 +0000) on "linux_x64"
+ test 1.25.7
+ test_api 0.7.2
+ test_core 0.6.4
Whenever possible, include a full reproduction example that we can run to observe the problem.
This will need Docker Compose.
git clone https://github.com/atsign-foundation/at_libraries.git
cd at_libraries/
git checkout tests-testing
cd tests/at_onboarding_cli_functional_tests/
echo "127.0.0.1 vip.ve.atsign.zone" | sudo tee -a /etc/hosts
dart pub get
sudo docker compose up -d
dart run check_docker_readiness.dart
dart test --concurrency=1
echo $?
@mit-mit this is the thing I mentioned to you last week at I/O Connect, so this is the GitHub issue you asked me to open.