rebar3
rebar3 copied to clipboard
Published
20 hours ago •
erlang
erlang
rebar3 ct tries to run removed CT modules
Rebar3 report
version 3.14.1
generated at 2020-09-30T20:46:12+00:00
=================
Please submit this along with your issue at https://github.com/erlang/rebar3/issues (and feel free to edit out private information, if any)
-----------------
Task: ct
Entered as:
ct
-----------------
Operating System: x86_64-apple-darwin17.7.0
ERTS: Erlang/OTP 22 [erts-10.7] [source] [64-bit] [smp:12:12] [ds:12:12:10] [async-threads:1] [hipe]
Root Directory: /nix/store/ckqmqmzi9a2ix9980l562rfdazs9hj1w-erlang-22.3/lib/erlang
Library directory: /nix/store/ckqmqmzi9a2ix9980l562rfdazs9hj1w-erlang-22.3/lib/erlang/lib
-----------------
Loaded Applications:
bbmustache: 1.6.0
certifi: 2.3.1
cf: 0.2.2
common_test: 1.18.2
compiler: 7.5.3
crypto: 4.6.5
cth_readable: 1.4.3
dialyzer: 4.1.1
edoc: 0.11
erlware_commons: 1.3.1
eunit: 2.4.1
eunit_formatters: 0.5.0
getopt: 1.0.1
hipe: 3.19.3
inets: 7.1.2
kernel: 6.5.2
providers: 1.7.0
public_key: 1.7.2
relx: 3.28.0
sasl: 3.4.2
snmp: 5.5
ssl_verify_fun: 1.1.3
stdlib: 3.12
syntax_tools: 2.2.1
tools: 3.3.1
-----------------
Escript path: /nix/store/mgqmgi5w92hcq2949mgp0sz72wwq779x-rebar3-3.14.1/bin/rebar3
Providers:
app_discovery as clean compile compile cover ct deps dialyzer do edoc escriptize eunit fmt generate get-deps help install install_deps list lock new path pkgs release relup report repos shell state tar tree unlock update upgrade upgrade upgrade version xref
Current behaviour
When I rename/remove the common test suite module then running rebar3 ct will still try to run that, which will fail as this module no longer exist. Example from running first tests on current master of OpenTelemetry and then checking out version before rename:
[0;36m===> Expanded command sequence to be run: [{default,app_discovery},
{default,install_deps},
{default,lock},
{default,ct}]
[0m[0;36m===> Provider: {default,app_discovery}
[0m[0;36m===> Provider: {default,install_deps}
[0m[0;32m===> Verifying dependencies...
[0m[0;36m===> Provider: {default,lock}
[0m[0;36m===> Provider: {default,ct}
[0m[0;36m===> Compile (apps)
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang", pre_hooks, compile) -> no hooks defined
[0m[0;36m===> sh info:
cwd: "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang"
cmd: mkdir -p /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry_api/test
[0m[0;36m===> opts: [{use_stdout,false},abort_on_error]
[0m[0;36m===> Port Cmd: mkdir -p /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry_api/test
Port Opts: [exit_status,
{line,16384},
use_stdio,stderr_to_stdout,hide,eof,binary]
[0m[0;36m===> sh info:
cwd: "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang"
cmd: cp -Rp /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/test/test_helper.exs /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/test/open_telemetry_test.exs /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/test/opentelemetry_api_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/test/otel_metrics_SUITE.erl "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry_api/test"
[0m[0;36m===> opts: [{use_stdout,true},abort_on_error]
[0m[0;36m===> Port Cmd: cp -Rp /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/test/test_helper.exs /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/test/open_telemetry_test.exs /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/test/opentelemetry_api_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/test/otel_metrics_SUITE.erl "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry_api/test"
Port Opts: [exit_status,
{line,16384},
use_stdio,stderr_to_stdout,hide,eof,binary]
[0m[0;36m===> sh info:
cwd: "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang"
cmd: mkdir -p /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry/test
[0m[0;36m===> opts: [{use_stdout,false},abort_on_error]
[0m[0;36m===> Port Cmd: mkdir -p /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry/test
Port Opts: [exit_status,
{line,16384},
use_stdio,stderr_to_stdout,hide,eof,binary]
[0m[0;36m===> sh info:
cwd: "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang"
cmd: cp -Rp /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_batch_processor_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/opentelemetry_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_samplers_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_test_utils.hrl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_resource_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_metric_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_sweeper_SUITE.erl "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry/test"
[0m[0;36m===> opts: [{use_stdout,true},abort_on_error]
[0m[0;36m===> Port Cmd: cp -Rp /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_batch_processor_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/opentelemetry_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_samplers_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_test_utils.hrl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_resource_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_metric_SUITE.erl /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_sweeper_SUITE.erl "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry/test"
Port Opts: [exit_status,
{line,16384},
use_stdio,stderr_to_stdout,hide,eof,binary]
[0m[0;36m===> Compile (project_apps)
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api", pre_hooks, compile) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry", pre_hooks, compile) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api", pre_hooks, erlc_compile) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry", pre_hooks, erlc_compile) -> no hooks defined
[0m[0;32m===> Analyzing applications...
[0m[0;32m===> Compiling opentelemetry_api
[0m[0;36m===> erlopts [{d,'COMMON_TEST'},
debug_info,
{d,'TEST'},
nowarn_export_all,
{parse_transform,cth_readable_transform}]
[0m[0;36m===> files to compile ["/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_http_status.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_observer.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_meter_provider.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_tracer.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_meter_noop.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_instrument.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_counter.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_sum_observer.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_value_observer.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_ctx.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_updown_counter.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_span.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_updown_sum_observer.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_baggage.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_tracer_noop.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_propagation.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_meter.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/opentelemetry.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_value_recorder.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src/ot_tracer_provider.erl"]
[0m[0;36m===> Starting 0 worker(s)
[0m[0;32m===> Compiling opentelemetry
[0m[0;36m===> erlopts [{d,'COMMON_TEST'},
debug_info,
{d,'TEST'},
nowarn_export_all,
{parse_transform,cth_readable_transform}]
[0m[0;36m===> files to compile ["/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_utils.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_resource_env_var.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_tracer_server.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_meter_default.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_exporter.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_resource_app_env.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_span_sup.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_exporter_tab.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_propagation_http_b3.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_metric_aggregator.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_batch_processor.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_metric_controller_push.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_resource.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_propagation_http_w3c.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_meter_server.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_metric_exporter_stdout.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_exporter_stdout.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_span_processor.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_metric_aggregator_mmsc.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_tracer_default.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_span_utils.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_metric_aggregator_last_value.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_metric_aggregator_sum.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_span_ets.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_exporter_pid.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_metric_accumulator.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_metric_exporter.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_sampler.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_span_sweeper.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/opentelemetry_sup.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_metric_sup.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_metric_integrator.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/opentelemetry_app.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src/ot_metric_aggregator_array.erl"]
[0m[0;36m===> Starting 0 worker(s)
[0m[0;36m===> erlopts [{i,"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/src"},
{d,'COMMON_TEST'},
debug_info,
{d,'TEST'},
nowarn_export_all,
{parse_transform,cth_readable_transform}]
[0m[0;36m===> files to compile ["/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/test/otel_metrics_SUITE.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api/test/opentelemetry_api_SUITE.erl"]
[0m[0;36m===> Starting 0 worker(s)
[0m[0;36m===> erlopts [{i,"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/src"},
{d,'COMMON_TEST'},
debug_info,
{d,'TEST'},
nowarn_export_all,
{parse_transform,cth_readable_transform}]
[0m[0;36m===> files to compile ["/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_sweeper_SUITE.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_metric_SUITE.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_resource_SUITE.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_samplers_SUITE.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/opentelemetry_SUITE.erl",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry/test/ot_batch_processor_SUITE.erl"]
[0m[0;36m===> Starting 0 worker(s)
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api", post_hooks, erlc_compile) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry", post_hooks, erlc_compile) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api", pre_hooks, app_compile) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry", pre_hooks, app_compile) -> no hooks defined
[0m[0;36m===> sh info:
cwd: "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang"
cmd: git log -n 1 --pretty=format:"%h
"
[0m[0;36m===> opts: [{use_stdout,false},
return_on_error,
{cd,"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry"}]
[0m[0;36m===> Port Cmd: git log -n 1 --pretty=format:"%h
"
Port Opts: [{cd,"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry"},
exit_status,
{line,16384},
use_stdio,stderr_to_stdout,hide,eof,binary]
[0m[0;36m===> sh info:
cwd: "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang"
cmd: git -c color.ui=false log --oneline --no-walk --tags --decorate
[0m[0;36m===> opts: [{use_stdout,false},
return_on_error,
{cd,"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry"}]
[0m[0;36m===> Port Cmd: git -c color.ui=false log --oneline --no-walk --tags --decorate
Port Opts: [{cd,"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry"},
exit_status,
{line,16384},
use_stdio,stderr_to_stdout,hide,eof,binary]
[0m[0;36m===> sh info:
cwd: "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang"
cmd: git describe --tags --abbrev=0
[0m[0;36m===> opts: [{use_stdout,false},
return_on_error,
{cd,"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry"}]
[0m[0;36m===> Port Cmd: git describe --tags --abbrev=0
Port Opts: [{cd,"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry"},
exit_status,
{line,16384},
use_stdio,stderr_to_stdout,hide,eof,binary]
[0m[0;36m===> sh info:
cwd: "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang"
cmd: git rev-list HEAD
[0m[0;36m===> opts: [{use_stdout,false},
{cd,"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry"},
{debug_abort_on_error,
"Getting rev-list of git dependency failed in /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry"}]
[0m[0;36m===> Port Cmd: git rev-list HEAD
Port Opts: [{cd,"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry"},
exit_status,
{line,16384},
use_stdio,stderr_to_stdout,hide,eof,binary]
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api", post_hooks, app_compile) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry", post_hooks, app_compile) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry_api", post_hooks, compile) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/apps/opentelemetry", post_hooks, compile) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang", post_hooks, compile) -> no hooks defined
[0m[0;36m===> cover compiling "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry/ebin"
[0m[0;36m===> cover compiling "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry_api/ebin"
[0m[0;32m===> Running Common Test suites...
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang", pre_hooks, ct) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang", pre_hooks, ct) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang", pre_hooks, ct) -> no hooks defined
[0m[0;36m===> ct_opts [{auto_compile,false},
{logdir,"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/logs"},
{dir,["/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry/test",
"/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/lib/opentelemetry_api/test"]},
{abort_if_missing_suites,true},
{ct_hooks,[cth_surefire,cth_readable_failonly,
cth_readable_compact_shell,cth_retry]}]
[0m%%% opentelemetry_SUITE: [0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m
%%% ot_batch_processor_SUITE: [0;32m.[0m[0m
%%% ot_metric_SUITE: [0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m
%%% ot_resource_SUITE: [0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m
%%% ot_samplers_SUITE: [0;32m.[0m[0m[0;32m.[0m[0m
%%% ot_sweeper_SUITE: [0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m
%%% ot_sweeper_SUITE ==> error_in_suite: [0;31mFAILED[0m[0m
%%% ot_sweeper_SUITE ==> [0;31m'otel_batch_processor_SUITE cannot be compiled or loaded'[0m[0m
%%% ot_sweeper_SUITE ==> error_in_suite: [0;31mFAILED[0m[0m
%%% ot_sweeper_SUITE ==> [0;31m'otel_metric_SUITE cannot be compiled or loaded'[0m[0m
%%% ot_sweeper_SUITE ==> error_in_suite: [0;31mFAILED[0m[0m
%%% ot_sweeper_SUITE ==> [0;31m'otel_resource_SUITE cannot be compiled or loaded'[0m[0m
%%% ot_sweeper_SUITE ==> error_in_suite: [0;31mFAILED[0m[0m
%%% ot_sweeper_SUITE ==> [0;31m'otel_samplers_SUITE cannot be compiled or loaded'[0m[0m
%%% ot_sweeper_SUITE ==> error_in_suite: [0;31mFAILED[0m[0m
%%% ot_sweeper_SUITE ==> [0;31m'otel_sweeper_SUITE cannot be compiled or loaded'[0m[0m
%%% opentelemetry_api_SUITE: [0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m
%%% otel_metrics_SUITE: [0;32m.[0m[0m[0;32m.[0m[0m[0;32m.[0m[0m
%%% otel_metrics_SUITE ==> error_in_suite: [0;31mFAILED[0m[0m
%%% otel_metrics_SUITE ==> [0;31m'otel_propagators_SUITE cannot be compiled or loaded'[0m[0m
EXPERIMENTAL: Writing retry specification at /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/logs/retry.spec
call rebar3 ct with '--retry' to re-run failing cases.
All 34 tests passed.
[0;36m===> Cover data written to "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/cover/ct.coverdata".
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang", post_hooks, ct) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang", post_hooks, ct) -> no hooks defined
[0m[0;36m===> run_hooks("/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang", post_hooks, ct) -> no hooks defined
[0m[0;36m===> sh info:
cwd: "/Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang"
cmd: rm -rf /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/logs/last
[0m[0;36m===> opts: [{use_stdout,false},abort_on_error]
[0m[0;36m===> Port Cmd: rm -rf /Users/hauleth/Workspace/open-telemetry/opentelemetry-erlang/_build/test/logs/last
Port Opts: [exit_status,
{line,16384},
use_stdio,stderr_to_stdout,hide,eof,binary]
[0m
rebar3 as test clean do not fixes the problem
Expected behaviour
Old test shouldn't be ran.
Yeah that's an unfortunate limitation of how we have to set things up for CT which wants a specific weird file layout. We end up copying files into an extra directory from which we run CT, and when files are renamed, we don't detect in that directory that the old one doesn't exist and instead the file is left hanging there. Next invocations of CT find it and try to run it regardless.
Yea, that and we don't just want to delete and re-copy all files since we use timestamp to do compilation -- and symlinks aren't an option because they'd put beam files in the test dir.
Is the best way to just do a pre or post copy check that looks to see if each module has a corresponding module in the original test dir and delete if not?
@tsloughter TBH I would expect tests to be recompiled on each rebar3 ct call as environment could change in the meantime and it could also invalidate generated modules.
They are recompiled if they changed. The environment and compile options are tracked in the DAG and if you modify them, things get recompiled. But when we can safely infer that nothing changed in the files (and we drive the analysis starting from the source files, not the build artifacts), we don't recompile.
The thing is that CT asks that the source be present next to the .beam files in a directory that executes tests. So the thing we need to do to please CT is to copy the source file along with the .beam files as if they were a build artifacts, but just for common tests and nothing else ever. If you go look into _build/test/lib/<yourapp>/test/ you will see that the directory is special and is a copy of all your test files with the .beam files right in there rather than in ebin/. If you have many test directories, they all get merged down to this temporary one. This is the weird thing that CT requires but is never really documented. The only great way to work around it is to pair and track each source file in that magic directory with any place where they'd be from, and do some pairwise comparison to know if a file disappeared from the source location to take it out. This tends to cause funky issues if you moved the file because you could risk deleting a source file that was moved but not renamed and then I'm not 100% sure what would happen.
The fix was generally considered so shitty and expensive (it would require duplicating and tweaking compiler-related code for one-off magic rules that other test frameworks do not require) that we more or less decided to eat the loss on it and just have it misbehave when files are renamed since we assume people won't just reorganize their repo every week or so. People generally had the habit of blowing away the _build directory when things went bad and it's one of the cases where they had to fall back to that pattern.
As an aside this pairing of location and source file is also why the rebar3 ct --retry hook cheat thing tends to break a lot. The reinvocation of CT in that directory has it try to understand things based on source files that aren't necessarily pre-loaded properly (I'm not 100% sure) and it tries to complete paths from the incomplete picture and keeps choking.
My approach would be to remove _build/test/lib/<app>/tests after each rebar3 ct, but I assume that it is unwanted solution.
In my mix_erlang library I am compiling test modules in memory only, but I assume that it is also not something that is expected.
yeah, removing them all would essentially massively slow down builds for people who have hundreds of applications and files in their repositories. They'd need to be recompiled every time and that can represent a significant amount of time as the compiler slowed down a lot in the last versions in attempts to do more optimization passes.
Compiling to memory would be interesting, but that would require compiler changes as well, and just for CT, since the common test handler currently just calls to the same compiler modules we use for everything else (.yrl, .mib, .erl, etc.). If we are to drive custom compiler changes, it feels to me like it would be simpler to track the copied .erl files as another build artifact in the DAG, which would let us auto-remove them when we detect the original source is gone. It's just that tracking these files is not needed anywhere else and it really doesn't make sense for any other compiling job to do that.
Yea, much simpler to just compare files across the two directories.
I realized it should be done prior to the compilation because the rebar3 compiler already has the code for removing files from the DAG, so if CT can remove files after its copy but before the call to the compiler then it would handle the rest itself.
@ferd I feel like we didn't understood each other. I was talking about always recompiling only Common Test files (if that is feasible in Rebar3), rest of the compilation artifacts should be left as is.
So was I. People with hundred of OTP applications in their projects also have hundreds if not thousands of test files that wouldn't always need recompiling unless they modified them.