dd-trace-rb
dd-trace-rb copied to clipboard
Add parametric system-tests workflow
What does this PR do?
This PR adds a GitHub workflow for parametrics system-tests
Motivation:
The SCA Env Var system-tests is run as a parametric test, and I wanted to test it on the CI.
Additional Notes:
The FORCE_TESTS env var does not work as the -F option on system-tests only work for the DEFAULT scenario right now. There are currently 4 tests that needs to be fixed:
- [ ] tests/parametric/test_dynamic_configuration.py::TestDynamicConfigSamplingRules::test_trace_sampling_rules_with_tags
- [ ] tests/parametric/test_otel_span_methods.py::Test_Otel_Span_Methods::test_otel_span_started_with_link_from_datadog_headers
- [ ] tests/parametric/test_otel_span_methods.py::Test_Otel_Span_Methods::test_otel_span_started_with_link_from_w3c_headers
- [ ] tests/parametric/test_otel_span_methods.py::Test_Otel_Span_Methods::test_otel_span_started_with_link_from_other_spans
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 98.11%. Comparing base (
eab4b6e) to head (eddd956). Report is 2 commits behind head on master.
Additional details and impacted files
@@ Coverage Diff @@
## master #3698 +/- ##
=======================================
Coverage 98.11% 98.11%
=======================================
Files 1225 1225
Lines 72743 72743
Branches 3479 3479
=======================================
Hits 71369 71369
Misses 1374 1374
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Hey @vpellan is this PR still relevant?
Hey @vpellan is this PR still relevant?
This is still relevant (it is useful to run parametric system-tests) but needs to be reworked. We still have it in AppSec backlog (although it is not specific to AppSec)
Ack! Do you want to keep this PR around as reference, or shall we close it for now?
Ack! Do you want to keep this PR around as reference, or shall we close it for now?
I think it's better to keep it as reference, we can close it to reduce the number of open PR but I may reopen it at some point
Nah, no rush, let's keep it open then! :)
Well actually this has been fixed by a61a09e (#4174) so we can close this PR