dd-trace-rb icon indicating copy to clipboard operation
dd-trace-rb copied to clipboard
verified publisher icon DataDog

Test: Raise test error for unexpected warn/error messages

Open marcotc opened this issue 7 months ago • 3 comments

Oh boy, this might take a while to fix all tests...

Change log entry No

How to test the change?

marcotc avatar May 21 '25 13:05 marcotc

Datadog Report

Branch report: tmp/marco-test-this Commit report: 389084c Test service: dd-trace-rb

:x: 833 Failed (0 Known Flaky), 18367 Passed, 1293 Skipped, 23m 18.48s Total Time :snowflake: 169 New Flaky

:x: Failed Tests (833)

This report shows up to 5 failed tests.

  • ActiveRecord multi-database implementation when databases are configured with a String that's a URL an invalid value for a typical server fails to configure - rspec - Details

    Expand for error 
    nexpected Datadog.logger.error(): Failed to resolve key "some invalid string". Cause: URI::InvalidURIError Source: /opt/jruby/lib/ruby/stdlib/uri/rfc2396_parser.rb:177:in \`split'

ailure/Error: raise "Unexpected Datadog.logger.error(): #{msg}"

untimeError:
 Unexpected Datadog.logger.error(): Failed to resolve key "some invalid string". Cause: URI::InvalidURIError Source: /opt/jruby/lib/ruby/stdlib/uri/rfc2396_parser.rb:177:in \`split'
/spec/support/log_helpers.rb:24:in \`block in included'
/lib/datadog/tracing/contrib/active_record/configuration/resolver.rb:97:in \`parse_matcher'
/lib/datadog/tracing/contrib/configuration/resolver.rb:51:in \`get'
/lib/datadog/tracing/contrib/configurable.rb:49:in \`configure'
..

  • contrib integration testing dynamic configuration with dynamic configuration data for tracing_header_tags changes the HTTP header tagging for span - rspec - Details

    Expand for error 
    xpected 2 items, got 1

ailure/Error: expect(spans).to have(2).items
 expected 2 items, got 1
/spec/datadog/tracing/contrib/suite/integration_spec.rb:177:in 'block (5 levels) in <top (required)>'
/spec/support/http_server_helpers.rb:55:in 'block in HttpServerHelpers::ClassMethods#http_server'
/spec/datadog/tracing/contrib/suite/integration_spec.rb:19:in 'block (3 levels) in <top (required)>'
usr/local/bundle/gems/climate_control-1.2.0/lib/climate_control.rb:24:in 'block in ClimateControl#modify'
usr/local/bundle/gems/climate_control-1.2.0/lib/climate_control.rb:15:in 'Monitor#synchronize'
usr/local/bundle/gems/climate_control-1.2.0/lib/climate_control.rb:15:in 'ClimateControl#modify'
..

  • Datadog::AppSec::Component#reconfigure when the new processor is not ready does not swap the processor instance and finalize the old processor - rspec - Details

    Expand for error 
    nexpected Datadog.logger.error():

ailure/Error: raise "Unexpected Datadog.logger.error(): #{msg}"

untimeError:
 Unexpected Datadog.logger.error(): 
/spec/support/log_helpers.rb:24:in \`block in included'
/lib/datadog/appsec/processor.rb:77:in \`create_waf_handle'
/lib/datadog/appsec/processor.rb:20:in \`initialize'
/lib/datadog/appsec/component.rb:96:in \`block in reconfigure'
..

  • Datadog::AppSec::Processor::RuleLoader#load_ruleset when ruleset is a non existing path returns nil`` - rspec - Details

    Expand for error 
    nexpected Datadog.logger.error():

ailure/Error: raise "Unexpected Datadog.logger.error(): #{msg}"

untimeError:
 Unexpected Datadog.logger.error(): 
/spec/support/log_helpers.rb:24:in \`block in included'
/lib/datadog/appsec/processor/rule_loader.rb:33:in \`load_rules'
/spec/datadog/appsec/processor/rule_loader_spec.rb:25:in \`block in <main>'
/spec/datadog/appsec/processor/rule_loader_spec.rb:69:in \`block in <main>'
..

  • Datadog::AppSec::Processor::RuleLoader#load_ruleset when ruleset is not parseable returns nil`` - rspec - Details

    Expand for error 
    nexpected Datadog.logger.error():

ailure/Error: raise "Unexpected Datadog.logger.error(): #{msg}"

untimeError:
 Unexpected Datadog.logger.error(): 
/spec/support/log_helpers.rb:24:in \`block in included'
/lib/datadog/appsec/processor/rule_loader.rb:33:in \`load_rules'
/spec/datadog/appsec/processor/rule_loader_spec.rb:25:in \`block in <main>'
/spec/datadog/appsec/processor/rule_loader_spec.rb:100:in \`block in <main>'
..

New Flaky Tests (169)

  • ActiveJob with active_job instrumentation instruments active_job.enqueue_retry and active_job.retry_stopped - rspec - Last Failure

    Expand for error 
    
untimeError:
/spec/support/log_helpers.rb:12:in \`block (2 levels) in included'
/lib/datadog/tracing/contrib/extensions.rb:97:in \`block in configure'
/lib/datadog/tracing/contrib/extensions.rb:78:in \`configure'
/spec/datadog/tracing/contrib/rails/rails_active_job_spec.rb:53:in \`block (3 levels) in <top (required)>'
/spec/datadog/tracing/contrib/rails/support/log_configuration.rb:7:in \`block (2 levels) in <top (required)>'
/spec/datadog/tracing/contrib/support/tracer_helpers.rb:96:in \`block (2 levels) in <module:TracerHelpers>'
/spec/spec_helper.rb:266:in \`block (2 levels) in <top (required)>'
/spec/spec_helper.rb:151:in \`block (2 levels) in <top (required)>'
..

  • ActiveJob with active_job instrumentation instruments discard - rspec - Last Failure

    Expand for error 
    
untimeError:
/spec/support/log_helpers.rb:12:in \`block (2 levels) in included'
/lib/datadog/tracing/contrib/extensions.rb:97:in \`block in configure'
/lib/datadog/tracing/contrib/extensions.rb:78:in \`configure'
/spec/datadog/tracing/contrib/rails/rails_active_job_spec.rb:53:in \`block (3 levels) in <top (required)>'
/spec/datadog/tracing/contrib/rails/support/log_configuration.rb:7:in \`block (2 levels) in <top (required)>'
/spec/datadog/tracing/contrib/support/tracer_helpers.rb:96:in \`block (2 levels) in <module:TracerHelpers>'
/spec/spec_helper.rb:266:in \`block (2 levels) in <top (required)>'
/spec/spec_helper.rb:151:in \`block (2 levels) in <top (required)>'
..

  • ActiveJob with active_job instrumentation instruments enqueue - rspec - Last Failure

    Expand for error 
    
untimeError:
/spec/support/log_helpers.rb:12:in \`block (2 levels) in included'
/lib/datadog/tracing/contrib/extensions.rb:97:in \`block in configure'
/lib/datadog/tracing/contrib/extensions.rb:78:in \`configure'
/spec/datadog/tracing/contrib/rails/rails_active_job_spec.rb:53:in \`block (3 levels) in <top (required)>'
/spec/datadog/tracing/contrib/rails/support/log_configuration.rb:7:in \`block (2 levels) in <top (required)>'
/spec/datadog/tracing/contrib/support/tracer_helpers.rb:96:in \`block (2 levels) in <module:TracerHelpers>'
/spec/spec_helper.rb:266:in \`block (2 levels) in <top (required)>'
/spec/spec_helper.rb:151:in \`block (2 levels) in <top (required)>'
..

  • ActiveJob with active_job instrumentation instruments enqueue_at under the "enqueue" span - rspec - Last Failure

    Expand for error 
    
untimeError:
/spec/support/log_helpers.rb:12:in \`block (2 levels) in included'
/lib/datadog/tracing/contrib/extensions.rb:97:in \`block in configure'
/lib/datadog/tracing/contrib/extensions.rb:78:in \`configure'
/spec/datadog/tracing/contrib/rails/rails_active_job_spec.rb:53:in \`block (3 levels) in <top (required)>'
/spec/datadog/tracing/contrib/rails/support/log_configuration.rb:7:in \`block (2 levels) in <top (required)>'
/spec/datadog/tracing/contrib/support/tracer_helpers.rb:96:in \`block (2 levels) in <module:TracerHelpers>'
/spec/spec_helper.rb:266:in \`block (2 levels) in <top (required)>'
/spec/spec_helper.rb:151:in \`block (2 levels) in <top (required)>'
..

  • ActiveJob with active_job instrumentation instruments perform - rspec - Last Failure

    Expand for error 
    
untimeError:
/spec/support/log_helpers.rb:12:in \`block (2 levels) in included'
/lib/datadog/tracing/contrib/extensions.rb:97:in \`block in configure'
/lib/datadog/tracing/contrib/extensions.rb:78:in \`configure'
/lib/datadog/tracing/contrib/rails/framework.rb:37:in \`setup'
/lib/datadog/tracing/contrib/rails/patcher.rb:84:in \`setup_tracer'
/lib/datadog/tracing/contrib/rails/patcher.rb:78:in \`block in after_initialize'
/lib/datadog/core/utils/only_once.rb:26:in \`block in run'
/lib/datadog/core/utils/only_once.rb:21:in \`synchronize'
..

datadog-datadog-prod-us1[bot] avatar May 21 '25 14:05 datadog-datadog-prod-us1[bot]

Benchmarks

Benchmark execution time: 2025-05-21 14:19:44

Comparing candidate commit 389084c5a0ffaf5d36a8398f63a1ccd150521281 in PR branch tmp/marco-test-this with baseline commit 2deebe6eeac70da8e3908c0a185f1f9f1ca8781a in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 39 metrics, 6 unstable metrics.

pr-commenter[bot] avatar May 21 '25 14:05 pr-commenter[bot]

Hey, is this PR still relevant?

ivoanjo avatar Nov 27 '25 09:11 ivoanjo