tools icon indicating copy to clipboard operation
tools copied to clipboard

incorrect branch and line coverage with const values

Open christopherfujino opened this issue 2 years ago • 1 comments

For the file:

// main.dart

abstract class SuperKlass {
  const SuperKlass();
}

class Klass extends SuperKlass {
  const Klass() : super();
  double divide(double numerator, double denominator) {
    if (denominator != 0) {
      return numerator / denominator;
    } else {
      return double.nan;
    }
  }
}

With the test:

import 'package:dart_coverage_test/dart_coverage_test.dart';
import 'package:test/test.dart';

void main() {
  test('calculate', () {
    expect(const Klass().divide(4, 2), 2);
  });
}

The coverage report looks like:

Screenshot from 2023-02-14 17-40-20

Which has the following problems:

  1. There are failing branch coverage for the two const constructors. It is surprising to me that these are detected as branches, and it seems a bug that they are failing branch coverage.
  2. Line coverage is 100%, even though the else clause is not reached from the test. Note, if I substitute double.nan for a non-const value, it is correctly detected as missing line coverage. Also interesting, this is detected as missing branch coverage.

I used Dart SDK version 3.0.0-233.0.dev (Mon Feb 13) and package:coverage version 1.6.3.

christopherfujino avatar Feb 15 '23 01:02 christopherfujino

All the code from this example I have hosted (along with the generated HTML reports) at https://github.com/christopherfujino/dart_coverage_issue

christopherfujino avatar Feb 15 '23 01:02 christopherfujino

@mosuem This problem also seems to occur in Dart v3.6.1 (stable). Any updates?

acn-masatadakurihara avatar Feb 17 '25 07:02 acn-masatadakurihara

This problem also seems to occur in Dart v3.6.1 (stable). Any updates?

I don't know much about package:coverage, I just transferred the issue ;) @liamappelbe maybe?

mosuem avatar Feb 17 '25 08:02 mosuem

@mosuem Unfortunately, this also seems to occur with the flutter test --branch-coverage command.

acn-masatadakurihara avatar Feb 17 '25 08:02 acn-masatadakurihara

Oops, I guess I missed this bug.

@acn-masatadakurihara The original post mentions two issues. Which are you running into?

  1. There are failing branch coverage for the two const constructors. It is surprising to me that these are detected as branches, and it seems a bug that they are failing branch coverage.
  2. Line coverage is 100%, even though the else clause is not reached from the test. Note, if I substitute double.nan for a non-const value, it is correctly detected as missing line coverage. Also interesting, this is detected as missing branch coverage.

Number 2 isn't really a bug, it's pretty much just how line coverage has always worked in Dart (it's based on function calls rather than lines/branches). That gap is what branch coverage is supposed to fill, and it's working correctly here (the else case has a missing branch coverage icon).

Not sure what's going on with number 1. I'll need to investigate.

liamappelbe avatar Feb 17 '25 23:02 liamappelbe

@liamappelbe Thanks for the response. Unfortunately, the problem with number 1 still seems to be occurring.

Image

The Flutter environment is as follows

[✓] Flutter (Channel stable, 3.27.3, on macOS 15.3 24D60 darwin-arm64, locale ja-JP)
    • Flutter version 3.27.3 on channel stable at /Users/masatada.kurihara/fvm/versions/3.27.3
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision c519ee916e (4 weeks ago), 2025-01-21 10:32:23 -0800
    • Engine revision e672b006cb
    • Dart version 3.6.1
    • DevTools version 2.40.2

acn-masatadakurihara avatar Feb 19 '25 05:02 acn-masatadakurihara

@liamappelbe any update?

acn-masatadakurihara avatar Mar 14 '25 11:03 acn-masatadakurihara

The way coverage collection works for const constructors is a bit of a hack. After ordinary coverage collection is complete and all ordinary source ranges have been added to the report, SourceReport::CollectConstConstructorCoverageFromScripts iterates through all the const constructors and adds extra ranges to the source report that report them as being hit. The old ranges are still in the report, reporting the constructors as missed, but the miss is overwritten by the hit when package:coverage turns the source report into a coverage report.

That hacky approach won't work for branch coverage. Instead I'll have to refactor the source report logic so that instead of just appending those ranges at the end of the report, they're used to correctly report hits and misses in the main set of ranges.

liamappelbe avatar Apr 09 '25 05:04 liamappelbe

Hello @liamappelbe my name is Viral and I wanted to know if you would be working on this issue alone or If it works with you I would really like to help in this issue :).I have been understanding the package for the past days and do have a fair idea about its working and would really like to work on this with you

vvs-personalstash avatar Apr 09 '25 08:04 vvs-personalstash

@vvs-personalstash The changes needed to fix this bug are in the Dart VM, not this repo. You probably don't want to try and dive into the VM repo just yet 😉

liamappelbe avatar Apr 09 '25 08:04 liamappelbe

I see 😅that does seems of out of my skill range right now. I would look for another opportunity to contribute . Thank You.

vvs-personalstash avatar Apr 09 '25 08:04 vvs-personalstash

The current approach also has a bug where calls inside the const constructor are not hit. For example, if we move the super() call to the next line, it's counted as missed:

Image

liamappelbe avatar Apr 10 '25 03:04 liamappelbe