tarpaulin icon indicating copy to clipboard operation
tarpaulin copied to clipboard

Potential bug when computing coverage using combinators

Open mihaigalos opened this issue 1 year ago • 2 comments

Describe the bug Using combinators (and_then, or_else)), coverage seems to not report the initial line before the combinator.

To Reproduce

pub struct Parser;

impl Parser {
    pub fn new() -> Self {
        return Self {};
    }
    pub fn scheme<'a>(&self, input: &'a str) -> Option<&'a str> {
        let split: Vec<&str> = input.split("://").collect();

        match split.len() {
            2 => return Some(split[0]),
            _ => return None,
        };
    }
}

pub fn canonicalize<'a>(parser: &Parser, input: &'a str) -> String {
    parser
        .scheme(input)
        .and_then(|s| Some(s.to_string() + "://"))
        .or_else(|| Some("".to_string()))
        .unwrap()
}

#[cfg(not(tarpaulin_include))]
fn main() {}

#[cfg(test)]
mod tests {
    use super::*;
    #[test]
    fn test_scheme_works_when_typical() {
        let input = "https://www.example.co.uk:443/blog/article/search?docid=720&hl=en#dayone";
        let scheme = Parser::new().scheme(input);
        assert_eq!(scheme.unwrap(), "https");
    }

    #[test]
    fn test_scheme_works_when_no_port() {
        let input = "https://www.example.co.uk/blog/article/search?docid=720&hl=en#dayone";
        let scheme = Parser::new().scheme(input);
        assert_eq!(scheme.unwrap(), "https");
    }

    #[test]
    fn test_scheme_works_when_no_scheme() {
        let input = "www.example.co.uk/blog/article/search?docid=720&hl=en#dayone";
        let scheme = Parser::new().scheme(input);
        assert!(scheme.is_none());
    }

    #[test]
    fn test_canonicalize_works_when_typical() {
        let input = "https://www.example.co.uk/blog/article/search?docid=720&hl=en#dayone";
        let expected = "https://";

        let parser = Parser::new();
        let result = canonicalize(&parser, input);
        assert_eq!(result, expected);
    }

    #[test]
    fn test_canonicalize_works_when_no_scheme() {
        let input = "www.example.co.uk/blog/article/search?docid=720&hl=en#dayone";
        let expected = "";

        let parser = Parser::new();
        let result = canonicalize(&parser, input);
        assert_eq!(result, expected);
    }
}

Output:

cargo tarpaulin -v                                                                                                                                   mihai@galos 08/08/22 07:47:34
Aug 08 07:47:34.902 DEBUG cargo_tarpaulin: set up logging
Aug 08 07:47:34.902  INFO cargo_tarpaulin::config: Creating config
Aug 08 07:47:34.915  INFO cargo_tarpaulin: Running Tarpaulin
Aug 08 07:47:34.915  INFO cargo_tarpaulin: Building project
Aug 08 07:47:34.915  INFO cargo_tarpaulin::cargo: Cleaning project
   Compiling foo v0.1.0 (/tmp/tmp.EpBqspzJZA/foo)
     Running `rustc --crate-name foo --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=f6f74d0397cdf000 -C extra-filename=-f6f74d0397cdf000 --out-dir /tmp/tmp.EpBqspzJZA/foo/target/debug/deps -C incremental=/tmp/tmp.EpBqspzJZA/foo/target/debug/incremental -L dependency=/tmp/tmp.EpBqspzJZA/foo/target/debug/deps -C debuginfo=2 --cfg=tarpaulin -C link-dead-code`
    Finished test [unoptimized + debuginfo] target(s) in 0.51s
  Executable `/tmp/tmp.EpBqspzJZA/foo/target/debug/deps/foo-f6f74d0397cdf000`
Aug 08 07:47:35.478  INFO cargo_tarpaulin::process_handling::linux: Launching test
Aug 08 07:47:35.478  INFO cargo_tarpaulin::process_handling: running /tmp/tmp.EpBqspzJZA/foo/target/debug/deps/foo-f6f74d0397cdf000

running 5 tests
test tests::test_canonicalize_works_when_no_scheme ... ok
test tests::test_canonicalize_works_when_typical ... ok
test tests::test_scheme_works_when_no_port ... ok
test tests::test_scheme_works_when_no_scheme ... ok
test tests::test_scheme_works_when_typical ... ok

test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

Aug 08 07:47:35.721  INFO cargo_tarpaulin::report: Coverage Results:
|| Uncovered Lines:
|| src/main.rs: 18
|| Tested/Total Lines:
|| src/main.rs: 34/35 +0.00%
|| 
97.14% coverage, 34/35 lines covered, +0% change in coverage

Expected behavior 100% coverage instead of 97.14% in the provided example.

Seems that line 19 (.scheme(input)) is not covered. I suspect the return None branch in line 12 for scheme() is at play.

I would expect the and_then + or_else to work as an if-else on the original value. Is this not the case?

mihaigalos avatar Aug 08 '22 05:08 mihaigalos

Sorry for replying so late. And they kinda do work the same the issue is more how instructions align with ptrace and the inherent issues in how unix decided to trace processes (a lot built around an assumption that everything is single threaded). Potentially --engine llvm may solve the accuracy issues - it does for a lot of people

xd009642 avatar Jan 26 '23 21:01 xd009642

Seems to have no effect:

» cargo tarpaulin -v --engine llvm
..
97.14% coverage, 34/35 lines covered, +0% change in coverage

mihaigalos avatar Jan 29 '23 15:01 mihaigalos