opa test should show readable errors when equality failed

[grosser]

given a test like this:

test_canonify_cpu_m {
  canonify_cpu("1m") == 2
}

• the test output (without --verbose) does not tell me what the actual number was
• the test output with --verbose does not help either (actual output was 1)

data.k8sresourcelimits.test_canonify_cpu_m: FAIL (100.397µs)

  Enter data.k8sresourcelimits.test_canonify_cpu_m = _
  | Enter data.k8sresourcelimits.test_canonify_cpu_m
  | | Enter data.k8sresourcelimits.canonify_cpu
  | | | Fail is_number(orig)
  | | Enter data.k8sresourcelimits.canonify_cpu
  | | | Enter is_number(orig)
  | | | | Fail is_number(orig)
  | | Fail __local290__ = 2
  | | Enter data.k8sresourcelimits.canonify_cpu
  | | | Enter is_number(orig)
  | | | | Fail is_number(orig)
  | | | Fail not endswith(orig, "m")
  | Fail data.k8sresourcelimits.test_canonify_cpu_m = _

please either show "expected 3 to equal 2" (at least in vebose) or a helper method like equal(a, b) that then generates a readable error message

[tsandall]

This would be a nice feature. Personally, I write & test inside of VS Code in which case I can just highlight the part of the test that I'm interested in (e.g., canonify_cpu("1m")) and OPA: Evaluate Selection. If you're on the command-line the options would be opa eval (which is a bit low-level) or the REPL.

This is similar to #1581.

[grosser]

thx, the opa eval helped me build a script I can copy-paste my test-cases into :D

[anderseknert]

It would be nice if this could use the same kind of logic for dealing with undefined as the upcoming print function, so that assertions failing on undefined values would still print something:

test_something {
    some_rule := result with input as { ... }
    
    result.number == 5
}

The above would then render something like Assertion in test_something failed: <undefined> == 5"

[schneefisch]

:+1:

[anderseknert]

If anyone is looking for this before it's solved in OPA, I wrote a little test assertion library to help with this some time back.