pflua icon indicating copy to clipboard operation
pflua copied to clipboard

Need better regression tests for the optimizer

Open wingo opened this issue 10 years ago • 4 comments

Sometimes we find optimizer bugs that have a precise test case we can run. It would be good to be able to drop input and expected output files into a directory so we can test; e.g.

regression-tests/foo

#!/bin/sh
pflua-optimize <<END
{ "if",
  { "!=",
    "len",
    0 },
  { ">",
    1,
    { "uint32",
      { "+",
        { "uint32",
          { "-",
            4294967295,
            { "uint32",
              { "/",
                1,
                "len" } } } },
        1 } } },
  { "fail" } }
END

regression-tests/foo.expected

Optimizing:
{ "if",
  { "!=",
    "len",
    0 },
  { ">",
    1,
    { "uint32",
      { "+",
        { "uint32",
          { "-",
            4294967295,
            { "uint32",
              { "/",
                1,
                "len" } } } },
        1 } } },
  { "fail" } }
Result:
{ "if",
  { "!=",
    "len",
    0 },
  { "<",
    { "uint32",
      { "+",
        { "-",
          4294967295,
          { "/",
            1,
            "len" } },
        1 } },
    1 },
  { "fail" } }

Then the harness would run the input file, as an executable, and cmp its output to the expected output. If the optimizer changes we might get spurious diffs but we should be seeing the changes, and a simple makefile rule can regenerate expected output. Dunno. Point being, we should let the property tester find the bugs, and the regression tests keep them from happening again.

wingo avatar Feb 03 '15 17:02 wingo

Yes; we need unit tests for a handful of known bugs that we've fixed, whether or not we fit them into the regressions test framework.

kbara avatar Feb 03 '15 17:02 kbara

@kbara would you mind hacking up something for this? that would be swell :)

wingo avatar Feb 03 '15 17:02 wingo

I'll think it over, then hack something up, yes.

kbara avatar Feb 03 '15 17:02 kbara

@andywingo Are you happy with the state of tests/ir-reg/* for this?

kbara avatar May 13 '15 11:05 kbara