ocaml icon indicating copy to clipboard operation
ocaml copied to clipboard

Broken test-generation for exercise space-age

Open georgyo opened this issue 1 year ago • 2 comments

This exercise has been marked as having a broken test generator. There are a lots of reasons for test generation to break for a given exercise including

  • New addition of new tests that example.ml does not handle correctly. All that is required here is to fix or replace the example!
  • Changes to the problem-specification's canonical-data.json that our test-gen cannot handle. This could be then additional of field names, changing field names, or addition of tests that not fit with Ocaml's way of doing things. The task here is figure out what changed in the problem-specification and fix or add rules to special_cases.ml.

Useful links

GitHub doesn't allow for linking the diff of an individual file making the relevant canonical-data.json diff impossible to link to. It may be easier to run this in your terminal

git submodule update --init
git -C problem-specifications diff 2af3c9b0074f16c62366c5c533eaacd3ff27b583 -- exercises/space-age/canonical-data.json

Check List

  • [ ] Delete templates/space-age/.broken
  • [ ] Fix test-generation
  • [ ] Test the test generation with make space-age.gentest

georgyo avatar Mar 11 '23 17:03 georgyo

A test case was added to problem-specifications:

      "input": {
        "planet": "Sun",
        "seconds": 680804807
      },
      "expected": {"error": "not a planet"}

One option would be to support this test case: we would require solutions to define a type like

type celestial_body = ... | ... | ... | Sun

We would also need to decide how errors should be reported for this exercise, for example using exceptions or Result.

The other option would be to omit this test case, using include = false

keiravillekode avatar Nov 25 '23 09:11 keiravillekode

I don't think we need a celestial_body type. The test says the name of the planet is Sun, but our Sun is not a planet it is a star. There is no consideration for anything that is not a planet.

Likely excluding this test is the best case, because currently Sun would be a compiler error, ocaml doesn't need any explicit tests to prevent or catch the input of Sun.

In order to even accept Sun as input we would need to change the everything to work with strings instead of a variant type. This would make the easiest solution to pattern match on strings instead of a variant type, which is not ideal ocaml.

Any way you chose to solve this is good by me, just consider what would be best for a person who is just starting to learn Ocaml.

georgyo avatar Nov 25 '23 14:11 georgyo