prompting icon indicating copy to clipboard operation
prompting copied to clipboard

Ensure that math challenges contain necessary information

Open steffencruz opened this issue 1 year ago • 2 comments

There are edge cases where math challenges modify or otherwise corrupt the underlying math problem. We can address this by validating the challenge. Once flagged, the options are to regenerate the challenge or to reject the task altogether.

Two ways to detect inconsistent math challenges:

  1. check that all numeric values in the query are also present
  2. each math task context specifies all essential keywords (including numbers) that must be present

steffencruz avatar Feb 24 '24 17:02 steffencruz

@bkb2135 is this now implemented?

steffencruz avatar Apr 18 '24 19:04 steffencruz

Not currently, I thought that this would be solved when we implement the pre-generated paraphrase protocol, but I haven't gotten around to that yet. It's definitely on my list though.

bkb2135 avatar Apr 18 '24 20:04 bkb2135