Add `iospec`

[cairomassimo]

iospec is a tool to generate parsers for, validate and transform input/output files, given a specification of the format in an expressive language, as well as to generate documentation of the format itself.

Goals

• Support a very large set of I/O formats (virtually for all IOI-style problems),
• By writing the spec just once
  • Generate graders and templates automatically in many programming languages (incl. all those allowed in IOIs)
  • Validate automatically the format for input/output files
  • Validate automatically (a large set of) the assumptions about input/output files
  • Support the creation of I/O generators, validators and checkers, by generating I/O parsers for them automatically
  • Generate a description of the I/O format and assumptions in LaTeX, relying on user-provided macros for the natural-language parts
• Be reasonably user-friendly to use
• Be flexible and future-proof

What's implemented already?

• Generating graders and templates in C and C++
• Validating inputs/outputs against the spec
• A large set of formats with numeric-only data (incl. multi-dimensional array with non-uniform dimensions, branches depending on input values, array dimesions that are arithemtic functions of input values, etc.)
• rustc-like diagnostic messages for most errors when parsing a spec
• a testing framework that compiles and runs generated graders

Proposed new API / contact points for problem developers

• gen/IOSPEC file with description of input formats, assumptions (incl. subtask-specific ones), and solution interface,
• extending the task-maker command (and/or adding one or more new task-maker-tools commands) to generate the following files from gen/IOSPEC (if given):
  • sol/template.{ext} for every configured and supported language,
  • sol/grader.{ext} for every configured and supported language,
  • gen/genlib.hpp, support lib for C++ generators, validators and checkers (see below),
  • statement/lib.tex, TeX macros for statements (see below),
  • statement/messages.{lang}.tex, TeX macros with translatable text (see below),
• adding the following steps in the DAG:
  • validate iospec
    • runs task-maker-tools iospec-check (once),
    • before everything,
    • reports issues in gen/IOSPEC, and stops if any if found,
  • input pre-validation
    • for each test case, runs task-maker-tools iospec-check <input.txt> --cfg subtask_name=<name>,
    • after input generation, before validation (if any),
    • used to check input format and assumptions in gen/IOSPEC,
    • so that actual validator (if any) can ignore the input format and just check more complex assumptions (e.g. "graph is a DAG"),
  • output validation
    • for each test case, runs task-maker-tools iospec-check <input.txt> <output.txt> --cfg subtask_name=<name> on every input/output,
    • after output generation, before testing solutions,
    • used to check output format and assertions in IOSPEC (conditions on the output),
• API provided by genlib.hpp:
  • struct IoData { ... } with public members corresponding to I/O data,
  • GENERATOR_MAIN(function1, function2, ...) macro to use in generators,
    • where function1, function2, etc. accept an arbitrary number of parameters and return IoData,
    • if only one argument function1, parameters are parsed automatically from argv,
    • else, the argv[1] is used to choose which of the functions to call, then as above,
    • can use primitive types as paramaters, as well as a random number generator Rng (API TBD) initialized from a seed,
  • VALIDATOR_MAIN(validator_function) macro to use in validators,
    • where validator_function is a void validator_function(IoData data);
    • return type TBD,
  • or, instead of the above, VALIDATOR_GRADER() macro to use in validators,
    • no arguments,
    • validators should implement the same functions as a solution, and perform their checks there,
  • CHECKER_MAIN(checker_function),
    • where checker_function is a void checker_function(IoData correct, IoData submission);
    • parameters are respectively the correct input/output data, and the input/output data for the submission,
    • note: the input is available as part of both correct and submission,
    • return type TBD,
  • std::string get_subtask_name() (or similar name, TBD) returning the current subtask name
• TeX macros provided by statement/lib.tex (in a future PR):
  • task assumptions (TBD),
  • subtask table with scores and assumptions (TBD),
  • input format specification (TBD),
  • all macros in turn use language-specific macros defined in messages.{lang}.tex (e.g., a language-specific macro like for every #1 used to generate for every $i=0,\dots,N$:)
  • generated macros can be customized with annotations in gen/IOSPEC (TBD)

[veluca93]

High level comment: why don't we leave figuring out of formatting to appropriate language-specific tooling? (i.e. clang-format, autopep8 or whatever)