rcv Support precinct summary file comparison for regression testing

Regression tests need to compare actual tabulation result file to expected result files. Our tabulator "discovers" precinct IDs during the course of tabulation. We would need some way of exposing these IDs to the test harness, and mapping them to a list of known files.

Dec 17 '18 06:12 moldover

Do we need this for certification? It would seem likely to me.

Jun 10 '19 19:06 freedomcounts

Note that this also needs to work for the sequential case. I'm fixing that output logic in #291.

Jun 22 '19 07:06 tarheel

I think this is the same issue I ran into today: precinct tests like precinct_example and missing_precinct_example currently only compare the overall .json summary to a single expected file. They should compare each precinct summary as well (including the missing one in the latter case).

Jul 03 '19 05:07 HEdingfield

Are we still comfortable that this is non-essential? I'm working on #189 and would need this issue to be addressed in order to automate testing of my work.

Jul 07 '19 06:07 tarheel

(Note that we also labeled #189 as non-essential, so we're being consistent.)

Jul 07 '19 06:07 tarheel