Add total ballots cast pop up validation during tabulation

[chughes297]

trafficstars

Request

Have a pop up validation question asking if the expected total number of ballots cast = ##### before running the full tabulation.

Implementation probably looks like having this popup appear after all CVRs are parsed but before the round-by-round count proceeds.

This allows admins to check ballot totals before they get round-by-round results, which could help avoid issues like this.

Implementation

• This is initiated by clicking "Tabulate" in the RCTab menu
• If validation fails, this popup will never show and focus returns to the main window that will have the exception in the log section.

Before tabulate is clicked

• Immediately, we start reading CVRs in order to fill in "Number of Ballots" directly under "Number of CVRs". Tabulate button will be greyed out until this is completed. Since this could take a couple seconds, there should be some moving visual indicator near "Number of Ballots" section that work is happening. Or just the moving progress bar with log text at the bottom of the popup saying Reading CVRs (X of Y) where Y=total number of configured CVRs and X=number of those CVRs that have been parsed through. Once the work is complete, that indicator will be replaced with the number.
• Operator name will be red until any text is put in there. At which point it will be normal background
• Once config has been saved, operator name is put in, and Number of Ballots has been calculated then tabulate button will be clickable
• Progress bar is empty or even hidden

After tabulate is clicked

• Tabulate button is disabled to prevent another click
• Progress bar is moving and text shows Tabulating Round by Round Tallies.

After tabulation is complete

• Tabulate button turns into Open Results Folder
• Progress bar is hidden
• If successful, logging text shows "Tabulation successful in [duration]"
• If unsuccessful, shows "Tabulation unsuccessful. See main window for details."

Open Questions // TODOs

• As of 4/16 when file picker is canceled while saving config the popup is hidden behind the main window until the main window is clicked
• When tabulation is in progress and this pop up modal is closed everything continues, just without the modal. I don't think we need to make tabulation cancelable, but we should think through the implications of this. What should happen when we are reading the CVRs to get total number of ballots and someone closes the modal?
• We don't call out explicitly if we are tabulating by precinct or batch. For now it is lumped under the Round by Round Tallies section. Do we need to separate this and call it out explicitly?

[yezr]

Confirming what we expect the workflow to be here.

1. User configures contest
2. User clicks 'Tabulate'
3. Before Round 1 tally, popup shows the number of ballots cast based on the configured CVR files. It would say

Contest as configured with X CVR targets has Y total ballots cast. Proceed? [Yes/No]

4. Selecting Yes continues as usual, selecting No halts tabulation.

X should be the number of rows configured in the CVR Files tab. Since they can technically be both folders (with a bunch of CVR files within) and explicit files, I changed the language to "CVR Targets." Eventually, #675 will have ALL that specific information elsewhere.

Y should be the same as the Total Number of Ballots in the summary.csv

[yezr]

@chughes297 I changed the text to "CVR Targets" what do you think? Since it can be both individual files AND a folder with N actual CVR files within.

[artoonie]

I think it's worth thinking about a larger confirmation page here. We already ask for the user name before tabulation, ask them to re-save any changes, and now this -- it would create, potentially, four sequential popups.

Perhaps when the user hits "Tabulate," we instead pop up a single confirmation page.

Here's a quick sketch of what I'm thinking:

This is nice because it presents all the things you need to do to begin tabulation in one place, and hopefully for developers or casual users, it can make an easy way to circumvent all of these changes at once -- the bottom row can have a "Skip Checks & Force Tabulation" button for certain files (e.g. files with version set to "TEST").

[yezr]

I like it. A couple questions

• What would it say in Configuration if we are starting from scratch? No File Saved?
• Where do we get the file name? Is that from a previous saved config or if we loaded a config?
• What do we do when configuration file name overflows? Give it a ... at the end?
• The tabulate button would be greyed out until all numbers are confirmed, config is saved, and name is entered? What if it takes a while to generate those numbers?

And I think "Number of Voters" should be "Number of Ballots"

[artoonie]

• No file saved seems right
• Yes, have the file name either previously saved or as loaded, that's how we can check if any modifications have been made
• Probably a ... at the start so you can see the end of the filename? maybe even implement a scrollable area
• Yes, tabulate button greyed out
• Number of ballots is the only item it may take a while to generate, I think. Perhaps, as such, we combine this with the Progress Bar Issue #133, so the progress bar shows both before loading this, and after hitting Tabulate.
• Agreed, Number of Ballots. Also, the "Number of Candidates" and "Number of CVRs" is just for demonstration of what we can do, and I'll only implement if requested.

[yezr]

For a progress bar are we talking about inside of that popup somewhere? Like at the top or bottom? Or maybe even within the "Number of Ballots" header since that is the only thing I think that would take time?

And we would set the % complete to be the number of configured CVR targets that have been checked for ballot count?

[artoonie]

New thoughts:

Left: before/during tabulation Right: after tabulation

the bottom rectangle is a progress bar. Underneath the progress bar is an explainer of the current step (during tabulation) or the total ballots (after tabulation).

Progress bar will have two phases:

1. Reading CVRs (% complete = % of CVRs read)
2. Tabulating (% complete = % of candidates with a "decision" made, elected/eliminated)

[artoonie]

Would the pop-up above satisfy the needs of #674? This feels cleaner than adding more info to the tab, but I'm fine doing both.

[yezr]

Would the pop-up above satisfy the needs of #674? This feels cleaner than adding more info to the tab, but I'm fine doing both.

For users who don't know that this information is going to eventually show up in this pop-up, I think that having the CVR count show in the "CVR Files" tab somewhere would be still be helpful. They might need to see and confirm that before they feel comfortable hitting Tabulate.

[yezr]

I like where this popup is headed!!! Confirming some things

• This is initiated by clicking "Tabulate" in the RCTab menu
• If validation fails, this popup will close (or never show) and focus returns to the main window that will have the exception in the log section.

Before tabulate is clicked

• For configuration, what happens once it is saved? The red 'Save Modified File' button is gone and it's just the path of the file? Still thinking about how loooong file paths are handled...
• Can "Number of Ballots" show up directly under "Number of CVRs". Before tabulation, when the ballot count is calculated, can it say something like "Number of Ballots: calculated during tabulation" and when complete it will be replaced with "Number of Ballots: 1,000,000"
• Operator name will be red until any text is put in there. At which point it will be normal background
• Once config has been saved and operator name is put in then tabulate button will be clickable
• Progress bar is empty or even hidden

After tabulate is clicked

• Tabulate button is disable to prevent another click
• First step is Reading CVRs (X of Y) where Y=total number of configured CVRs and X=number of those CVRs that have been parsed through. Progress bar shows X/Y. Number of Ballots count is updated in the popup once all CVRs have been parsed.
• Once first step is complete, second step is Tabulating Round by Round Tallies. Progress bar shows (Count of Candidates Elected/Eliminated)/(Total Count of Candidates)

After tabulation is complete

• Tabulate button turns into Open Results Folder
• Progress bar is full or even hidden
• If successful, shows "Tabulation successful in [duration]"
• If unsuccessful, shows "Tabulation unsuccessful. See main window for details."

[artoonie]

Would the pop-up above satisfy the needs of #674? This feels cleaner than adding more info to the tab, but I'm fine doing both.

For users who don't know that this information is going to eventually show up in this pop-up, I think that having the CVR count show in the "CVR Files" tab somewhere would be still be helpful. They might need to see and confirm that before they feel comfortable hitting Tabulate.

Continuing this conversation in #674

[artoonie]

I like where this popup is headed!!! Confirming some things

• This is initiated by clicking "Tabulate" in the RCTab menu

Correct

• If validation fails, this popup will close (or never show) and focus returns to the main window that will have the exception in the log section.

I haven't given this too much thought, but I expect it will stay open, show a brief error message, and refer users to the log to see more.

Before tabulate is clicked

• For configuration, what happens once it is saved? The red 'Save Modified File' button is gone and it's just the path of the file?

Still thinking about how loooong file paths are handled... I think the button will be grayed out and it will save "Saved" or "Up to date". Long filenames will probably be handled via side-scroll, though it depends on what javafx components are available and easy to use.

• Can "Number of Ballots" show up directly under "Number of CVRs". Before tabulation, when the ballot count is calculated, can it say something like "Number of Ballots: calculated during tabulation" and when complete it will be replaced with "Number of Ballots: 1,000,000"

Yes, though I'm not sure the UX there is as intuitive. Once people feel that they "completed" the top section, they may be less likely to look back at it. To me, it makes more visual sense to have it at the bottom, along with a done message, like "Successfully tabulated N ballots". Either way is easy enough, and we can easily change from one to the other after we feel it out.

• Operator name will be red until any text is put in there. At which point it will be normal background

Correct

• Once config has been saved and operator name is put in then tabulate button will be clickable

Correct

• Progress bar is empty or even hidden

Correct

After tabulate is clicked

• Tabulate button is disable to prevent another click

Correct

• First step is Reading CVRs (X of Y) where Y=total number of configured CVRs and X=number of those CVRs that have been parsed through. Progress bar shows X/Y. Number of Ballots count is updated in the popup once all CVRs have been parsed.

Yes, except:

• Progress bar may show (X/Y) * N, where N = the estimated percent of time spent reading CVRs versus tabulating (~90%?).
• Number of ballots might be better to show after tabulation is complete, rather than after CVRs read, just for UX reasons (esp. with the text "Successfully tabulated __ ballots")

• Once first step is complete, second step is Tabulating Round by Round Tallies. Progress bar shows (Count of Candidates Elected/Eliminated)/(Total Count of Candidates)

Yes, or (if following the bullet point two above this one), instead of Step 2 going from 0-100 it will just go from (e.g.) 90-100: N + #Decided/#TotalCandidates * (1 - N)

After tabulation is complete

• Tabulate button turns into Open Results Folder

Correct

• Progress bar is full or even hidden

Correct

• If successful, shows "Tabulation successful in [duration]"

Correct, or "Successfully tabulated N ballots" (duration is in the audit log if they care)

• If unsuccessful, shows "Tabulation unsuccessful. See main window for details."

Something like this, yes

[artoonie]

Another note: I suspect that Tabulate By X may be a Step 3/3 here.

[artoonie]

Somehow JavaFX's interface makes everything look 20 years old. Here's my mockup.

[yezr]

@artoonie how much work is it to show a "Number of Ballots: Calculating..." line under Number of CVRs in the "Configuration Summary" section that would fire off a thread to read the CVRs BEFORE tabulation. Once that thread has read through the CVRs to get a total ballot count it would show "Number of Ballots: XXX". On top of that, as you said, once tabulation is finished it would say something like "N total ballots tabulated" under the progress bar.

Testing timing it took ~20 seconds for the 1,050,000 ballots of the Minneapolis Mayor scale race. I would like to give the user the option of waiting for that before tabulating (to avoid a misconfiguration during a results livestream for example) but also provide less patient users the ability to tabulate immediately. Tabulating immediately you would still be shown clearly the ballot count in this new modal with the "N total ballots tabulated" under the progress bar.

If a user kicks off Tabulate while that task is running it would automatically kill it before running the actual tabulation. Any concerns about memory issues with running that on its own thread or race conditions if someone decides to just hit tabulate before it is complete?

[artoonie]

Technically I don’t think this would be difficult. There may be other concerns, like:

1. What goes in the audit log?
2. Do we re-read the CVRs after they hit tabulate?
3. If so, what if a CVR changes — would the UI show the wrong number or update to show the correct number?
4. Is there a benefit to doing this automatically versus having a button that says “Count CVRs without tabulating” or similar?

On Thu, Apr 11, 2024 at 4:48 PM Mathew Ruberg @.***> wrote:

@artoonie https://github.com/artoonie how much work is it to show a "Number of Ballots: Calculating..." line under Number of CVRs in the "Configuration Summary" section that would fire off a thread to read the CVRs BEFORE tabulation. Once that thread has read through the CVRs to get a total ballot count it would show "Number of Ballots: XXX". On top of that, as you said, once tabulation is finished it would say something like "N total ballots tabulated" under the progress bar.

If a user kicks off Tabulate while that task is running it would automatically kill it before running the actual tabulation. Any concerns about memory issues with running that on its own thread or race conditions if someone decides to just hit tabulate before it is complete?

— Reply to this email directly, view it on GitHub https://github.com/BrightSpots/rcv/issues/679#issuecomment-2050512957, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEDFZC6GE26VNRQ3AOHU73Y43ZLPAVCNFSM6AAAAAAYUGWLFSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANJQGUYTEOJVG4 . You are receiving this because you were mentioned.Message ID: @.***>

[yezr]

I added an Implementation header to the original comment with a summary of the current state of this request. Since we've had a bit of back and forth I wanted to try to capture the latest.

@artoonie The way I wrote implementation the first step, reading CVRs to calculate and display the total number of ballots, happens automatically. You discussed a proposal with distinct buttons for the distinct steps. Either way will satisfy our requirements. Can you either provide a brief writeup on how distinct buttons will work or if it is easy enough just code it up so we can take a look?

[yezr]

Created a followup ticket #825 for a more granular progress bar once we are done with initial implementation. For this ticket, the bouncing progress bar while working is enough.

[artoonie]

This was easy enough to implement:

Here's a downside I've discovered in the process: The current way that Tabulation works, it's a little hard to avoid re-reading all CVRs during the tabulation step. Not impossible, but should probably be part of another ticket.

The reason is that multi-step IRV re-reads the CVRs each time, adding an excluded candidate at each round. We'd have to change the way multi-step IRV works to support this.

Options:

1. Try to load CVRs in the background to calculate the number of ballots. Downside: not clear how to handle the audit log here.
2. "Tabulate" actually does "Read" + "Tabulate"
3. (2) for now, and then we change it in another PR

[HEdingfield]

@artoonie it looks like @chughes297 updated the original comment (see section "Open Questions // TODOs") with some of the stuff I noticed when testing last night. Could you please make sure all of those are addressed before we finalize this?

[artoonie]

Sure thing:

As of 4/16 when file picker is canceled while saving config the popup is hidden behind the main window until the main window is clicked

This is fixed

When tabulation is in progress and this pop up modal is closed everything continues, just without the modal. I don't think we need to make tabulation cancelable, but we should think through the implications of this. What should happen when we are reading the CVRs to get total number of ballots and someone closes the modal?

This is fixed by preventing closing the modal

We don't call out explicitly if we are tabulating by precinct or batch. For now it is lumped under the Round by Round Tallies section. Do we need to separate this and call it out explicitly?

@chughes297 I assume this is referring to the output files? @yezr can we make a ticket to address this?

[yezr]

We don't call out explicitly if we are tabulating by precinct or batch. For now it is lumped under the Round by Round Tallies section. Do we need to separate this and call it out explicitly?

@chughes297 I assume this is referring to the output files? @yezr can we make a ticket to address this?

Created https://github.com/BrightSpots/rcv/issues/836 as a P2 to investigate estimating the time these post-tabulation tasks will take and including that in our progress bar.

As the other bullet points are fixed, I'll remove that "Open Questions // TODOs" section!