Python errors in benchmark toolset

[bdw429s]

Please see the discussion in this thread:

https://github.com/TechEmpower/FrameworkBenchmarks/discussions/6338

It contains the details, steps to reproduce, and findings in the CSV file which seem to be causing the error.

I stopped getting any sort of replies from the community a month ago, so I'm creating a ticket in hopes of some traction.

[NateBrady23]

@bdw429s It's unlikely we're going to be able to address this atm. The new toolset is almost ready for release, but we are extremely busy and have very low bandwidth.

[bdw429s]

@nbrady-techempower Thanks for the reply. To be clear, I mentioned the 1 month time frame not to point out that it wasn't fixed yet, but simply that no one was even replying. It honestly only takes about 15 seconds for someone to reply and say, "Yep, look like a bug, please enter a ticket". You have an extremely low-volume discussion platform, but no one could spare that 15 seconds over the last 29 days.

[NateBrady23]

@bdw429s I didn't get an alert for it. The discussions tab is fairly new. I've never actually gotten a proper alert from it even when pinged. Your best bet for a quick reply is opening an issue.

[bdw429s]

Noted. I originally started on your Google group, but was told to use Github discussions instead. Some communities yell at you if you put in tickets which are actually discussions, lol.

[NateBrady23]

I don't really mind either way. I'm just not yet in the habit of checking discussions without a notification, especially with the current workload. I try to get in PR reviews daily and issue notifications usually jump out.

[gwbrown]

I'm running into this issue as well, I managed to work around it by just skipping the column on IndexError. I have no idea what the real consequences of this are (hence why I'm not submitting this as a PR), but it works well enough to let me visualize the results of my benchmark runs.

Here's my diff if anyone wants it, it shouldn't matter for a snippet this tiny but just in case I hereby license this CC0 so do whatever you want with it.

diff --git a/toolset/utils/results.py b/toolset/utils/results.py
index 8e3e723af..f64863c27 100644
--- a/toolset/utils/results.py
+++ b/toolset/utils/results.py
@@ -468,8 +468,12 @@ class Results:
                         row_dict[nextheader] = dict()
                 header = ""
                 for item_num, column in enumerate(row):
-                    if len(main_header[item_num]) != 0:
-                        header = main_header[item_num]
+                    try:
+                        if len(main_header[item_num]) != 0:
+                            header = main_header[item_num]
+                    except IndexError as e:
+                        log("failed to read column header")
+                        continue
                     # all the stats are numbers, so we want to make sure that they stay that way in json
                     row_dict[header][sub_header[item_num]] = float(column)
                 stats_dict[time] = row_dict