shuffleboard Shuffleboard data disappears from widgets

We have been seeing a problem with Shuffleboard that is still happening on 2023.4.2 (robot and SB). What we see is that after a period of time of successful deploys and test runs, you will approach the DS to run a test and SB widgets will be blank, though they had data moments before. Normally, the widgets always display the last data sent. We have to restart SB to get our widgets to display data. This happens one or two times a day. Haven't observed the data disappear, it's just gone the next time you look at it. Have seen this after DS PC has been closed overnight, but not always. Also observe this after a deploy, but no problem on most deploys. When the widgets are blank, SB indicates Network tables are connected. Been trying to define this issue further without success.

This is a scary proposition for competition.

Mar 02 '23 17:03 Crossle86

Can you upload Shuffleboard log files from when this happened? Shuffleboard log files are located at ~/Shuffleboard/.

Mar 19 '23 16:03 Starlight220

I am watching for it do it again, will get the log...hasn't happened in a match but lots of times in the pits and our lab.

Mar 21 '23 21:03 Crossle86

In multiple logs I've seen there are messages that might indicate problems connecting between live NT sources and the sources in a layout file, but I'm not sure if that's this issue.

Mar 21 '23 21:03 Starlight220

We've also seen this behavior, and have had a similarly hard time pinning it down.

Might it be possible to put a button on the interface to hard re-sync connections to Network Tables if this happens? If that could be reliable, it could at least be a workaround for competition until we can figure out why it's happening.

If you think this is a good idea, can you point me to where in the code base to look? I'm happy to try to PR this, but it would be my first time working on this repo, so I'd need a bit of guidance to get started, particularly around code base organization and how best to test such a thing.

Mar 27 '23 03:03 nihonjinrxs

So we continue to have this problem on a regular basis. I have two log files that one of which should cover a failure.

shuffleboard.2023-03-24-18.36.22.log shuffleboard.2023-03-25-08.28.33.log

Mar 27 '23 16:03 Crossle86

Yep, there indeed is an informative error in that first log (haven't looked at the second). Something is throwing a ConcurrentModificationException when trying to reconnect a source.

Mar 27 '23 17:03 Starlight220

Great. Caught those files at a competition so I have not looked at them. Now that I think about what you found, it may well be that this happens after SB loses connection to the robot. I can't say I have seen it after a deploy but I'm sure I see it after connection to robot is lost, though it seems happen some time after the disconnection event. Seems I notice it when returning to SB after robot has been offline and is now back online for operation.

Mar 27 '23 20:03 Crossle86

@Crossle86 is it specific sources that don't display data, or all sources? The relevant exception in the log points to a specific data source (network_table:///Shuffleboard/Drivetrain/Front Right Module/Current Velocity); I'm not sure how a failure in restoring one source would break all sources.

Mar 28 '23 09:03 Starlight220

All sources do not display. The source you mention set very early in our start up. It and a number of other sources are created dynamically in code by our swerve code. The rest of the sources we display are in the SB layout file. Note we are using the NT3 support in the ShuffleBoard class.

Mar 28 '23 16:03 Crossle86

Can you test with the newly-released 2023.4.3? If this wasn't fixed, reopen the issue.

Mar 29 '23 08:03 Starlight220

I believe we have the latest version and are experiencing this issue still. Where can I find the Shuffleboard logs exactly and do I just need to get them when it happens?

Mar 31 '23 03:03 EddieDL

Shuffleboard's logs are located at ~/Shuffleboard/. On Windows this would be C:/Users/<yourusername>/Shuffleboard/. When the issue happens, close shuffleboard and upload the most recent log to here.

Mar 31 '23 08:03 Starlight220

It just happened to me and this is the most recent log file. shuffleboard.2023-03-31-12.20.36.log

Let me know if you need anything more.

Mar 31 '23 16:03 EddieDL

I'm just back from vacation so I have not observed a failure myself but my students updated the SB to the latest version and told me that they have observed the same failure. I don't have a log file at this time but will catch one the next time it fails.

Apr 04 '23 18:04 Crossle86