Scott Yeager

Hi @iwanbk, this is a node running on mainnet, so there's no chance of debug logging or running any commands via SSH. We might be able to ask the farmer...

This appears to have resolved itself for the case at hand, but I still think it's worth looking into how we can prevent it from happening again.

Just following up here. The issue is back for the farmer who originally reported it. I'm trying to collect a view of the node's console to see if the IP...

Here's another look at some of the data, this time for a bit more of a "real world" test. I'm also executing both ping and SSH probes against two VMs...

Thanks for the notes @LeeSmet. The challenges with tcp over tcp certainly make sense. That's also good to know about the throughput testing—I'll keep an eye out for any possible...

The thing is that the user never tried to attach a GPU to the deployed VM. We saw another example of this recently, and I was able to reproduce with...

> The timestamps of your logs also a bit weird. That's the default view when using Grafana to query logs from Loki. It feels more natural in a web browser...

Let's do the following to collect info that will be helpful for debugging this: * Enable debug logging for local mycelium (add `-d` in the systemd service) * At the...

> Can't think even how you were able to spot this one. I'll spare the details, but let's just say it was a productive coincidence :slightly_smiling_face: > * We only...

> Note that we didn't `lose` any uptime unless the node dies before it send the next one because once the (next) uptime is sent it will include the full...