Libuv is not shut down on pem shutdown.

[sync-by-unito[bot]]

To reproduce the error, deploy a cluster and then log onto a node that has a pem, and then kill the pem:

gcloud compute ssh <node-name>
sudo kill $(pidof pem)

By adding some logging, you can observe that we fall through to the while loop in the Manager::Stop() method, but, running_ is never set to false, i.e. libuv does not exit.

Status Manager::Stop(std::chrono::milliseconds timeout) {
  // Already stopping, protect against multiple calls.
  if (stop_called_) {
    return Status::OK();
  }
  stop_called_ = true;

  dispatcher_->Stop();
  auto s = StopImpl(timeout);

  // Wait for a limited amount of time for main thread to stop processing.
  std::chrono::time_point expiration_time = time_system_->MonotonicTime() + timeout;
  while (running_ && time_system_->MonotonicTime() < expiration_time) {
    std::this_thread::sleep_for(std::chrono::milliseconds{100});
  }

  return s;
}

┆Issue is synchronized with this Jira Bug by Unito