Add extended database logging

[roosterfish]

Microcluster takes an interesting approach when it comes to dqlite connections. A cluster member is requesting POST /core/internal/datbase of another cluster member setting the Upgrade: dqlite header in order to upgrade the connection which is then getting passed to dqlite.

As this connection is using mTLS, there is no visibility into the traffic.

To allow for extended database logging and tracing of the underlying traffic, the dqliteNetworkDial function should be extended to enable this type of logging (debug). One option might be to create a new struct which embeds the net.Conn type. This allows implementing both the Read and Write functions to be able to interject what is read and written from and to the connection.

An example implementation:

type logConn struct {
	net.Conn
}

func (l logConn) Read(b []byte) (n int, err error) {
	n, err = l.Conn.Read(b)
	if err != nil {
		return 0, err
	}

	logger.Error("Reading from connection", logger.Ctx{"msg": string(b)})
	return n, nil
}

func (l logConn) Write(b []byte) (n int, err error) {
	logger.Error("Writing to connection", logger.Ctx{"msg": string(b)})
	return l.Conn.Write(b)
}

Special care has to be taken when printing the contents to ignore any special characters and to be able to log the actual contents.

In case this causes too many content to be logged, another option would be to either have an additional flag in the daemon's args which enables this type of logs or adding an environment variable that controls whether or not the daemon is using this extended type of logging.

That was a request coming out of https://github.com/canonical/microcluster/pull/313 in order to have more visibility into the underlying traffic.