Package varwidth Warning: “Failed to reprocess entire contents” if `hyperref` is loaded

[dbitouze]

The following MCE:

\documentclass{article}
\usepackage{varwidth}
% \usepackage{hyperref}
\begin{document}
\begin{varwidth}[t]{.5\linewidth}
\section{A Section}\label{sec:section}
Cf. section \ref{sec:section}.
\end{varwidth}
\end{document}

is okay but, as soon as hyperref is loaded, it triggers (twice) the following warning:

Package varwidth Warning: Failed to reprocess entire contents on input line 8.