Users with hidden diary entries have the hidden entries included within their diary count
A user on the live site has 1 hidden diary entry and no un-hidden ones. Their "user" page shows "Diary 1", but when you click on diary it says "No diary entries".
In a sense what it's saying is correct, but I'd imagine it could be confusing to users.
That's no different to the other counters, and probably hard to fix because counter caches are a bit a hacky.
Yes I just checked and counter caches don't seem to have any way to specify extra conditions - they just count the number of records of one type that are associated with a record of a different type.
So there is no way to limit it to visible records only.
There are a few third party gems that can do it.