Using padding props causes scroll to jump when clicks happen

[hipstersmoothie]

trafficstars

With padding props:

https://user-images.githubusercontent.com/1192452/210189821-e8b74dab-ca98-4f50-a02f-b59378d67ada.mp4

Without

https://user-images.githubusercontent.com/1192452/210189826-aa13b542-4529-4eca-88d9-efe3178a5fb0.mp4

Bug present here:

https://github.com/hipstersmoothie/fwoosh

Steps to start

yarn
yarn build
cd examples/fwoosh
yarn start