feat: pinned reference integrity check API

[notanatol]

Checklist

• [X] I have read the coding guide.

Description

This is a new debug API endpoint that performs the same validation as the pin integrity check command line tool: https://hackmd.io/@F-mGf2lBSoGylLGQE6rRkA/ByVp6Ac9p

To trigger chunk validation for all pins: send an http get request to /check/pin, if you want to limit to a specific reference add a query parameter:

curl -v -XGET "localhost:1635/check/pin?ref=0874f94e9009628a70247c7b77f9034c8729c38d153a9fa42058bf5c7b341495"

The data will be flushed periodically and is .jq friendly: jq 'select(.invalid>0) | .ref' out.json

The background job is responsive to interruptions if you want to stop the pin validation process.

Because the validation process is disk bound it's not desirable to trigger it on a busy instance (while downloading/uploading).

Closes: #4566

[notanatol]

@ldeffenb updated the description: https://hackmd.io/@F-mGf2lBSoGylLGQE6rRkA/ByVp6Ac9p

[ldeffenb]

@ldeffenb updated the description: https://hackmd.io/@F-mGf2lBSoGylLGQE6rRkA/ByVp6Ac9p

Thank you. That one is readable by the general public!