Common: consider deleting functionality related to S3-discovery

[chuwy]

In #232 we moved entirely to SQS discovery, but left some functionality related to discovering data on S3, mostly in ShreddedType modeul. I think that it will be helpful later to check integrity of the batch (do periodical check for abandoned batches and notify operators about it) and decided to not delete yet.

[benjben]

What do you mean by "checking integrity of the batch" ?

[chuwy]

That for example there are no unexpected folders, like:

run=2021-01-18-23-32-30/vendor=com.acme/name=expected/format=tsv/model=1
run=2021-01-18-23-32-30/something=unexpected_stuff

Or model=A.