Feature request: opt-in for `MetadataProvider` failovers

[jamesbraza]

Tonight, Semantic Scholar was having some downtime, which meant I couldn't build an index.

It would be nice to have some opt-in flag to allow discarding failures from MetadataProviders if there are 2+ providers that play the same role (e.g. Crossref + Semantic Scholar).

[mskarlin]

This should be the current behavior -- unless you've requested a field that comes specifically from S2 (shouldn't be the case for the defaults). Were you seeing an uncaught exception bubble up from S2 and was that why? Otherwise there would only be an issue if a paper was missing from S2 and crossref.

[jamesbraza]

Yeah I didn't record the stack, but somewhere within SemanticScholarProvider._query was getting 500 or 504's, eventually blew through retries, and then crashed get_directory_index