Sean MacAvaney

This is what ir-datasets uses: https://github.com/allenai/ir_datasets/blob/master/ir_datasets/formats/trec.py#L286-L334

The new [pyterrier-quality](https://github.com/terrierteam/pyterrier-quality) repo has an example. The key bits are: - [metadata file creation](https://github.com/terrierteam/pyterrier-quality/blob/main/pyterrier_quality/qual_cache.py#L72-L78) - [entry point](https://github.com/terrierteam/pyterrier-quality/blob/main/setup.py#L38-L40) Some things I'm still considering: - I think we can drop the...

`artifact` branch now on pyterrier-pisa: https://github.com/terrierteam/pyterrier_pisa/tree/artifact

And on pyterrier-dr: https://github.com/terrierteam/pyterrier_dr/tree/artifact

Entry points act as a registry of all the artifacts installed. Since they're metadata about the package itself, they do not involve loading any modules at runtime to establish what's...

A prototype of the artifact API is in [pyterrier-alpha](https://github.com/seanmacavaney/pyterrier-alpha). Integrated with extension packages: - https://github.com/terrierteam/pyterrier_pisa/pull/21 - https://github.com/terrierteam/pyterrier_dr/pull/20 - https://github.com/seanmacavaney/pyterrier-caching/pull/4 Still to integrate: - [ ] [pyterrier-quality](https://github.com/terrierteam/pyterrier-quality) `QualCache` - [x] [pyterrier-adaptive](https://github.com/terrierteam/pyterrier_adaptive)...

Sounds reasonable! The idea would be that it would detect if it was loading a run file (or similar) and return it as a dataframe? We can experiment with this...

For results, it might make more sense to have a special URI-style format for loading with `pt.io.read_results`? E.g., `pt.io.read_results('tirex:///')`?

``` save_mode : SAVEMODE_TYPE = 'error', ``` but the docstring says: ``` Default is "warn". ``` (as does the name of this PR)

Sorry for the silence— i’ve been working on putting together a Round 3 baseline and ability to daily run pre-trained models. Hopefully pushed out on Friday.