osf.io
osf.io copied to clipboard
[ENG-6266][ENG-6241] send monthly usage to trove as supplementary metadata
Purpose
add monthly view counts to metadata indexed for search
Changes
- add monthly report(er) to compute views/downloads in a COUNTER-compliant way (from #10760)
- support sending metadata to SHARE in multiple parts with different query params (
is_supplementary
andexpiration_date
; see https://github.com/CenterForOpenScience/SHARE/pull/825 and https://github.com/CenterForOpenScience/SHARE/pull/826 ) - partition each item's osfmap metadata (in
osf.metadata.osf_gathering
) into three parts that are handled differently:-
OsfmapPartition.MAIN
- the same old descriptive metadata record we know and love
- sent to SHARE without
is_supplementary
norexpiration_date
-
OsfmapPartition.SUPPLEMENT
- includes metadata less about the information content of the resource and more about how it's handled on osf (currently empty; will include stuff like addons used and osfstorage region)
- sent to SHARE with
is_supplementary
but withoutexpiration_date
-
OsfmapPartition.MONTHLY_SUPPLEMENT
- includes monthly view/download counts
- sent to SHARE with
is_supplementary
and anexpiration_date
two months after the reporting period
-
QA Notes
Please make verification statements inspired by your code and what your code touches.
- Verify search indexing still works
- Verify last month's view/download counts show in (dashboards powered by) the search api shortly after the monthly reporter completes
What are the areas of risk?
Any concerns/considerations/questions that development raised?