osf.io icon indicating copy to clipboard operation
osf.io copied to clipboard

[ENG-6266][ENG-6241] send monthly usage to trove as supplementary metadata

Open aaxelb opened this issue 4 months ago • 0 comments

Purpose

add monthly view counts to metadata indexed for search

Changes

  • add monthly report(er) to compute views/downloads in a COUNTER-compliant way (from #10760)
  • support sending metadata to SHARE in multiple parts with different query params (is_supplementary and expiration_date; see https://github.com/CenterForOpenScience/SHARE/pull/825 and https://github.com/CenterForOpenScience/SHARE/pull/826 )
  • partition each item's osfmap metadata (in osf.metadata.osf_gathering) into three parts that are handled differently:
    • OsfmapPartition.MAIN
      • the same old descriptive metadata record we know and love
      • sent to SHARE without is_supplementary nor expiration_date
    • OsfmapPartition.SUPPLEMENT
      • includes metadata less about the information content of the resource and more about how it's handled on osf (currently empty; will include stuff like addons used and osfstorage region)
      • sent to SHARE with is_supplementary but without expiration_date
    • OsfmapPartition.MONTHLY_SUPPLEMENT
      • includes monthly view/download counts
      • sent to SHARE with is_supplementary and an expiration_date two months after the reporting period

QA Notes

Please make verification statements inspired by your code and what your code touches.

  • Verify search indexing still works
  • Verify last month's view/download counts show in (dashboards powered by) the search api shortly after the monthly reporter completes

What are the areas of risk?

Any concerns/considerations/questions that development raised?

Documentation

Side Effects

Ticket

ENG-6266

aaxelb avatar Oct 01 '24 17:10 aaxelb