support provenance links to other packages

[mbjones]

trafficstars

The current provenance editor in MetacatUI provides the ability to pick items in the current package for source and derivation links, as well as scripts. The original mockups included a text box in the modal to allow users to also provide an identifier to an object in another package. We need that functionality. It would be even better if it did a dynamic validation to ensure that the identifier exists in DataONE.

Another frequent request is to point provenance traces at objects that are on the web but are not part of DataONE. This could be done either 1) to a DOI outside of DataONE, or 2) to a persistent web location outside of DataONE. These would also be good to support, but are more fragile because of the ephemeral nature of web links.

So, let's discuss and decide which of the following to support:

• [ ] Provenance links to identifiers in other packages in DataONE
• [ ] Provenance links to persistent identifiers (DOI, Arks) outside of DataONE
• [ ] Provenance links to other URIs outside of DataONE

[gothub]

Related issue: https://github.com/NCEAS/metacatui/issues/359

[laurenwalker]

Provenance links to identifiers in other packages in DataONE

This should be straightforward to add to the prov editor.

Provenance links to persistent identifiers (DOI, Arks) outside of DataONE Provenance links to other URIs outside of DataONE

These will require some testing to make sure indexing and prov chart display works correctly. The prov editor code wouldn't need many changes, but the display likely will, since it assumes everything is a DataONE object.

[laijasmine]

Noting that this is still needed. We can add this to data packages behind the scenes but will be helpful to also allow users to see the provenance links from the landing page.

[jeanetteclark]

It would also be helpful for users to be able to add links to other packages themselves. In the training we are running, a participant expressed frustration that in publishing synthesis datasets, they have found it difficult to appropriately credit the source datasets, and as a result have actually stopped publishing synthesis datasets. I'm not sure what the solution here is exactly, but this issue keeps coming up in discussion with researchers. Even if the prov relationships showed up as just a list indicating that packages are linked I think that would be an improvement.

[mbjones]

Thanks, I agree this is an issue, and one we've discussed in the past and had indicated we should prioritize. I am going to bump the priority on this to High, and we can discuss where it fits in the pantheon of features when we plan our next phase of releases for this year.

[laurenwalker]

This issue has already been marked high priority and was requested again today by one of our hosted repos, OPC. I am going to move this to the Next column and look into getting it worked in in a near future release.

[amoeba]

@jeanetteclark built a package with package-to-package prov (isDerivedFrom). See (requires login) https://arcticdata.io/catalog/view/urn:uuid:38bf9407-3dcb-4251-9d57-3d84c893cbb8 which will be updated with a DOI soon and made public. Should make a good test case for any UI we build.

[yvanlebras]

Amazing functionality! Any possibility that this provenance can be recorded in the EML file?

[amoeba]

Hi @yvanlebras, recording provenance in the EML is an option. To be clear, this ticket is about displaying provenance from DataONE Data Packages and it's not implemented yet. Including structured provenance info in EML is technically possible using EML 2.2.0 or higher and with Semantic Annotations and there are a variety of prov-adjacent elements in EML such as usageCitation. Feel free to touch base on Slack or in another issue if you have any questions.

[yvanlebras]

Thank you Bryce for this rapid feedback! Totally agree!

[jeanetteclark]

commenting here with another issue found. the description doesn't match the unit for nanomolePerKilogram here:

https://github.com/NCEAS/eml/blob/fe77f8f9a34b08bc181857d1ad1240bcd99bead4/eml-unitDictionary.xml#L2029

  <unit id="nanomolePerKilogram" name="nanomolePerKilogram" unitType="" parentSI="molePerKilogram"
    multiplierToSI="0.000000001" abbreviation="nmol/kg" udunitsSynonym="nanomole/kilogram">
    <description>micromoles per kilogram</description>
  </unit>

[iannesbitt]

The CIB hosted repository has also expressed interest in the ability to add external provenance relationships to datasets.