DataPackageView does not re-render when data package is synced

[robyngit]

trafficstars

Describe the bug

There is an intermittent issue where the dataset landing page sometimes loads without the expected data objects. A cmd + refresh often resolves the issue, showing the data objects again. The issue seems to occur more frequently when the user is logged in. Also, console logs indicate some calls are returning a 404 error.

To Reproduce

1. Navigate to https://search.dataone.org/view/10.24431_rw1k8f7_20240731T162106Z
2. Sometimes the data objects are not displayed.
3. Perform a shift + refresh, and the data objects appear.

Desktop:

Reproduced in both:

• Chrome: v127.0.6533.122
• Firefox: v129.0.2 (64-bit)

[AdrienneCanino]

I am seeing this problem as well on the Research Workspace member Node. But a refresh does not resolve the issue for me. Seeing here, https://doi.org/10.24431/rw1k8fz, where version: 10.24431_rw1k8fz_20241021T144253Z , and a data object of 3 csv files are expected. Chrome: Version 131.0.6778.69 (Official Build) (64-bit) Firefox: Version 132.0 (64-bit)

[mbjones]

@AdrienneCanino is it intermittent, or delayed? Its not unusual to have a delay from when you submit a new version until when it gets harvested (which is on a round robin schedule), and between when it is harvested to when it is indexed for search. Only at that point will all of the objects show up. We have been working on speeding up our indexing process, but for now the service sometimes gets backlogged with a bunch of requests. @iannesbitt can you please check and see if that is the issue for these datasets objects (to see if they are scheduled but not yet indexed)?

[robyngit]

This looks like an issue with the underlying dataset, or indexing, rather than the "Intermittent missing data objects" bug.

[taojing2002]

"https://search.dataone.org/view/10.24431_rw1k8f7_20240731T162106Z" It works fine to me.

@AdrienneCanino Based on your resource map, this package only has one metadata and data object: https://doi.org/10.24431/rw1k8fz

[AdrienneCanino]

Sorry, https://search.dataone.org/view/10.24431_rw1k8f7_20240731T162106Z is not the problem I am reporting.

I am reporting a problem with this dataset: https://search.dataone.org/view/10.24431/rw1k8fz

Which to answer mbjones's question, it isn't delayed. It consistently doesn't show up. Which now I know more about, because taojing2002 is explaining the resource map doesn't include the files I expect. Thanks for another clue in the puzzle, taojing! Also thanks to robyngit! The indexing was forced on D1 side, so I don't think that's it.

I was directed to this issue, but it seems like my problem is not related to this one after all, thanks all for the help investigating!

[taojing2002]

You are welcome, Adrienne! If you need further help, please let us know.

On Mon, Nov 18, 2024 at 12:09 PM AdrienneCanino @.***> wrote:

Sorry, https://search.dataone.org/view/10.24431_rw1k8f7_20240731T162106Z is not the problem I am reporting.

I am reporting a problem with this dataset: https://search.dataone.org/view/10.24431/rw1k8fz

Which to answer mbjones's question, it isn't delayed. It consistently doesn't show up. Which now I know more about, because taojing2002 is explaining the resource map doesn't include the files I expect. Thanks for another clue in the puzzle, taojing! Also thanks to robyngit! The indexing was forced on D1 side, so I don't think that's it.

I was directed to this issue, but it seems like my problem is not related to this one after all, thanks all for the help investigating!

— Reply to this email directly, view it on GitHub https://github.com/NCEAS/metacatui/issues/2508#issuecomment-2484021442, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB5QQDACSSPNB2VTRX477GL2BJCRBAVCNFSM6AAAAABNSAMURKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBUGAZDCNBUGI . You are receiving this because you commented.Message ID: @.***>

[rushirajnenuji]

from slack, by @mbjones

@Robyn@Rushiraj We were looking at the Ofir Levy dataset today, and I noticed that the file hierarchy is showing up on DataONE (https://search.dataone.org/view/doi:10.5063/F1Z899CZ) but it is missing all the files on the KNB pages (https://knb.ecoinformatics.org/view/doi:10.5063/F1Z899CZ). Any ideas why? Can you fix the configuration, and can we figure out how to keep this from happening?

[robyngit]

Renamed this issue form the former Intermittent missing data objects on dataset landing page because I found the root of the issue, which is causing a similar problem: When multiple resource maps exist for the same metadata document, MetacatUI sometimes displays an obsolete RM even though the correct (latest) one is found.

This is the general chain of events:

• MetadataView calls insertPackageDetails() once some solr results are returned
• insertPackageDetails calls getDataPackage() for the first RM in the list if there are multiple
• Then insertPackageDetails looks for the latest RM and re-calls getDataPackage() with the latest pid if it's different from what's already been rendered.

However, the DataPackageView does not get re-rendered on updates. This is the same cause of the empty-data-table problem: the table sometimes renders before anything has synced, leading to an empty table.

Adding a simple this.listenTo(this.dataPackage, "sync", this.render); listener to the render() method of the DataPackageView seems to fix the problem. Before we implement this, we should do a quick audit of the listeners that are already set in the DataPackageView.

A note that the table title (which shows the resource map id) is set in MetadataView, and does not update even when the DataPackageView re-renders. That logic should be moved into the table view itself so the title stays consistent with the rendered package.

The issue was first observed in package https://sfwmd.dataone.org/view/urn:uuid:82fffaa8-4392-4c64-a6f9-531ed7561dec, which has two resource maps: one that obsoletes the other. The older one is rendered and the table never updates.

Once we have the ResourceMapResolver complete in issue #2678, then we can use it in the MetadataView to help get the correct resource map from the start. However, we still need to make sure that the DataTable re-renders on updates.

[robyngit]

Possibly related to: https://github.com/NCEAS/metacatui/issues/948