synda
synda copied to clipboard
Take local filename from source URL instead of "title" field
During a discussion on the slack channel about replica files appearing on other nodes with invalid .nc_[0-9]
suffixes [1] it became apparent that Synda appears to use the title
field from the corresponding Solr document as the local filename (see code at [2]). However, it turns out that field is not actually a filename but rather a unique ID, which while it may sometimes be the same as the source filename it doesn't have to be. Given that and also the raison d'être of Synda is to replicate a dataset (in DRS compliant layout) from another node, it would seem a better idea to take the local filename from the source URL. Alternatively, the publisher would need to add another field to the Solr document with the actual filename which then Synda could rely on.
[1] https://esgf-chat.slack.com/archives/C803C3KSQ/p1560478499103600 [2] https://github.com/Prodiguer/synda/blob/sdt/3.9/sdt/bin/sdprepare_file_attr.py#L88