taxonworks
taxonworks copied to clipboard
SpeciesFileGroup
decimal longitude test via DwC Occurrence Importer
Feature or enhancement
A decimalLatitude value outside the possible value range does not cause the importer to fail, nor does it generate any sort of error message once it's in TaxonWorks (via the DwC Importer).
See TDWG BDQ test VALIDATION_DECIMALLONGITUDE_INRANGE
Scenario.
- Values in decimalLongitude to be considered mapped to dwc:decimalLongitude (NOT verbatim)
- For "verbatimCoordinates" we need to add a field to the importer that maps directly to this concept
- For any decimalLongitude provided in the Occurrence the value would be in the expected range == [-180 to 180]
- IF the value is out of expected bounds, the record does not Import, and says Not Ready and the
- error message is something like: Impossible coordinate value. Longitudes must be in the range (-180 to 180)
- Would be great to have soft validation for such issues (maybe an out-of-bounds filter option)?
Did this test on
- sandworm
- project: retest clone CVT
- see dataset called test CE EventID Namespace generated by TW 631 records.
- using Firefox
Scope.
- same type of test needed for decimalLatitude, see VALIDATION_DECIMALLATITUDE_INRANGE
- also same in Importer, need decimalLatitude to be considered mapping to dwc:decimalLatitude.
- IF people are really providing verbatimCoordinates we need a column for that, which we don't adjust or change.
Location
DwC Importer for Occurrence on sandworm
Screenshot, napkin sketch of interface, or conceptual description
No response
Your role
Community Liaison
So questions we want to ask are:
- How do I as a user of TaxonWorks find latitude or longitude values outside expected ranges?
- Once they are findable ... then we can fix ... this seems possibility for Stepwise task.
- How do we in software make it so that on import or on create a record, this is impossible?
- Would it be hard to add a script that evaluates the data on export to flag this type of issue before sending to aggregator / publisher, or another colleague, for example.
In the same manner, in Task = New Collecting Event, I can put values out-of-range in Verbatim and click on "Georeference from Verbatim" and it creates / allows this? I'm not sure why that would be possible?
When I click "Save" then validation runs and I see the following.
- reference to 0.0 in the top of the record
- reference to a longitude 178.500000 -- that I have no idea where it comes from
Is https://github.com/SpeciesFileGroup/taxonworks/issues/2097 related?
Not quite sure what you mean mapping to dwc:decimalLatitude/Longitude, we don't have such fields (besides letting the importer set up as DataAttribute automatically which already does if you have in your project those set up), closest would be to map as Georeference and leave existing import code for dwc:verbatimLatitude/Longitude instead. But please see the other issue.
About filters I don't remember if we already have a search for specific soft validations problems? I suspect we do not, I don't remember any caching fields for that. Not any good ideas of how to extend filters to make searching for out of range coords possible without being too specific.
Is #2097 related?
Not quite sure what you mean mapping to dwc:decimalLatitude/Longitude, we don't have such fields (besides letting the importer set up as DataAttribute automatically which already does if you have in your project those set up), closest would be to map as Georeference and leave existing import code for dwc:verbatimLatitude/Longitude instead. But please see the other issue.
About filters I don't remember if we already have a search for specific soft validations problems? I suspect we do not, I don't remember any caching fields for that. Not any good ideas of how to extend filters to make searching for out of range coords possible without being too specific.
@LocoDelAssembly perhaps best to discuss on next Wednesday. There are several issues and I'm not sure where to start. The quest here is how to prevent these types of errors ... and if we can't prevent them ... we need ways to help folks discover and fix.
@jlpereira We are using the verbatim values for display in 2 places, the preview, and in the list of the georeferences stubbed in the modal, in pre save. Perhaps we need to somehow indicate this with a flag. Post-save we could update the values to reflect the encoding of the Georeference.
Oh, I see this is 2 issues, not one. Please split this into individual issues @debpaul .