bdq
bdq copied to clipboard
tdwg
TG2-AMENDMENT_COORDINATES_FROM_VERBATIM
| Field | Value |
|---|---|
| GUID | 3c2590c7-af8a-4eb4-af57-5f73ba9d1f8e |
| Label | AMENDMENT_COORDINATES_FROM_VERBATIM |
| Description | Propose amendment to the values of dwc:decimalLatitude and dwc:decimalLongitude from information in the verbatim coordinates terms. |
| Output Type | Amendment |
| Darwin Core Class | Location |
| Information Elements | dwc:decimalLatitude |
| dwc:decimalLongitude | |
| dwc:verbatimCoordinates | |
| dwc:verbatimLatitude | |
| dwc:verbatimLongitude | |
| dwc:verbatimCoordinateSystem | |
| dwc:verbatimSRS | |
| Expected Response | INTERNAL_PREREQUISITES_NOT_MET if either dwc:decimalLatitude or dwc:decimalLongitude was not EMPTY, or either dwc:verbatimLatitude and dwc:verbatimLongitude, or dwc:verbatimCoordinates were not unambiguously interpretable into valid coordinates; FILLED_IN the values of dwc:decimalLatitude and dwc:decimalLongitude if unambiguous values can be interpreted from dwc:verbatimCoordinates or dwc:verbatimLatitude and dwc:verbatimLongitude, plus dwc:verbatimCoordinateSystem and dwc:verbatimSRS; otherwise NOT_AMENDED |
| Data Quality Dimension | Completeness |
| Term-Actions | COORDINATES_FROM_VERBATIM |
| Warning Type | Amended |
| Parameter(s) | |
| Source Authority | |
| Examples | [dwc:verbatimLatitude="-23.712", dwc:verbatimLongitude="", dwc:verbatimCoordinates="139.92", dwc:verbatimSRS="EPSG:4326", dwc:verbatimCoordinateSystem="decimal degrees": Response.status=FILLED_IN, Response.result=dwc:decimalLatitude="-23.712", dwc:decimalLongitude="139.923", dwc:geodeticDatum="EPSG:4326", Response.comment="Input fields contain interpretable values"] |
| [dwc:verbatimLatitude="-23.712", dwc:verbatimLongitude="", dwc:verbatimCoordinates="", dwc:verbatimSRS="EPSG:4326", dwc:verbatimCoordinateSystem="decimal degrees", dwc:decimalLatitude="", dwc:decimalLongitude="": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:verbatimLongitude is EMPTY"] | |
| Source | ALA |
| References |
|
| Example Implementations (Mechanisms) | |
| Link to Specification Source Code | |
| Notes |
Do we need to make a prerequisite that either or both dwc:decimalLatitude and dwc:decimalLongitude are EMPTY?
Agreed at TDWG 2018 DQIG meeting that the name TG2-AMENDMENT_COORDINATES_FROM_VERBATIM is satisfactory.
In preparing test data, it would seem that the Expected Response should be
INTERNAL_PREREQUISITES_NOT_MET if Verbatim coordinates (either dwc:verbatimLatitude and dwc:verbatimLongitude or dwc:verbatimCoordinates) were not interpretable into coordinates as decimal degrees or Verbatim coordinates were EMPTY or either dwc:decimalLatitude or dwc:decimalLongitude was not EMPTY; AMENDED if dwc:decimalLatitude and dwc:decimalLongitude were populated from information in verbatim coordinate information (dwc:verbatimCoordinates or dwc:verbatimLatitude and dwc:verbatimLongitude, plus dwc:verbatimCoordinateSystem and dwc:verbatimSRS); otherwise NOT_CHANGED
?
Similar to the countryCode amendment situation. Could we not make an interpretation for the decimal coordinates if they are NOT EMPTY, but also not valid?
On Mon, Nov 1, 2021 at 12:48 AM Arthur Chapman @.***> wrote:
Makes sense to me
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tdwg/bdq/issues/32#issuecomment-955903384, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADQ72YECLZR7LKMPLULDCTUJYE2BANCNFSM4EKSLOSA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
Thanks @tucotuco. One wonders where we draw the line. In the case of #73, I figure it is 'relatively easy' to lookup dwc:countryCode but there must be infinite 'shades of grey' comparing verbatim coordinates (which could be anything) to existing values for dwc:decimalLatitude and dwc:decimalLongitude? Or, like I've suggested in #73, IF we have valid interpretable verbatim coordinates, we overwrite dwc:decimalLatitude and dwc:decimalLongitude? I feel uncomfortable with that strategy.
I'd say VALID dwc:decimalLatitude and dwc:decimalLongitude take precedence over verbatim coordinates, so any differences would tend to deprecate the verbatim's potential to amend.
IF there was additional spatial information as in dwc:country or dwc:countryCode, and the error in dwc:decimalLatitude or dwc:decimalLongitude was 'gross' (e.g., sign swap or obvious degree differences), a fix could be applied.
We are talking AI :)
If you have VALID dwc:decimalLatitude and dwc:decimalLongitude then we don't run this test do we?
I think we need a meeting to sort some of this out.
I am not sure that the AMENDMENTS are entirely stand alone or we wouldn't need Validations. In many (all?) cases we only run an amendment if a VALIDATION fails don't we?
@ArthurChapman : Yes, I am presuming an AMENDMENT will only be run if the VALIDATION reports "NOT_COMPLIANT".
I will continue with the test data for the last 5 AMENDMENTs for SPACE, and then get file out to the group. I've completed the data for AMENDMENTS for NAME, TIME and OTHER. The test data certainly helps to clarify the Expect Responses.
I am not sure that the AMENDMENTS are entirely stand alone or we wouldn't need Validations. In many (all?) cases we only run an amendment if a VALIDATION fails don't we?
I disagree. Again, I think we are confounding the tests themselves with what we expect to be sensible ways to string them together. We shouldn't. The process uses the tests, the tests don't depend on the process. They are separate levels of product.
Agreeing with @chicoreus, the tests must be be able to stand independently.
I think this needs discussing in a forum - we seem to have gone in a circle here in the relation between VALIDATIONS and AMENDMENTS. It has been so long between discussions, I seem to be getting confused as to where we have been and where we are going. We spent a lot of time making sure that AMENDMENTS had corresponding VALIDATIONS. If AMENDMENTS are entirely stand alone - then why do we need many of the VALIDATIONS? We could include the failure state within the AMENDMENT - i.e. reporting more fully why INTERNAL_PREREQUESITES failed - extending that to explain why, could remove the need for the VALIDATION. This would mean a total revisit to the way we do the tests and I don't recommend we go there. I can see lots of places running the VALIDATIONS and not the AMENDMENTS.
I think this needs a lot more discussion - including what you mean by standalone in this case.
I'm fine with a live discussion. By stand-alone, I mean that any given test has no dependence on the existence or execution of any other test. AMENDMENTS don't have to be alone, but they must be able to be. The VALIDATIONS need to be in place to inform. It doesn't really matter in implementation if some code is shared between them. I think the two need to remain as independent tests.
On Tue, Nov 2, 2021 at 5:39 PM Arthur Chapman @.***> wrote:
I think this needs discussing in a forum - we seem to have gone in a circle here in the relation between VALIDATIONS and AMENDMENTS. It has been so long between discussions, I seem to be getting confused as to where we have been and where we are going. We spent a lot of time making sure that AMENDMENTS had corresponding VALIDATIONS. If AMENDMENTS are entirely stand alone - then why do we need many of the VALIDATIONS? We could include the failure state within the AMENDMENT - i.e. reporting more fully why INTERNAL_PREREQUESITES failed - extending that to explain why, could remove the need for the VALIDATION. This would mean a total revisit to the way we do the tests and I don't recommend we go there. I can see lots of places running the VALIDATIONS and not the AMENDMENTS.
I think this needs a lot more discussion - including what you mean by standalone in this case.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tdwg/bdq/issues/32#issuecomment-958148768, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADQ727WNKQTI5F5E5OUQCDUKBD7BANCNFSM4EKSLOSA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
I like the philosophy of stand-alone, as the dependencies are truly legion, as we started to discuss in Gainesville. We do however have at least one 'process': Run VALIDATIONs, run AMENDMENTs, re-run VALIDATIONs.
Yes, there is the level of suggested workflows using the tests that we have not fully explored. It may be beyond our scope to do so.
On Wed, Nov 3, 2021 at 6:47 PM Lee Belbin @.***> wrote:
I like the philosophy of stand-alone, as the dependencies are truly legion, as we started to discuss in Gainesville. We do however have at least one 'process': Run VALIDATIONs, run AMENDMENTs, re-run VALIDATIONs.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tdwg/bdq/issues/32#issuecomment-960151874, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADQ7245MKROLOC6DLPQ2Q3UKGUXRANCNFSM4EKSLOSA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
In the Expected Response we have "information" twice in the same sentence - is this OK or need changing
"were populated from information in verbatim coordinate information"
I would skip the "Verbatim coordinates" and just cut to the chase (dwc:....) in all cases, nd skip the second information.
Not quite that simple, because "verbatim coordinates" cover a multitude of sins (dwc:verbatimCoordinates or dwc:verbatimLatitude and dwc:verbatimLongitude, plus dwc:verbatimCoordinateSystem and dwc:verbatimSRS)
My point is if we are angling to being specific, I would much prefer to use "dwc:verbatimLatitude and dwc:verbatimLongitude or dwc:verbatimCoordinates" rather than "verbatim coordinates (dwc:verbatimLatitude and dwc:verbatimLongitude or dwc:verbatimCoordinates)"
I'd agree with @tasilee, in the expected response (specification), it is clearer to reference the specific information elements than a general information element that we may have defined elsewhere. Phrasing just need to be clear that intent is (dwc:verbatimLatitude and dwc:verbatimLongitude) or dwc:verbatimCoordinates.
Don't forget the "plus dwc:verbatimCoordinateSystem and dwc:verbatimSRS"
I have edited accordingly
In accordance with discussions 16 April, I have edited the Expected Response to
INTERNAL_PREREQUISITES_NOT_MET if either dwc:decimalLatitude or dwc:decimalLongitude was not EMPTY, or either dwc:verbatimLatitude and dwc:verbatimLongitude, or dwc:verbatimCoordinates were not unambiguously interpretable into valid coordinates; FILLED_IN the values of dwc:decimalLatitude and dwc:decimalLongitude if unambiguous values can be interpreted from dwc:verbatimCoordinates or dwc:verbatimLatitude and dwc:verbatimLongitude, plus dwc:verbatimCoordinateSystem and dwc:verbatimSRS; otherwise NOT_AMENDED
I added an IF phrasing after the FILLED_IN...to align with current usage
Changed case of "epsg" to "EPSG" to match usual usage of this pseudo-namespace. Added an example of a conversion of a verbatim UTM coordinate.
Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted"