bdq icon indicating copy to clipboard operation
bdq copied to clipboard

TG2-AMENDMENT_GEODETICDATUM_ASSUMEDDEFAULT

Open iDigBioBot opened this issue 7 years ago • 77 comments

TestField Value
GUID 7498ca76-c4d4-42e2-8103-acacccbdffa7
Label AMENDMENT_GEODETICDATUM_ASSUMEDDEFAULT
Description Proposes an amendment to fill in dwc:geodeticDatum using a prameterized value if the dwc:geodeticDatum is empty.
TestType Amendment
Darwin Core Class dcterms:Location
Information Elements ActedUpon dwc:geodeticDatum
dwc:coordinateUncertaintyInMeters
Information Elements Consulted dwc:decimalLatitude
dwc:decimalLongitude
Expected Response If dwc:geodeticDatum is bdq:Empty, fill in dwc:geodeticDatum using the value of bdq:defaultGeodeticDatum, report FILLED_IN and, if dwc:coordinateUncertaintyInMeters, dwc:decimalLatitude and dwc:decimalLongitude are bdq:NotEmpty, amend the value of dwc:coordinateUncertaintyInMeters by adding the maximum datum shift between the specified bdq:defaultGeodeticDatum and any other datum at the provided dwc:decimalLatitude and dwc:decimalLongitude and instead report AMENDED; otherwise NOT_AMENDED.
Data Quality Dimension Completeness
Term-Actions GEODETICDATUM_ASSUMEDDEFAULT
Parameter(s) bdq:defaultGeodeticDatum
Source Authority bdq:defaultGeodeticDatum default = "EPSG:4326"
Specification Last Updated 2024-11-12
Examples [dwc:geodeticDatum="[null]", dwc:decimalLatitude="-30.00", dwc:decimalLongitude="130.00", dwc:coordinateUncertaintyInMeters="50": Response.status=AMENDED, Response.result=dwc:geodeticDatum="EPSG:4326", dwc:coordinateUncertaintyInMeters="2836", Response.comment="dwc:godeticDatum is bdq:Empty so filled in with default and dwc:coordinateUncertaintyInMeters amended to maximum possible value"]
[dwc:geodeticDatum="WGS84", dwc:decimalLatitude="", dwc:decimalLongitude="", dwc:coordinateUncertaintyInMeters="": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:geodeticDatum contains a value"]
Source ALA, GBIF
References
  • Maptiler (2019) EPSG.io. https://epsg.io/
  • EPSG (2024) About the EPSG Dataset. https://epsg.org/
  • Spatial Reference (2024) What is SpatialReference.org. https://spatialreference.org/
  • Geomatic Solutions (2018) Georepository. Version 9.0.0.1062. https://georepository.com/
  • Chapman AD and Wieczorek JR (2020) Georeferencing Best Practices. Copenhagen: GBIF Secretariat. https://doi.org/10.15468/doc-gg7h-s853
  • Wieczorek C and Wieczorek J (2021) Georeferencing Calculator. http://georeferencing.org/georefcalculator/gc.html
Example Implementations (Mechanisms)
Link to Specification Source Code
Notes The value of dwc:geodeticDatum applies to dwc:decimalLatitude and dwc:decimalLongitude, thus EPSG:4326 https://epsg.org/crs_4326/WGS-84.html is the appropriate EPSG code as it applies to the WGS84 datum used with a geographic coordinate system. If the dwc:coordinateUncertaintyInMeters is bdq:Empty, not interpretable, or not valid, this amendment should not provide a dwc:coordinateUncertaintyInMeters. If the dwc:coordinateUncertaintyInMeters is bdqNotEmpty and is valid, this amendment should add to the dwc:coordinateUncertaintyInMeters the uncertainty contributed by the maximum datum shift at the given coordinates. Since different systems have differing requirements for what the default datum should be, it is left unspecified, but should match whatever the target datum is in AMENDMENT_COORDINATES_CONVERTED (620749b9-7d9c-4890-97d2-be3d1cde6da8). After the amendment is performed, the dwc:geodeticDatum field should be the assumed default datum as parameterized. An example implementation to determine the uncertainty added by asserting a default datum (datum shift) where a known datum is not declared can be found in datumshiftproj.py in the source code for the Georeferencing Calculator (Wieczorek & Wieczorek 2021). Included in the source code is a 5-degree grid of datum shifts from an unknown datum to WGS84.

iDigBioBot avatar Jan 05 '18 16:01 iDigBioBot

Comment by Paul Morris (@chicoreus) migrated from spreadsheet: See 7e0c0418-fe16-4a39-98bd-80e19d95b9d1 GEODETIC_DATUM_INVALID (VALIDATION_GEODETICDATUM_NOTSTANDARD) for the Validation method and MEASURE_GEODETICDATUM_SINGLE_COMPLETENESS for the measurement method. Potential ammendment ordering dependency, should run after an amendment that proposes a value for coordinateUncertantyInMeters

iDigBioBot avatar Jan 05 '18 16:01 iDigBioBot

Parameter, as @ArthurChapman pointed out in #178 is which geodetic datum to use, not which vocabulary of geodetic datums to use.

bdq:sourceAuthority (default = http://epsg.io/)

Can only be interpreted as dwc:geodeticDatum is assumed to be "http://epsg.io" when not specified. This is clearly nonsensical, the default value should be WGS84, or the appropriate EPSG code, probably EPSG:4326.

Parameter is definitely needed but should be:

bdq:defaultGeodeticDatum (default=EPSG:4326)

or

bdq:defaultGeodeticDatum (default=https://epsg.io/4326)

With the notes specifying that the geodetic datum is taken from the http://epsg.io/ vocabulary....

chicoreus avatar Aug 21 '19 01:08 chicoreus

@chicoreus This is the default sourceAuthority not the default value - so the value should be a value within that default source authority - whether it is ESPG:4326 (WGS84), EPSG:4674 (SIRGAS2000) or EPSG:7844 (GDA2020), etc. What you are testing for is a value in the field that is consistent with the a value in http://epsg.io/ - the same as you look for a country code in the ISO standard. On thinking about it - probably doesn't need to be Paramaterized and just accept that authority as THE authority.

ArthurChapman avatar Aug 21 '19 02:08 ArthurChapman

@ArthurChapman - you raise a nice distinction: A source (in this case again unique), but then the default value, of which there can be plenty. I can see therefore two scenarios for Parameterised?

  1. A choice of "specified source authority" and/or
  2. A choice of a default value from a "specified source authority"

Tasilee avatar Aug 21 '19 03:08 Tasilee

@Tasilee This is the case for all tests (especially NOTSTANDARD and STANDARDIZED tests) that use an external Standard - ISO, DCMI, in this case EPSG, of any Vocabulary. The vocabulary, standard, etc. is the bdq:sourceAuthority and you are checking to see if the value in the record is a valid record in the bdq:sourceAuthority (in the case of Validations) or can be amended to conform with a value in the bdq:sourceAuthority (in the case of Amendments). In nearly all cases, there is only one sourceAuthority (except as @chicoreus mentions with Taxon names), so there is no choice of sourceAuthority needed, only the choice of a value from that sourceAuthority. Those few cases where there is a choice of sourceAuthority (taxon names) brings in both your 1) and 2) above. Thus, I agree with @chicoreus that we don't need as many Paramaterized tests as we have previously so tagged. Unless @tucotuco has justifications for them that we have not thought of.

ArthurChapman avatar Aug 21 '19 05:08 ArthurChapman

Phrasing of Amended text is unclear. Don't know how to interpret the comma:

AMENDED if the field dwc:geodeticDatum was EMPTY or was uninterpretable, the value of dwc:geodeticDatum was set to a predefined default value;

Perhaps

AMENDED if the field dwc:geodeticDatum was EMPTY or was uninterpretable and the value of dwc:geodeticDatum was set to a predefined default value;

chicoreus avatar Aug 31 '19 23:08 chicoreus

I commented in @chicoreus email of September 1. We do need to standardize the phrasing.

Tasilee avatar Sep 01 '19 23:09 Tasilee

I just checked the editing history - back in August last year the wording had a "therefore" after the comma AMENDED if the field dwc:geodeticDatum was EMPTY or was uninterpretable, therefore the value of dwc:geodeticDatum was set to a predefined default value;

ArthurChapman avatar Sep 02 '19 00:09 ArthurChapman

Sounds good to me. I've edited the ER as issues to discuss/fix are building.

Tasilee avatar Sep 03 '19 00:09 Tasilee

Needs discussing in conjunction with #60. I have not altered to conform with #59 as it is complicated. Parameterization should be removed I think in line with #59.

ArthurChapman avatar Apr 01 '20 23:04 ArthurChapman

@ArthurChapman: I think this IS parameterised as it requires a default value for dwc:geodeticDatum? The Parameter isn't EPSG as a sourceAuthority, but which EPSG code is the default?

The Example should have the assumption, e.g., "dwc:geodeticDatum is NULL and has been assumed EPSG:4326"?

Tasilee avatar Apr 02 '20 00:04 Tasilee

In line with recent discussions, there is ONE dq:sourceAuthority for the GDs, epsg.io, therefore that isn't a Parameter. In this case, why are we even including "specified source authority" other than than in References and possibly Notes to recommend use of EPSG codes? There is no need to access the source authority in the Expected response, simply to look up the Parameter value, e.g., EPSG:4326 as this is implementation dependent. There is no need for "EXTERNAL_PREREQUISITES_NOT_MET".

I would also suggest amending the example to "dwc:geodeticDatum is NULL, so amended to dwc:geodetciDatum EPSG:4326".

Tasilee avatar Apr 06 '20 01:04 Tasilee

@Tasilee Parameterized as different users might want different defaults. This one is appropriate to have a parameter.

chicoreus avatar Apr 06 '20 01:04 chicoreus

No, @Tasilee - you still have to go to an external source to look up the EPSG Code - so if the EPSG Server is down, then the EXTERNAL_PREREQUISITES_NOT_MET because the bdq:sourceAuthority (i.e. epsg.io) could not be found.

@chicoreus - it is not Paramaterized, because there is only one bdq:sourceAuthority, therefore there is no other Parameter that one can chose - we are only using the EPSG codes

ArthurChapman avatar Apr 06 '20 01:04 ArthurChapman

@ArthurChapman needs a parameter, not bdq:sourceAuthority, but defaultCRS, with epsg:4326 as the default.

chicoreus avatar Apr 06 '20 01:04 chicoreus

OK @chicoreus - I can agree - so if you are in Brazil, and the geodeticDatum is empty - you can set it to EPSG:4674 (i.e. the SIRGAS 2000 geodetic datum). I am happy with that. So with what @Tasilee said - you don't need to go to an External source - you set it as EPSG:4326 or some other value you add in the Parameter (for example EPSG:4674)

ArthurChapman avatar Apr 06 '20 01:04 ArthurChapman

I disagree @ArthurChapman. When running the test, you don't need to look up anything at epsg.io. The Parameter is a specific GD!

Tasilee avatar Apr 06 '20 01:04 Tasilee

@Tasilee I concur.

chicoreus avatar Apr 06 '20 01:04 chicoreus

I am going to edit expected response, parameter, example and notes so please check.

Tasilee avatar Apr 06 '20 01:04 Tasilee

I still have a few problems with this one (my suggested changes in bold

  1. In Expected Response: Now "therefore the value of dwc:geodeticDatum was set to a default value; otherwise..." - I think this would better read "therefore the value of dwc:geodeticDatum was set to the parameterized EPSG value; otherwise..." It is not a default value - someone (Brazil could use EPSG:4674 for example) - it is a parameterized value - which if not set defaults to the default value (EPSG:4326). I think we should mentioon EPSG in this somewhere.
  2. In the Notes curreently read "Since different systems have differing requirements for what the default datum should be, it is left unspecified, but should match whatever the target datum is in #43 . After the amendment is performed, the dwc:geodeticDatum field should be the assumed default datum as parameterized." I think should be changed to "Since different systems have differing requirements for what the parameterized datum should be, it is left unspecified, but should match whatever the target datum is in #43 . After the amendment is performed, the dwc:geodeticDatum field should be the assumed parameterized datum." It is not the default datum - it is the parameterized datum.

ArthurChapman avatar Apr 06 '20 02:04 ArthurChapman

Thanks @ArthurChapman. It isn't a 'parameterized EPSG value" or a "parameterized datum". I think it is a parameter which takes an EPSG code/value. I'd prefer

Expected response:

INTERNAL_PREREQUISITES_NOT_MET if the Parameter is not set; AMENDED to the Parameter value if dwc:geodeticDatum was EMPTY; otherwise NOT_CHANGED

Notes:

The test is Parameterised as different installations will have different defaults for dwc:geodeticDatum. If dwc:geodeticDatum is EMPTY, the amendment will result in dwc:geodeticDatum taking the Parameter value.

In reviewing this test, some things become apparent. EXTERNAL_PREREQUISITES_NOT_MET is misspelled EXTERNAL_PREREQUESITES_NOT_MET in at least 10 tests. I'll fix those next.

Second, if dwc:geodeticDatum is NOT_EMPTY, we do nothing, regardless if it is interpretable or not.

Tasilee avatar Apr 06 '20 07:04 Tasilee

Also...I don't think we need to use "the field...." more simply, for example, "dwc:..."

Tasilee avatar Apr 06 '20 07:04 Tasilee

I agree with @Tasilee - this test needs rewording - I've looked at it in several ways but not yet come up with an answer.

Basically, if it is not EMPTY we do nothing so couldn't we say something like (a little redundancy)

INTERNAL_PREREQUISITES_NOT_MET if dwc:geodeticDatum is not EMPTY, or is an uninterpretable value or the Parameter is not set; AMENDED to the Parameter value if dwc:geodeticDatum was EMPTY or is not an interpretable value; otherwise NOT_CHANGED

Still not really happy with this.

ArthurChapman avatar Feb 07 '22 22:02 ArthurChapman

I'm thinking perhaps as simple as:

Parameter: bdq:defaultGeodeticDatum, default value = 'EPSG:4326'

CHANGED to the value of bdq:defaultGeodeticDatum if dwc:geodeticDatum is EMPTY, otherwise NOT_CHANGED.

If we specify the default value for the parameter, then we don't have to worry about EXTERNAL_PREREQUISITES_NOT_MET, as the parameter will be available even if not specified, and there isn't an internal data value that would cause us to specify INTERNAL_PREREQUISITES_NOT_MET.

chicoreus avatar Feb 07 '22 22:02 chicoreus

@chicoreus - that would seem OK

We need to allow the setting of a different parameter for bdq:defaultGeodeticDatum. We allowed for this to be parameterized because some countries legislate a datum for that country - e.g. Brazil and the South American Datum (SAD69(96) epsg:5534 from memory)

ArthurChapman avatar Feb 07 '22 22:02 ArthurChapman

I like @chicoreus's comment. I like simple. @ArthurChapman: All the suggested Parameter is implying is that if it isn't set, use EPSG:4326. I've edited the Specifications accordingly.

Tasilee avatar Feb 09 '22 21:02 Tasilee

Happy with that

ArthurChapman avatar Feb 09 '22 21:02 ArthurChapman

Changed "AMENDED" to "FILLED_IN" in accordance with discussions April 16.

Tasilee avatar Apr 18 '22 22:04 Tasilee

coordinateUncertaintyInMeters is listed as an information element and discussed in the notes, but not mentioned in the expected response.

chicoreus avatar Sep 04 '22 01:09 chicoreus

The logic of the notes also results in a conflicting response.status - dwc:geodeticDatum would be FILLED_IN, but dwc:coordinateUncertaintyInMeters could be AMENDED, and the response structure does not allow for multiple status values.

chicoreus avatar Sep 04 '22 01:09 chicoreus