bdq
bdq copied to clipboard
tdwg
TG2-AMENDMENT_EVENTDATE_FROM_YEARSTARTDAYOFYEARENDDAYOFYEAR
| TestField | Value |
|---|---|
| GUID | eb0a44fa-241c-4d64-98df-ad4aa837307b |
| Label | AMENDMENT_EVENTDATE_FROM_YEARSTARTDAYOFYEARENDDAYOFYEAR |
| Description | Proposes an amendment to the value of dwc:eventDate from values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear. |
| TestType | Amendment |
| Darwin Core Class | dwc:Event |
| Information Elements ActedUpon | dwc:eventDate |
| Information Elements Consulted | dwc:year |
| dwc:startDayOfYear | |
| dwc:endDayOfYear | |
| Expected Response | INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate is bdq:NotEmpty or any of dwc:year, dwc:startDayOfYear, or dwc:endDayOfYear are bdq:Empty; FILLED_IN the value of dwc:eventDate from values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear if the values in each are independently interpretable and if the value of dwc:startDayOfYear is less than the value of dwc:endDayOfYear; otherwise NOT_AMENDED |
| Data Quality Dimension | Completeness |
| Term-Actions | EVENTDATE_FROM_YEARSTARTDAYOFYEARENDDAYOFYEAR |
| Parameter(s) | |
| Source Authority | |
| Specification Last Updated | 2024-08-18 |
| Examples | [dwc:year="1901", dwc:startDayOfYear="15", dwc:endDayOfYear="25", dwc:eventDate="": Response.status=FILLED_IN, Response.result=dwc:eventDate="1901-01-15/1901-01-25", Response.comment="dwc:eventDate was interpreted from dwc:year, dwc:startDayOfYear and dwc:endDayOfYear"] |
| [dwc:year="1901", dwc:startDayOfYear="25", dwc:endDayOfYear="15", dwc:eventDate="": Response.status=NOT_AMENDED, Response.result="", Response.comment="dwc:startDayOfYear > dwc:endDayOfyear"] | |
| Source | TG2-Gainesville |
| References |
|
| Example Implementations (Mechanisms) | Kurator:event_date_qc |
| Link to Specification Source Code | https://github.com/FilteredPush/event_date_qc/blob/4665e4d3b43ce7ddf319b3d7a5d3dbfee1411250/src/main/java/org/filteredpush/qc/date/DwCEventDQ.java#L828 Unit Test at: https://github.com/FilteredPush/event_date_qc/blob/96a8981d997cceb2f39ba47d63f0b98c1b56680c/src/test/java/org/filteredpush/qc/date/DwcEventDQTest.java#L402 |
| Notes | An attempt to populate dwc:eventDate from dwc:verbatimEventDate should be made before this test is run. While year=1999, startDayOfYear=123 could be validly represented as an ISO date as either 1999-123 or 1999-05-03, the latter of these two forms SHOULD be used, thus, do not simply concatenate dwc:year and dwc:startDayOfYear. This test is only for cases that fall within the one year (as given in dwc:year) and hence "dwc:startDayOfYear will always be less than dwc:endDayOfYear". [or do we just leave this as being obvious from the Expected Response. |
~~Note "An attempt to populate dwc:eventDate from dwc:verbatimEventDate should be made before this test is run." should be removed.~~
I've added some potential text about the validity of 1999-123 (or 1999-123/1999-125) and proposed that these valid ISO date forms should not be used as the return value from this amendment. Not sure if this should be a SHOULD or MUST.
As discussed elsewhere @chicoreus - this dependency was added because of three tests to populate eventDate that should be run in order of a priority. These dependencies, from memory, were to be temporary until we had a workflow and would be removed from the description.
@ArthurChapman you are right, my goof. Must have though I was looking at the validation. We did indeed on a needed priority between the three amendments that can fill in an eventDate.
Updating the implementation of this, I'm finding the specification (expected response) to be unclear:
INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate was not EMPTY or dwc:year was EMPTY or both dwc:startDayOfYear and dwc:endDayOfYear were EMPTY or the values were not interpretable; AMENDED if dwc:eventDate was FILLED_IN from the values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear; otherwise NOT_AMENDED
If the clause " or the values were not interpretable" applies to year, startDayOfYear, and endDayOf year, then I don't see a path to the NOT_AMENDED response.status. But, if that clause applies only to startDayOfYear and endDayOfYear, then a non-empty, but invalid value for dwc:year would result in a NOT_AMENDED response.status.
Also, it isn't clear how to handle the case of startDayOfYear being empty and endDayOfYear containing a value, though it does seem logical to provide a single date if year and startDayOfYear have values but endDayOfYear doesn't.
Rephrasing to fit the latest implementation in event_date_qc would give:
INTERNAL_PREREQUISITES_NOT_MET if (1) dwc:eventDate was not EMPTY or (2) dwc:year was EMPTY or (3) both dwc:startDayOfYear and dwc:endDayOfYear were EMPTY or (4) if any not EMPTY values of dwc:year, dwc:startDayOfYear or dwc:endDayOfYear were not interpretable as numbers ; AMENDED if dwc:eventDate was FILLED_IN from the values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear;
Something that would not propose a change if startDayOfYear was not specified would be:
INTERNAL_PREREQUISITES_NOT_MET if (1) dwc:eventDate was not EMPTY or (2) dwc:year was EMPTY or (3) both dwc:startDayOfYear and dwc:endDayOfYear were EMPTY or (4) if any not EMPTY values of dwc:year, dwc:startDayOfYear or dwc:endDayOfYear were not interpretable as numbers ; AMENDED if (1) dwc:eventDate was FILLED_IN from the values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear or (2) if dwc:eventDate was FILLED_IN from the values in dwc:year, and dwc:startDayOfYear with dwc:endDayOfYear being EMPTY; otherwise NOT_AMENDED;
@chicoreus That looks good to me, but why would you not AMEND is startDayOfYear was EMPTY and endDayOfYear was not?
@tucotuco I propose that potential variant as year+startDayOfYear with no endDayOfYear feels like a logical choice that someone might make when mapping a collecting event date known to a single day onto year/startDayOfYear/endDayOfYear, the other logical choice being year+startDayOfYear, with endDayOfYear=startDayOfYear. Conversely year+endDayOfYear smells like some sort of error in mapping a date range of more than one day, so seeing that combination makes me more suspicious of it arising from an error condition of some sort.
Hmm. I start squirming when there is subjectivity like this. There are other explanations we haven't covered as well that might better inform the choice. Two that come immediately to mind are the cases 1) some day on or before endDayOfYear=x and 2) some day on or after startDayOfYear=y. If we can't distinguish these cases from simply incomplete cases, then I think we shouldn't be amending either of them.
@tucotuco Good. I like avoiding subjectivity. There is also the case where the eventDate spans more than one year, and potential variants in mapping that into start and end day of year. How about we change the specification from:
INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate was not EMPTY or dwc:year was EMPTY or both dwc:startDayOfYear and dwc:endDayOfYear were EMPTY or the values were not interpretable; AMENDED if dwc:eventDate was FILLED_IN from the values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear; otherwise NOT_AMENDED
To:
INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate was not EMPTY or any of dwc:year, dwc:startDayOfYear, or dwc:endDayOfYear were EMPTY or any of the values in dwc:year, dwc:startDayOfYear, or dwc:endDayOfYear were not interpretable; AMENDED if dwc:eventDate was FILLED_IN from the values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear; if the value of dwc:endDayOfYear is less than the value of dwc:startDayOfYear, NOT_AMENDED
I don't think this has a path that isn't covered, but if I've missed one, or to cover the potential, we could add an "otherwise" to the NOT_AMENDED clause.
Including otherwise, (which would cover any other cases individually valid values in year start/end day terms that aren't valid in combination e.g. endDayOfYear = 366 with year not a leap year):
INTERNAL_PREREQUISITES_NOT_MET if dwc:eventDate was not EMPTY or any of dwc:year, dwc:startDayOfYear, or dwc:endDayOfYear were EMPTY or any of the values in dwc:year, dwc:startDayOfYear, or dwc:endDayOfYear were not independently interpretable; AMENDED if dwc:eventDate was FILLED_IN from the values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear; if the value of dwc:endDayOfYear is less than the value of dwc:startDayOfYear, or otherwise NOT_AMENDED
Great @chicoreus and @tucotuco As a general principle, we should never AMEND something if there is doubt or where there is ambiguity. Comes into an earlier email where we have "X" as a date (month or day) - we have been inconsistent as to whether we AMEND it to "10" or not. If it ambiguous, we shouldn't AMEND.
Just looking at the last version by @chicoreus I don't understand the last part "if the value of dwc:endDayOfYear is less than the value of dwc:startDayOfYear", Don't you mean "greater than"?
@ArthurChapman "less than" should be correct. This is the case where it is NOT_AMENDED.
Ah yes @tucotuco I missed the semicolon - looking at it as a comma. By bad
Suggest a mod to the Expected Response (note also incorrect ";"):
AMENDED if dwc:eventDate was FILLED_IN from the values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear; if the value of dwc:endDayOfYear is less than the value of dwc:startDayOfYear, or otherwise NOT_AMENDED
to
AMENDED the value of dwc:eventDate from values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear if the value of dwc:endDayOfYear is less than the value of dwc:startDayOfYear, or otherwise NOT_AMENDED
Not needing "FILLED_IN" seems good.
OK but I would leave the "or" off "or otherwise, and add ";" for consistency
AMENDED the value of dwc:eventDate from values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear if the value of dwc:endDayOfYear is less than the value of dwc:startDayOfYear; otherwise NOT_AMENDED
@ArthurChapman - I think your logic is reverse? Don't we need
AMENDED the value of dwc:eventDate from values in dwc:year, dwc:startDayOfYear and dwc:endDayOfYear if the value of dwc:startDayOfYear is less than the value of dwc:endDayOfYear?
If you had ok values for dwc:year and dwc:startDayOfyear (cognisant of the Notes), the Expected Response would be INTERNAL_PREREQUISITES_NOT_MET. Is this correct?
I think that is correct sometimes - assuming eventDate is EMPTY, the conditions still would not allow for an amendment to be made if the dwc:endDayOfYear was EMPTY or not interpretable. In addition, I think that if the eventDate is EMPTY, the year is a valid year, and the endDayOfYear is less than the startDayOfYear that the expected response should be INTERNAL_PREREQUISITES_NOT_MET.
Added to Notes "This test is only for cases that fall within the one year (as given in dwc:year) and hence "dwc:startDayOfYear will always be less than dwc:endDayOfYear". [or do we just leave this as being obvious from the Expected Response."
Edited Note to change "year and startDayOfYear" to "dwc:year and dwc:startDayOfYear"
Splitting bdqffdq:Information Elements into "Information Elements ActedUpon" and "Information Elements Consulted".
Also changed "Field" to "TestField", "Output Type" to "TestType" and updated "Specification Last Updated"