Add field `IsValid` and/or `deploymentEndType` to the deployment properties

[PatrickAJansen]

Please add a field "IsValid" to the deployment properties. This helps users to flag deployments as invalid (for example based on camera malfunction or bad placement), while still keeping it in the database. The latter is important for keeping track of the sampling (e.g., check whether all sampling points have a deployment). The old Agouti had this field and it was very useful.

[peterdesmet]

@PatrickAJansen although useful, wouldn't an isValid flag lead to questions why a deployment was considered invalid? To provide that context, we could also adopt something similar to Movebank's deployment end type term, which provides a list of controlled values why a deployment was (prematurely) ended:

• analysis-end = the end time represents the end of the period of interest;
• captured = The tag remained on the animal but the animal was captured or confined;
• dead = The deployment ended with the death of the animal that was carrying the tag;
• dead/fall-off = The tag stopped moving, and it is not possible to determine whether it is due to death of the animal or unscheduled tag detachment;
• equipment-failure = The tag stopped working;
• fall-off = The attachment of the tag to the animal failed, and it fell of accidentally;
• other = other;
• released = The tag remained on the animal but the animal was released from captivity or confinement;
• removal = The tag was purposefully removed from the animal;
• scheduled-detachment = The tag was programmed to detach from the animal;
• transmission-end = The tag stopped transmitting usable data;
• unknown = The cause of the end of data availability or transmission is unknown.

Obviously the terms above don't all apply to camera trap data, but they do provide more context to the user than valid TRUE/FALSE and allow them to include/exclude deployments they want.

Question

For @PatrickAJansen @jimcasaer @Tim-Hofmeester and others ...

• [ ] Would a deployment end type field (like the one above) and associated controlled vocabulary be a sufficient replacement for an isValid flag?
• [ ] Is the extra context it provides needed? Or is a data-owner decided valid TRUE/FALSE sufficient?
• [ ] What controlled values would you include?

cameraFailure
cameraRemoval
...

[PatrickAJansen]

Yes, this makes sense and your solution would work. But what I would prefer, however, is a field "isValid" plus a field "InvalidReason" to specify why "Isvalid" = FALSE.

Levels could include:

• cameraFailure
• invalidPlacement
• invalidDate
• invalidTime
• invalidTimeZone
• invalidSettings
• disturbance *
• invalidDuration
• other (The list will probably be longer).

The thing with disturbance is that deployments may be good until the disturbance happens. Ideally, users could discard all material post-disturbance, and retain only the valid part.

[Tim-Hofmeester]

I think the combination of fields suggested by @PatrickAJansen would work best from a user point of view, but I think a deployment end type field could work too (I guess especially in the example above where a disturbance ends a deployment that would be more intuitive). In any case, I think it is necessary to provide information as to why the deployment was considered invalid by the project manager, as e.g., the wrong placement or settings for one protocol might still provide usable data for other purposes (e.g., as presence-only data).

[peterdesmet]

I think just having a deployment end type field would be sufficient. Whether it is valid depends on the use case and therefore the user, so I would not hardcode this as part of Camtrap DP. @Tim-Hofmeester do I understand correctly this aligns with your comment?

[PatrickAJansen]

No, it would not. Please return to the original question. Tagging bad deployments in projects is part of project management and data quality control. Deployments marked as invalid by a project owner should not even be considered data suitable for sharing. In my projects, I will ultimately delete all invalid deployments before archiving the data. Thus, datasets are clean.

[peterdesmet]

Ok, I understand there are two separate things:

1. A deployment is considered invalid by the data manager and should not be exported.
2. A deployment ended prematurely for reason X and should therefor not be used for analysis Y but can still contain useful occurrence data.

---

1. is the isValid field. It is definitely useful in a data management system like Agouti, so managers can mark invalid deployments. Since invalid deployments would/should not be exported, all deployments in a Camtrap DP export would be isValid=TRUE therefore making the field unnecessary.
2. is the deploymentEndType field. It is useful for users so they can make an assessment on what not to include/exclude in their deployment.

Would you agree with this interpretation?

[lrdijkhuis]

I would like to put this thread under attention again. I agree with @PatrickAJansen his initial comments. I would also like to add that it is odd that there is an option for invalid in the interface of Agouti, (edit/add)deployment-tab), which is very helpful for users to keep track of their management, but that this information is not parsed to the export.

@peterdesmet

A deployment is considered invalid by the data manager and should not be exported. <

Invalid can be for many reasons, i.e. missing calibration of the camera trap, while t the data might still be useful for other analysis. Hence I would not agree with your formulation and intepretation of point 1. It is up to the user to decide what to do with the invalid deployments: if the data is indeed completely unuseful on will probably decide to remove it, but when it is partly useful, the invalid-annotation is still an important field in preprocessing of the exported data set.

I would prefer a variable which contains the logical information whether or not a deployment is invalid. As a user you could put this in de comments as well, but in my opinion this is not the suitable place to do so. This is where i would like to track any edits or special remarks that i have regarding other, more fluid, findings. Thanks for considering, Laurens

[PatrickAJansen]

Users can mark deployments as "invalid" in Agouti, for example because the placement was wrong, but the export still contains all the deployments, without any argument showing that they are invalid. This needs to be fixed either in the camtrapdp (filter option) or directly in Agouti (invalid deployments are not exported).

[peterdesmet]

@PatrickAJansen thanks for suggesting to fix this on Agouti's end, removing invalid exports from the export. I'll leave this issue open and call it deploymentEndType, which is then a more granular approach in expressing why deployments maybe be ended (and could be considered invalid).