data-hazards
data-hazards copied to clipboard
very-good-science
🎓💖 Feedback: Facilitator Training 29th June
Use this issue to send us feedback, and comment with your suggestions for new safety precautions or examples for a Hazard label of your choice :).
We can then make your suggestions and add you to our contributors list!
Please tell us:
- which Hazard label you're suggesting a change to
- whether you are suggesting a change to the Examples or Safety Precautions
- what the suggested addition/change is
- [ ] Add these suggestions to the website - @NatalieZelenka
Lacks community involvement:
Suggesting a new Safety Precaution -
- consider where the benefit for data science work lies and ensure that communities receive benefit from this work that is desirable to them.
- Be active in reaching out to communities, even diasporic communities if you are not sure how to contact data source communities that are resident elsewhere.
Ranks Or Classifies People: Safety precaution: Consider the implication of sampling bias in the data used as inputs to classification model e.g. model identifying high harm/ high risk individuals that has been trained using existing law enforcement databases only ‘knows’ about individuals that have a criminal record or have been investigated by law enforcement with inherent collection bias.
I would love for there to be an AI Hype label with the drawing of a toaster!
We hear all too often about new AI systems being "sentient", "intelligent" and so on. In the meantime, the vast majority of us don't understand the underlying mechanics of these systems. If we did, I would imagine the hype would be mitigated.
A suggestion to for the Examples for Difficult to understand https://datahazards.com/contents/hazards/difficult-to-understand.html
The advanced use of spreadsheets with complicated formulae, macros, VB, multiple tabs with links. It often hides algorithms and assumptions.
I would suggest a change to the May Cause Direct Harm label, because as you see in the news, there have been many examples that can apply to this label. New Example: The availability of images of self-harm, weapons, drugs and violent material to young children on social media. It has led to acts of suicide and violent acts of violence e.g. Panorama episode: Thirteen-year-old Olly Stephens left home for the final time on a Sunday afternoon in January 2021, telling his parents he was meeting a friend nearby. Fifteen minutes later, he had been murdered. Lured out by a teenage girl and stabbed to death by two teenage boys she had met online, the entire attack was planned on social media and triggered by a dispute on a chat group. With exclusive access to Olly’s parents Amanda and Stuart, Panorama reporter Marianna Spring investigates the violent and disturbing world their son had been exposed to online and follows their campaign for tighter regulations on harmful content.
A suggestion for the Examples for General Hazards https://datahazards.com/contents/hazards/general-hazard.html
Issues around data collection/study design. Known missing data or data labels used as a 'catch all'
Automates decision making. Safety precaution: make sure people have an easy, accessible and timely way to challenge the automated decision making and/or not having it applied in the first place
[Lacks community involvement] + [Classifies and ranks people]
[Lacks community involvement] label mentions "technology is being produced without input from the community it is supposed to serve."
A lot of the time there is a bias in deeming some communities over others, including human species over non-human species. Example: research on welfare of horses and riders puts people over horses (is it supposed to serve horses too?), if the human dies the problem is deemed as bigger.
And this relates to [classifies and ranks people] hazard. Which the hazard in itself is not including the thought that some parts of nature are being ranked over others. What happens when ranking is inaccurate? Is speciesism inaccurate? This hazard relies on an ethical framework that is unclear. In the same way someone might not agree something is speciesist, or something is sexist.
A suggestion for the "Difficult to understand" hazard: I would advocate for publishing code if possible (although potentially this affects other harms, such as danger of misuse). The best documentation in the world doesn't help if there's an error or a misinterpretation by the developers, or something that is missing from the documentation that would otherwise go undetected.
[Reinforces existing bias] hazard would benefit from branching out into more specific sub-hazards.
What kind of bias is being reinforced? Racist? Sexist? Speciesist? Not adequate data? Cognitive bias in decision making? How do you you know if a bias is already existing?
Danger of misuse: Example 1: Misinterpreting statistical methods or failing to appreciate their limitations, for example assuming that psychometrics will accurately or definitively predict future human behaviour.
Additional e.g.: Re-appropriation of data for unintended purposes, for example, data collected for medical purposes being used for insurance adjustment.
Explicitly setting out assumptions, and any key research that is being relied upon for the project.
This can help the 'project owner' (project designer?) explain to the 'audience' (stakeholder representatives) why the project is designed this way - how it builds on existing work, and what we should learn/achieve from the project.
Have added all of these suggestions to the V1.0 update in PR #169 :) Thanks everyone ❤️ I have minorly re-worded a couple or made the more direct/specific but hopefully they all reflect your original thought process. The new version will be going up this week so your contributions will be live!