Do we need a project to imagine how this technology could be deliberately mis-used, and thus design mitigations?

[dwsinger]

We tend to design our technologies assuming benevolent intentions of those involved. The entire ARPAnet was built without a security, privacy, or attack model, because it was assumed to be used only by cooperating universities and government bodies.

I doubt that we'll 'tune' ML on the web without considering explicit the ways it might be or is being mis-used. However, obviously developing a database of such misuses risks writing a playbook for the nefarious – it would have to be done carefully, and in some respects contrary to our usual openness.