Open-Assistant
Open-Assistant copied to clipboard
LAION-AI
Ability to share a task?
Might it be useful for users to be able to share tasks with each other and/or admins?
oh i see i have a prompt id so maybe thats enough if ever need to debug with admins or anything like that.
on the sharing of tasks - i do wonder if could be some interesting data in that in allowing users to share tasks and then getting different responses from a specific user network. might be messy - but also could be some gamification stuff that might help there too potentially from a product point of view....
@k-nearest-neighbor had some ideas about having fixed task URLs. Since the website gives each task a fixed URL I think this is very doable and just requires changing the website/src/pages/{evaluate,create}/{task_type}.tsx page files accordingly to fetch a specific task ID if one is provided.
If you'd like, could you try making this work with one task type (without breaking anything that currently works)?
Perhaps on a higher level: being able to open a specific task with ID is a double edged sword.
Good for debugging, moderating, and user-reporting. Bad because it can be abused to target a specific task / response / user.
I am unsure if any of these points are a concern to us in this stage.
Right. I think in order for this to be safe from attacks, the server needs to be able to reject submissions to a task.
I think right now we should leave this open for discussion to evaluate the ways we can mitigate the risks and then turn it into a milestone with some tactical issues once we've figured them all out.
I also think it's a double edge sword (so there are cons but also could be some pros) as there could also be some signals in the types of tasks that people chose to share and then being able to easily get a tree from the shared task and all the different variants of responses to it from within a sharers network could actually turn out to be useful data in some way. Not really sure how but think there could eventually be a way to leverage shared tasks to maybe get slightly better than random response data. But have not thought about it much so don't have any concrete ideas, more of a git feeling that maybe might want to revisit and think about it later.
I feel like I might have missed the mark on what we need to protect here, but isn't/shouldn't the task inherently only be completeable by the assigned user? And the task ID, which is a GUID would have to be guessed, am I right?
[update as I write this] After writing that I realised that the original proposal was to share the task to get others to also try completing the same task, and I see how if that is the goal then we would be allowing people to submit to a task of their choice, which obviously lacks the inherent protections I mentioned and raises bigger questions about data bias/targeting.
One of the benefits I can see of having a page where the task can be seen is so that the e2e tests could pretend to be a user and complete a task and then check the task page to see that the backend successfully received and handled the result.
more so i think i was coming at it from the gamification and product point of view - eg if i had the ability to just share a task with someone else, so basically send them same task i just did with a new (child) guid perhaps. maybe that actually could be useful and rich data.
that said, am sure on backend we would be able to see same task question (eg hash of what actual task was regardless of id's) across multiple different task id's so maybe ability to share only might really give you fact that two of those backend tasks are actually linked in some way - one task_id gave birth to another task_id and so the users are maybe related in some way. Longer term maybe having ability to see those links could be useful but down side is could introduce some selection bias in task data itself then so maybe not exactly worth it.
from a product/gamification point of view - might it be more fun to be able to share a task? and then we just capture that fact and handle accordingly however we might want.