Open-Assistant
Open-Assistant copied to clipboard
LAION-AI
Idea how to detect ChatGPT
In "Reply as Assistant" task we can check whether the text was copy-pasted or entered manually (or pasted and then edited) into textbox. Of course, someone may be using an external editor to create a reply so the copied text cannot be 100% considered a copy-paste from ChatGPT, but replies that were only entered by copy-paste without later editing can be flagged as potential ChatGPT response.
We considered this but a lot of users have said they like to write their answers in another editor, then copy and paste into the OA website and submit without further editing
How about asking ChatGPT directly whether it wrote the text or not: "Consider the following text and provide only a percent value describing the likelihood that you wrote the text as your answer. ..." On the downside, it rates its responses as well as my texts with 0%. There are tools out there claiming to detect AI answers but those will fail when the LLM answer is slightly modified. As a side note, ChatGPT agrees with your assessment and explains that it cannot assess origin and authenticity of texts. Perhaps if you include timing from reading to response and correlate that with the amount of text than you may get somehere.
How about asking ChatGPT directly whether it wrote the text or not: "Consider the following text and provide only a percent value describing the likelihood that you wrote the text as your answer. ..." On the downside, it rates its responses as well as my texts with 0%. There are tools out there claiming to detect AI answers but those will fail when the LLM answer is slightly modified. As a side note, ChatGPT agrees with your assessment and explains that it cannot assess origin and authenticity of texts. Perhaps if you include timing from reading to response and correlate that with the amount of text than you may get somehere.
I don't think using ChatGPT directly is a good idea, but I do like the idea of including timing data in the user submissions. I don't think time is the only metric of work though, and some work produces many words in little time. For example, the task "correct the grammar in my essay" will produce an equally large amount of text but might take only a little bit of time to produce an answer for. Same with other tasks like sorting or code evaluation, where we can use automated tools to do the work.
What would be an issue of of using ChatGPT directly? The output of an AI cannot be copyrighted so there certainly is no legal challenge. It might be interesting to train at least a separate model on a curated output of chatgpt results. Also the way that AI detection models currently work is they rely on 2 metrixs, perplexity and bustiness which humans highly exhibit and chatbots generally dont, however you can ask chatgpt to exhibit those metrics and it will generate undetectable text as output, even by openais own detector. Regards, Ignas Cell +44 7938015569
"In order to achieve your goals, you must apply discipline and consistency."
On Thu, Feb 9, 2023 at 5:24 PM mashdragon @.***> wrote:
How about asking ChatGPT directly whether it wrote the text or not: "Consider the following text and provide only a percent value describing the likelihood that you wrote the text as your answer. ..." On the downside, it rates its responses as well as my texts with 0%. There are tools out there claiming to detect AI answers but those will fail when the LLM answer is slightly modified. As a side note, ChatGPT agrees with your assessment and explains that it cannot assess origin and authenticity of texts. Perhaps if you include timing from reading to response and correlate that with the amount of text than you may get somehere.
I don't think using ChatGPT directly is a good idea, but I do like the idea of including timing data in the user submissions. I don't think time is the only metric of work though, and some work produces many words in little time. For example, the task "correct the grammar in my essay" will produce an equally large amount of text but might take only a little bit of time to produce an answer for. Same with other tasks like sorting or code evaluation, where we can use automated tools to do the work.
— Reply to this email directly, view it on GitHub https://github.com/LAION-AI/Open-Assistant/issues/1392#issuecomment-1424549509, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVF6XTKI4FUWHSJLW3WZW33WWUR3JANCNFSM6AAAAAAUWT7WBY . You are receiving this because you are subscribed to this thread.Message ID: @.***>
What would be an issue of of using ChatGPT directly?
Their terms of use forbid it. See this comment on issue #471.
