vscode-copilot-release icon indicating copy to clipboard operation
vscode-copilot-release copied to clipboard

Responsible AI Service is way to sensitive

Open Kireobat opened this issue 1 year ago • 7 comments

Responsible AI Service is way to sensitive.

I was working with an Arduino and making some code for it. I asked copilot this question:

Can you write an explanation for the following connections on my D1 mini please:

TX, RX, 5 SCL, 4 SDA, 0, 2, GND, VBUS, RST, AO, 16, SCK 14, MISO 12, MCSI 13, SS 15, 3V3

Which may seem totaly innocent at first glance, but Responsible AI Services was smart enough to see that I had "SS" in the request meaning that I was up to no good and probably a nazi. Responsible AI Services did the right thing and blocked my horrible and dangerous request.

I eventually figured that the "SS" might be the problem so I asked this instead:

Can you write an explanation for the following connections on my D1 mini please:

TX, RX, 5 SCL, 4 SDA, 0, 2, GND, VBUS, RST, AO, 16, SCK 14, MISO 12, MCSI 13, 15, 3V3

And this was accepted. I hope that whoever is responsible for Responsible AI Services can adjust their regex pattern so that you don't get flagged for merely including "SS" in a question.

Thanks for reading

Kireobat avatar Mar 13 '24 19:03 Kireobat

That "responsible AI service" is just some stakeholders at github trying to exist. It is a very good argument to try other, better tuned, ai copilot.

SigismondPhilippe avatar Mar 14 '24 19:03 SigismondPhilippe

@Kireobat i'm having trouble reproducing this issue. when i use the same prompt, i get a response back from copilot chat in vs code. are you still having this problem with the same prompt?

thank you

Image

thispaul avatar Mar 19 '24 20:03 thispaul

@thispaul I just started a new chat and copy and pasted that prompt image

lramos15 avatar Mar 19 '24 20:03 lramos15

I still have the issue when including the "SS" image

Kireobat avatar Mar 19 '24 20:03 Kireobat

Thanks @Kireobat we are troubleshooting this with the RAI team

thispaul avatar Mar 19 '24 20:03 thispaul

I can confirm that my prompt "Please put the elapsed time in hh:mm:ss format" was also flagged.

GGLionCross avatar Aug 28 '24 17:08 GGLionCross

I have the same issue but for another prompt. However, I haven't found the reason why. Especially as I entered a prompt in German, where I haven't anything special.

Therefore, is there the possibility to get the reason, why the information was filtered, to troubleshoot the process myself?

ced-mos avatar Sep 27 '24 07:09 ced-mos

flagging double "s" is ridiculous. Mine got flagged for "killTask" which is a function in a library.

Simsalaba avatar Oct 11 '24 09:10 Simsalaba

+1 - trying to have it evaulate any JSON is a nightmare given the random characters that might hurt its feelings... makes it unusable.

sspage avatar Oct 21 '24 02:10 sspage

Thanks for creating this issue! We figured it's covering the same as another one we already have: https://github.com/microsoft/vscode/issues/253130. Please refer to that one for updates and discussions. Feel free to open a new issue if you think this is a different problem.

bhavyaus avatar Nov 17 '24 07:11 bhavyaus