recaptcha icon indicating copy to clipboard operation
recaptcha copied to clipboard

Recaptcha v3 always returns a 0.1 score

Open SeriaWei opened this issue 6 years ago • 81 comments

When i use recaptcha v3 with host "www.recaptcha.net" the score is always 0.1, but when i change to use host "www.google.com", the score is 0.9. So is it the v3 can not use with www.recaptcha.net? Scripts

https://www.recaptcha.net/recaptcha/api.js?render={key}

Siteverify

https://www.recaptcha.net/recaptcha/api/siteverify

SeriaWei avatar Aug 13 '18 01:08 SeriaWei

Thanks for the report - not able to reproduce this, do you have a sample page you ran this? What happened when you mixed and matched siteverify and api.js locations?

jasonfedor avatar Aug 13 '18 17:08 jasonfedor

No, it was just localhost. I will push it to live to test later. Thanks

SeriaWei avatar Aug 14 '18 01:08 SeriaWei

Push to live is the same, the score is always 0.1. This is my test keys:

Site key

6Lc[redacted]Kdz

Secret key

[redacted]

With "www.recaptcha.net"

image

Change to "www.google.com"

image

But when i change to use another key, it works perfect with both "www.recaptcha.net" and "www.google.com"

SeriaWei avatar Aug 14 '18 03:08 SeriaWei

Please do not post your secret key in bug reports. I've removed it from your comment, but please also go to https://www.google.com/recaptcha/admin#list to create a new key.

rowan-m avatar Aug 14 '18 08:08 rowan-m

@rowan-m Thanks! But it was just a testing key. :)

SeriaWei avatar Aug 14 '18 08:08 SeriaWei

After change the key, it works at beginning. But i found that the score would continue to drop from 0.9 until 0.1, and then it would have been 0.1.

SeriaWei avatar Aug 17 '18 02:08 SeriaWei

I'm also getting 0.1. But it's on localhost.

amarnathm avatar Aug 19 '18 22:08 amarnathm

Hello, I'm getting always score 0.1 on developer PC only. No matter it's localhost or normal address, it's always 0.1. If i try from other pc/mobile/tablet works like a charm It's a bit pain to test then dev pc is always "bot" 🤕

bankeris avatar Aug 29 '18 07:08 bankeris

If you are running automated tests in a staging/development environment, I would suggest creating a mock service for reCAPTCHA. I updated the various RequestMethod classes to accept a $siteVerifyUrl parameter meaning you can override the default location. Alternatively you could also create a RequestMethod class that just returns a hardcoded response.

rowan-m avatar Aug 29 '18 11:08 rowan-m

Sort of same problem here. Programmed ReCaptcha v3 into my own build multitenant framework and started implementing ReCaptcha for websites.. yesterday it all worked fine; 0.9 for me typing and filling contact form, etc.. But today I suddenly, and always, get a 0.1 score.. even when using my iPhone instead of my laptop (same wifi IP), or disabling the wifi.. (mobile provider IP) and this is when clearly showing i'm human by extensively using the mouse and navigating the website.. typing in the forms instead of using the browsers autofill, but nothing helps getting the score up..

No, I didn't change anything in the code between yesterday and today.. And it worked perfectly yesterday.. so something happened overnight at google? (additional scoring or tests or something?)

One other thing I noticed yesterday; when there's a login form at the startpage, and one hasn't got enough navigating on the website (because the only thing to do is fill in the login form), and the browser uses autofill, the score also always returns 0.1.. so really not workable I would say..

I'm clearly human and filling in forms by hand, moving mouse, etc, but still I get a 0.1 ?!?

update; I normally use Edge, but just tried with Chrome to no avail.. still the same result.. 0.1 score..

update; just used a phone from another person; works fine.. so I suggest putting some kind of feature in the admin screen to 'unblock/reset' a certain visiting ip address score if needed.. so when a costumer comes to me with the message 'I have someone who cant use my contact form on the website', I have the possibility to 'reset' the build up score of a website visitor.. because really.. if this happens to legit customers of my costumers than I have a big problem.. and meanwhile I cant do anything now! i'm implementing ReCaptcha in all my costumers websites, and cant do anything at the moment really being a bot who isn't a bot.. :( So I do not feel confident implementing this ReCaptcha v3 solution is the right thing to do at the moment..

marcotibben avatar Nov 15 '18 11:11 marcotibben

I am seeing this when I try on Safari, but Chrome returns 0.9. Also testing on localhost. Could localhost be the problem here?

ykcai avatar Nov 19 '18 15:11 ykcai

I've never used localhost; so for me that's not the problem.. it feels more like google is putting some data together overnight and does some calculations, and after those decides that i'm an abuser filling in those forms with a same kind of message (for my costumers, saying ReCaptcha is activated on their website)..

I've deleted the key and created a new key, and now i'm a 0.9 again..

marcotibben avatar Nov 20 '18 14:11 marcotibben

Scores you see returned on development machines are not reflective of production. If you need a predictable score returned in a given environment, then you should absolutely mock the service to return a configurable score. To determine where to set appropriate thresholds to act on the score, you need to look at your dashboard in the admin interface for your site and see the spread of scores being returned.

The score returned is also only an indicator of confidence in that request. If it's below your threshold then you need to determine the appropriate response. For example, if it's just viewing a product page maybe you want to rate limit the user but if it's something like completing a purchase maybe you want to add an SMS verification before letting the user continue.

rowan-m avatar Nov 20 '18 15:11 rowan-m

Well its happening again. 2 weeks in production and suddenly ReCaptcha keeps returning 0.1 for most of the requests even genuine ones. Customers are complaining. Apparently the machine learning is not good enough yet. I cannot keep creating new keys, just to keep the ReCaptcha performance up to scratch. V3 doesn't seem ready yet. Also I use actions, but most of the sites do not pick the actions up in the admin panel. Some do, but then just one action of the 4 I use.. This is not usable sadly.. It looked very promising, but its not delivering at the moment.

marcotibben avatar Dec 02 '18 21:12 marcotibben

Marco, please send your public site key and a url on which you're experiencing trouble to [email protected] so we can take a look.

jasonfedor avatar Dec 03 '18 16:12 jasonfedor

I have implemented reCAPTCHA v3 in our application and we are looking to move this into production soon. I am, however, concerned about some of the scores it returns and a handful of the comments I have read online about it. The comment above from @marcotibben in particular:

One of the points I am concerned with is the ability to test failures. There doesn’t appear to be a lot of advice in the official docs on how to test this accurately, so what I done was change my User Agent to “Googlebot” and then call the reCAPTCHA API. Initially this returned a failure with a score of 0.1, which is below our pre-defined threshold of 0.5 and would denote bot behaviour. However, now when I do this it returns a score of 0.9 and sometimes 0.999999999. This leads me to believe that Google is in some way whitelisting our staging server IP and returning an acceptable score regardless of what we do.

It would obviously be bad for bots to get through, but I’m more concerned about valid users having a score of 0.1 which was reported on this Github issue a day ago.

Can you advise on recommended testing procedures and how we can go about getting some confidence in this before we think about deploying this to production?

crmpicco avatar Dec 04 '18 02:12 crmpicco

it was bad for a couple of days, but it seems to have restored now and results are good again. Maybe it's part of the machine learning process?!?

I've made changes to my application in that I don't solely rely on recaptcha's opinion anymore.. bad score resulted in not sending e-mail from contact forms. Now I take the opinion of ReCaptcha in advice and thén decide on some other factors in the message if it's spam or not.. and I log all contact, replies, and failures from recaptcha in my database, so learn where it works and doesn't to improve my application.. I was hoping google would do all the work ;)

I sure hope ReCaptcha gets more reliable, so I can fully rely on it again, because the idea is great..

marcotibben avatar Dec 05 '18 11:12 marcotibben

I have this problem too. recaptcha returns always 0.1 on chrome. but everything is ok in firefox. I tested it a lot in chrome and I think it returns 0.1 for this reason.

But the main question is this: if recaptcha analysis my actions and blah blah, why he thinks I'm a robot?

mostafaznv avatar Dec 17 '18 19:12 mostafaznv

Seems the same problem still occur. How can you could deliver to public release with such major issue ?

doekia avatar Jan 16 '19 20:01 doekia

I can confirm this issue which the score keeps on 0.1 even user didn't visit our page before. After regen the sitekey it return score 0.9, that's ok,but i don't think keep changing the sitekey is a good idea. Any advice will be appreciated

justid avatar Jan 23 '19 04:01 justid

I'd be quite keen to read some reviews or blog posts on someone who has successfully integrated reCAPTCHA v3 and have released it to production with a positive outcome. It does appear that people are reluctant to deploy this into a production environment at the moment, certainly, I am one of those people.

crmpicco avatar Jan 23 '19 04:01 crmpicco

I will using reCAPTCHA v3 into our production environment, and after reading the doc repeatedly,I found these text maybe important:

reCAPTCHA works best when it has the most context about interactions with your site, which comes from seeing both legitimate and abusive behavior

And I will follow this lead in code, I will update the progress as soon as possible, our production has very large network flow and i think it's good for recaptcha's analytic model

justid avatar Jan 23 '19 04:01 justid

can anyone help me for adding recpatcha v3

neha425 avatar Feb 20 '19 07:02 neha425

I feel like going back to V2 if the V3 issue is not resolved for localhost. It's frustrating to test and fix if there is suddenly a low score. At least with V2 you can make sure you have a high score.

andrebruton avatar Mar 08 '19 05:03 andrebruton

I have the same issue in my site. After a dramatic climbing of suspicious rate someday, most results become 0.1.

Is there a mechanism to train the model by site or by key? If yes, maybe the unsupervised learning mechanism needs to be improved.

maxex0000 avatar Mar 21 '19 09:03 maxex0000

+1 , I load recaptcha v3 from a hidden plain web page in my iOS App, and the score drop from 0.9 until 0.1. Any idea to solve my problem ? 🤔

chxj1992 avatar Mar 22 '19 06:03 chxj1992

bumped into this today as well. We switched to www.recaptcha.net recently. One of our employees was getting 0.1 scores, even though she was logged-in on Chrome, no incognito or any crazy browser extensions etc... Weird.

gingerlime avatar Mar 27 '19 09:03 gingerlime

I'm having the same issue. I'm trying to test a code on my server and google keeps returning a low score, saying that I'm a bot. I really like v3 but, if I have to, I'll have to use v2.

CodemasterJamal avatar Apr 03 '19 21:04 CodemasterJamal

I'm having this issue as well. May be a little different I'm not sure.

When I log in using autofill on my form, google always gives me a 0.1 score. If I wait about 10 seconds after autofill I always get a 0.7 score. If I don't wait, but change something in the autofilled field, I get a 0.7 score as well.

I assume this isn't supposed to happen though? Many people would use autofill to login..

mennokreukniet avatar Apr 11 '19 12:04 mennokreukniet

One page on my site nearly always returns 0.1 for me. Another page returns 0.7 or 0.9. I am at a loss.

ltctech avatar Apr 18 '19 21:04 ltctech