keepass2-haveibeenpwned
keepass2-haveibeenpwned copied to clipboard
andrew-schofield
Returned status: Forbidden
I received this error when running the plugin, after clicking OK in the settings prompt for the plugin. I haven't previously used the plugin, so I do not know if this is a temporary issue, but https://haveibeenpwned.com/ is working. I believe it could be an HTTP 403.
Let me know if I can provide more relevant information.
happens to me aswell, tried it for the first time today because of the "Collection #1" list
I fired up fiddler to see what the response was and it appears that the plugin has breached the acceptable use policy (html returned below) but it isn't apparent if this is a rate limiting issue or if it is too many requests from single IP
<head>
<title>Request Blocked</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body>
<h1>You have been blocked from accessing this resource on Have I Been Pwned.</h1>
<p>This may be due to violating one or more of <a href="https://haveibeenpwned.com/API/v2#AcceptableUse">the acceptable use terms of the API</a>.</p>
<p>It may also be due to your traffic patterns being similar to other users who may have violated the acceptable use terms.</p>
<p>Tips to avoid requests being blocked include:</p>
<ol>
<li>Stick well within the published rate limit</li>
<li>Don't distribute requests over multiple IP addresses in an attempt to circumvent the rate limit</li>
<li>Only query the email addresses of people who have a reasonable expectation that you should do so</li>
<li>Avoid prolonged querying of the API over an extended period of time</li>
</ol>
<div class="cf-error-details cf-error-1020">
<h1>Access denied</h1>
<p>This website is using a security service to protect itself from online attacks.</p>
<ul class="cferror_details">
<li>Ray ID: 49a780168d393542</li>
<li>Timestamp: 2019-01-17 08:37:31 UTC</li>
<li>Your IP address: 79.68.4.48</li>
<li class="XXX_no_wrap_overflow_hidden">Requested URL: haveibeenpwned.com/api/v2/breachedaccount/Nathan </li>
<li>Error reference number: 1020</li>
<li>Server ID: FL_21F109</li>
<li>User-Agent: KeePass HIBP Checker/2.40.0.0</li>
</ul>
</div>
</body>
</html>```
Having checked the Pwnd Password docs here I don't believe its anything to do with rate limiting as that should return a 429 but instead I'm seeing a 403
The plugin should always be rate limited in its requests to the API, so I wonder if it had accidentally triggered the "multiple IP addresses" check because obviously there will be a lot of requests from different IPs. The plugin sends a versioned user agent string too, so I might reach out to Troy to find out what's going wrong.
Could this be due to the sheer number of checks from a single IP? When I tried running it, it went through ~half my username list, then started getting an error for every entry, even if I waited for several seconds between attempts. Also - it's impossible to gracefully cancel plugin execution is this case, had to terminate KeePass process to abort.
One more thing I'd noticed: if progress indicator is to be believed, it definitely checks faster then once every 1.6s (I'd say, at least couple entries per second).
I was accessing the API manually earlier and got the same forbidden page. I think there is something wrong with hibp's API for now. Accessing other endpoints (pastes. pwned-pws) are accessible though.
I also tweeted haveibeenpwnd about it to make sure.
Although the problem might be API related, there should be the possiblity to cancel the process. These error messages pop up over and over again ...
Just ran site/service and username checks and had no issues myself. Either it's a temporary hiccup that resolved or my username count is severely low compared to yours.
I can reproduce the error when checking site or password by selecting the option "Check all supported breaches" without this option checked it runs successfully.
When trying to run the check via username even without the "Check all supported breaches" I'm getting the forbidden error but it appears that it is trying to get breaches so guessing its related.
Same here Unable to check haveibeenpwned.com (return Status: Forbidden) See https://httpstatuses.com/403. Should this plugin be updated with credentials to authenticate against haveibeenpwned.com?
Even more anoying is that you have to dismiss the dialog for every single entry in your keepass file. That kept me busy for 15 minutes. :(
I'm stuck on this as well, under Ubuntu (see https://github.com/andrew-schofield/keepass2-haveibeenpwned/issues/58 ) ... Hint: you can close the progress dialog, and close KeePass to stop the dialog flood...
Same Error only with usernames @NathanGloyn
Having checked the Pwnd Password docs here I don't believe its anything to do with rate limiting as that should return a 429 but instead I'm seeing a 403
I'm not sure where to get the Error code 403 but this is about: no user agent: https://haveibeenpwned.com/API/v2#UserAgent
Same Error only with usernames @NathanGloyn
Having checked the Pwnd Password docs here I don't believe its anything to do with rate limiting as that should return a 429 but instead I'm seeing a 403
I'm not sure where to get the Error code 403 but this is about: no user agent: https://haveibeenpwned.com/API/v2#UserAgent
That shouldn't be the issue, as the plugin always sets the user agent appropriately https://github.com/andrew-schofield/keepass2-haveibeenpwned/blob/d78ade54c0710f4e1076f34a6a675a833f4f2515/HaveIBeenPwned/HaveIBeenPwnedExt.cs#L46 However if this is being stripped from the request for whatever reason, there's not much I can do.
However if this is being stripped from the request for whatever reason, there's not much I can do.
It's not stripped. Proper user-agent showed in cloudflare response.
Also same error happens for me by test link (from api page) opened in browser.
However if this is being stripped from the request for whatever reason, there's not much I can do.
It's not stripped. Proper user-agent showed in cloudflare response.
Also same error happens for me by test link (from api page) opened in browser.
Yeah, the direct link returns a 403 for me too.
Requests to the breaches and pastes APIs are limited to one per every 1500 milliseconds each from any given IP address (an address may request both APIs within this period).
As mentioned on the Api-Website
And you have already added Delay but only for usernameChecks and not for the other checks. Maybe this causes this issue.
https://github.com/andrew-schofield/keepass2-haveibeenpwned/blob/d78ade54c0710f4e1076f34a6a675a833f4f2515/HaveIBeenPwned/BreachCheckers/HaveIBeenPwnedUsername/HaveIBeenPwnedUsernameChecker.cs#L102
Requests to the breaches and pastes APIs are limited to one per every 1500 milliseconds each from any given IP address (an address may request both APIs within this period).
As mentioned on the Api-Website
And you have already added Delay but only for usernameChecks and not for the other checks. Maybe this causes this issue.
Line 102 in d78ade5
await Task.Delay(1600);
Unless Troy has changed something and not updated the docs, I don't think this is the issue.
The API page specifically says
There is no rate limit on the Pwned Passwords API.
Additionally the site/service check only does 1 call to retrieve the entire list of breaches, and then compares entries locally.
I have installed latest release today and can confirm that all of the searches are working correctly (regardless of if "Check all breaches" is checked or unchecked 👍
Unfortunately not for me:
Update reinstalled installed today, and with both ways, checked and uncheked
I just check with Keepass 2.41 and the plugin 1.3.1, but the issue is not resolved ("Returned status: Forbidden")
Right, I've fired off an email to Troy about this. Hopefully he can offer some insight into why this is happening.
I just got ran into this last night, here's what shows up when I manually attempt to query the API (example URL)
You have been blocked from accessing this resource on Have I Been Pwned
This may be due to violating one or more of the acceptable use terms of the API or for not complying with the API specifications. It may also be due to your traffic patterns being similar to other users who may have violated the acceptable use terms.
Tips to avoid requests being blocked include:
- Stick well within the published rate limit
- Don't distribute requests over multiple IP addresses in an attempt to circumvent the rate limit
- Only query the email addresses of people who have a reasonable expectation that you should do so
- Avoid prolonged querying of the API over an extended period of time
- Clearly identify your app in the user agent string per the API docs.
If you believe your request meets these requirements and was still blocked, please send this entire response body along with any communication you send regarding the error.
Access denied
This website is using a security service to protect itself from online attacks.
- Ray ID: 4a5297ee9b01c99f
- Timestamp: 2019-02-07 02:58:12 UTC
- Your IP address: 75.172.3.60
- Requested URL: haveibeenpwned.com/api/v2/breachedaccount/[email protected]
- Error reference number: 1020
- Server ID: FL_28F150
- User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36
So I've done a bit more investigation in my case, and the plugin is still popping up an error message saying Unable to check haveibeenpwned.com (returned Status: Forbidden).
All I have to do to reproduce it is check for breaches based on username and uncheck all boxes.
The error page from my previous comment seems to have been caused by the haveibeenpwned API rejecting requests from web browser user agents (as is documented in the API docs).
The above API URL does work over curl though (with the user agent KeePass HIBP Checker/1.3.1).
If you can reliably reproduce the error from within keepass, can you try capturing the error response by using fiddler (https://www.telerik.com/fiddler) or a similar tool?
Troy needs to see the exact response returned by cloudflare in order to debug this.
You're right that using a browser is not a valid test, as cloudflare rejects this based on the user agent, but it is interesting that the request fails from within keepass, but works via curl.
I haven't been able to reproduce the error on any of my systems (using at least 3 different IP addresses from different locations), either using the plugin, or with postman etc.
Downloaded and installed and "learned" Fiddler. Question
GET https://haveibeenpwned.com/api/v2/breachedaccount/****** HTTP/1.1
User-Agent: KeePass HIBP Checker/2.41.0.0
Accept: application/json
Host: haveibeenpwned.com
Cookie: __cfduid=dd4a76c0ce61505da0a219e7c1d168ae01549621013
Connection: Keep-Alive
First answer:
HTTP/1.1 403 Forbidden
Date: Fri, 08 Feb 2019 10:20:12 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
Cache-Control: max-age=15
Expires: Fri, 08 Feb 2019 10:20:27 GMT
X-Frame-Options: SAMEORIGIN
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
X-Content-Type-Options: nosniff
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Server: cloudflare
CF-RAY: 4a5d5cc34e15c4b0-DUS
Content-Length: 1951
<!DOCTYPE html>
<head>
<title>Request Blocked</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body>
<h1>You have been blocked from accessing this resource on Have I Been Pwned</h1>
<p>This may be due to violating one or more of <a href="https://haveibeenpwned.com/API/v2#AcceptableUse">the acceptable use terms of the API</a> or for not complying with <a href="https://haveibeenpwned.com/API/v2">the API specifications</a>. It may also be due to your traffic patterns being similar to other users who may have violated the acceptable use terms.</p>
<p>Tips to avoid requests being blocked include:</p>
<ol>
<li>Stick well within the published rate limit</li>
<li>Don't distribute requests over multiple IP addresses in an attempt to circumvent the rate limit</li>
<li>Only query the email addresses of people who have a reasonable expectation that you should do so</li>
<li>Avoid prolonged querying of the API over an extended period of time</li>
<li>Clearly identify your app in the user agent string <a href="https://haveibeenpwned.com/API/v2#UserAgent">per the API docs</a>.</li>
</ol>
<p>If you believe your request meets these requirements and was still blocked, please send this entire response body along with any communication you send regarding the error.</p>
<div class="cf-error-details cf-error-1020">
<h1>Access denied</h1>
<p>This website is using a security service to protect itself from online attacks.</p>
<ul class="cferror_details">
<li>Ray ID: 4a5d5cc34e15c4b0</li>
<li>Timestamp: 2019-02-08 10:20:12 UTC</li>
<li>Your IP address: *********</li>
<li class="XXX_no_wrap_overflow_hidden">Requested URL: haveibeenpwned.com/api/v2/breachedaccount/****** </li>
<li>Error reference number: 1020</li>
<li>Server ID: FL_48F23</li>
<li>User-Agent: KeePass HIBP Checker/2.41.0.0</li>
</ul>
</div>
</body>
</html>
Reconnect to provider some times to get new Ip adress -> Same answer
Start VPN it works
Thanks, I've forwarded the details on to Troy. It does look like cloudflare using an over enthusiastic IP range blocker that is causing this.