aws-google-auth icon indicating copy to clipboard operation
aws-google-auth copied to clipboard

ERROR:root:'NoneType' object has no attribute 'get'

Open LeRocque opened this issue 9 months ago • 29 comments

Attempting to use the tool this morning is resulting in

Screenshot 2024-05-01 081139

Is anybody else having this issue?

Thanks!

LeRocque avatar May 01 '24 12:05 LeRocque

I've been digging into this issue a bit (I work with @LeRocque) to just poke around at the responses this tool is getting back from Google.

Using our identity provider and secret provider to build the URL in this code

In google.Google

@property
def login_url(self):
    return self.base_url + "/o/saml2/initsso?idpid={}&spid={}&forceauthn=false".format(
        self.config.idp_id, self.config.sp_id)

Does indeed produce a valid URL allow for my federated Google identity to provide access to AWS. If I was already logged into my corporate Google account, it routes me directly to the AWS console. If I'm not logged into said account, I'm dropped to the normal Google account selection page.

I'm currently analyzing the issue from the perspective of the first page that URL loads (probably not reflective of what I'm seeing in foreground testing) is not the page this tool is expecting to receive.

From what I currently understand about BeautifulSoup (never used it before today), the listed stack trace shows an inability to pull a named 'continue's value. More concerning though, is that if you comment this out or just feed it some value, the subsequent line fails

In google.Google.do_login form = first_page.find('form', {'id': 'gaia_loginform'})

GAIA is Google's login and identity service (correct?) and there are no search hits on the text 'gaia' in the page I'm getting back at all.

I'm at a bit of a loss as to what to look into next here, so some input from a main dev would be greatly appreciated. Possibly worth noting here is that this issue started on May 1st, so a month transition (yes, we paid our bill). Likely nothing, but it's a more suspicious date of occurrence than most others.

lazerous42 avatar May 01 '24 19:05 lazerous42

It seems Google has changed the workflow and all the pages are different. I have hacked together enough to successfully login with captcha and TOTP MFA. See #281

mestuddtc avatar May 01 '24 21:05 mestuddtc

Same issue here, used to work just fine and all of a sudden started failing today. @mestuddtc I pulled down your fix and the process gets me to captcha but it always fails further on. Failure HTML look like that:

image

Looks like captcha field is not getting the text it's supposed to get?

fyodor-roam avatar May 02 '24 03:05 fyodor-roam

It seems Google has changed the workflow and all the pages are different. I have hacked together enough to successfully login with captcha and TOTP MFA. See #281

@mestuddtc I have tweaked your change to add support for the handle_dp method. I have raised the PR against your branch to simplify the process of delivering both. I assume the simplest thing to do would be for you to fast-forward your branch to the tip of mine which will close my PR and consolidate both fixes into your PR, but I leave that up to you.

https://github.com/mestuddtc/aws-google-auth/pull/3 <- my tweak of your PR

jonseymour avatar May 02 '24 06:05 jonseymour

Thank you everyone! I implemented your fix @jonseymour. It is working for me now.

LeRocque avatar May 02 '24 11:05 LeRocque

This fix is partially(?) working for me. I am getting through the password now but getting stuck on the captcha. Before this issue/fix, when I failed the captcha, it would give me an "invalid credential" type error. Now I am getting "Something went wrong - Could not find SAML response, check your credentials or use --save-failure-html to debug.", as shown in the pic. Is this the new "you failed captcha error"? or is this something else?

To implement this fix I replaced my "google.py" with the "google.py" from jonseymour's PR above.

image

Some people in my company have had success replacing their "google.py". The difference we've noticed is the people it works for have their installs in something like: /Unbuntu/usr/local/lib/python3.10/dist-packages/aws-google-auth While the people it doesn't work for are more: \Ubuntu\home\[username]\.local\lib\python3.10\site-packages\aws_google_auth

LukeMooney avatar May 02 '24 12:05 LukeMooney

To add to @LukeMooney's comment. It also seems to only work if you're using a MFA method of an authenticator token or by opening the Gmail app on your phone + clicking confirm. SMS is still broken

LeRocque avatar May 02 '24 14:05 LeRocque

@mestuddtc didn't work for me @jonseymour still have problems:

Something went wrong - Could not find SAML response, check your credentials or use --save-failure-html to debug

I was able to log in with @jonseymour code but with a little change:

https://github.com/juancarlosm/aws-google-auth/commit/fe70a3b6877818641a228e643e27ecc6f0d742ad

Now is working form me Hope that helps!

juancarlosm avatar May 02 '24 14:05 juancarlosm

@juancarlosm fix worked for me.

LukeMooney avatar May 02 '24 14:05 LukeMooney

Thanks @juancarlosm and @jonseymour -- officially saved my workday.

VicerExciser avatar May 02 '24 15:05 VicerExciser

I'm pulling fixes into my PR and fork.

If you have problems, run with the --save-saml-flow option. That outputs the request data and responses, and you can try debugging what's missing. The MFA methods I don't use aren't implemented. You should get an exception with the name of the unimplemented method. Thanks @jonseymour for implementing handle_dp. That solved some of my coworkers failures.

mestuddtc avatar May 02 '24 15:05 mestuddtc

Thanks @juancarlosm and @jonseymour -- officially saved my workday.

Right, this basically was my workday yesterday!

mestuddtc avatar May 02 '24 15:05 mestuddtc

I have taken the latest google.py from https://github.com/mestuddtc/aws-google-auth/blob/master/aws_google_auth/google.py but still it did not fix the issue for me. My installation path: /usr/local/Cellar/aws-google-auth/0.0.38_10/libexec/lib/python3.12/site-packages/aws_google_auth Here is the error: image

ismail0352 avatar May 03 '24 08:05 ismail0352

@ismail0352 that file doesn't have either of the fixes above. Try adding them manually.

LukeMooney avatar May 03 '24 12:05 LukeMooney

Did anyone else start getting "Match the number" MFA prompt after clicking on "it's me" verification? now I have 33% of chance to go through the auth.

sungmincs avatar May 05 '24 06:05 sungmincs

UPDATE: 7th May

It just started working for some reason.... very weird!


Is there a fix for this yet? I still get

ERROR:root:'NoneType' object has no attribute 'get' Traceback (most recent call last): File "/opt/homebrew/Cellar/aws-google-auth/0.0.38_10/libexec/lib/python3.12/site-packages/aws_google_auth/__init__.py", line 79, in cli process_auth(args, config) File "/opt/homebrew/Cellar/aws-google-auth/0.0.38_10/libexec/lib/python3.12/site-packages/aws_google_auth/__init__.py", line 243, in process_auth google_client.do_login() File "/opt/homebrew/Cellar/aws-google-auth/0.0.38_10/libexec/lib/python3.12/site-packages/aws_google_auth/google.py", line 259, in do_login passwd_challenge_url = 'https://accounts.google.com' + form.get('action') ^^^^^^^^ AttributeError: 'NoneType' object has no attribute 'get'

when I try to run the aws-google-auth -u ...

decidev22 avatar May 06 '24 06:05 decidev22

that file doesn't have either of the fixes above.

My master branch has all the fixes on it

image

That's in do_login_old, which means do_login chose the old workflow based on the login page google served. You'd have to use --save-saml-flow and debug the auth workflow google is serving to you.

mestuddtc avatar May 08 '24 19:05 mestuddtc

Hello, I'm also having the 'NoneType' error, however --save-saml-flow doesn't output/save anything. Does anybod know why?

joselitodasilva avatar May 23 '24 10:05 joselitodasilva

I tried the solution provided by @juancarlosm and @jonseymour. Also, your master branch @mestuddtc but still having the same issue

ERROR:root:'NoneType' object has no attribute 'get'
Traceback (most recent call last):
  File "/usr/local/Cellar/aws-google-auth/0.0.38_11/libexec/lib/python3.12/site-packages/aws_google_auth/__init__.py", line 79, in cli
    process_auth(args, config)
  File "/usr/local/Cellar/aws-google-auth/0.0.38_11/libexec/lib/python3.12/site-packages/aws_google_auth/__init__.py", line 243, in process_auth
    google_client.do_login()
  File "/usr/local/Cellar/aws-google-auth/0.0.38_11/libexec/lib/python3.12/site-packages/aws_google_auth/google.py", line 218, in do_login
    self.cont = first_page.find('input', {'name': 'continue'}).get('value')
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'get'

An error occurred (ExpiredToken) when calling the GetCallerIdentity operation: The security token included in the request is expired

vrathore18 avatar May 23 '24 23:05 vrathore18

@vrathore18 Are you sure you're using the new code? The stack trace does not line up with it. It looks much closer to the old master branch without the fixes.

mestuddtc avatar May 24 '24 14:05 mestuddtc

@vrathore18 - make sure you have the version of @mestuddtc's branch that includes d363cad02. I was happily using the slightly earlier revision I submitted until today, then started to experience the issues fyodor-roam reported. When I upgraded to d363cad02 which includes @juancarlosm's fixes, it started working again for me.

Prior to that time, the captcha request was silently failing and leaving the login sequence at the captcha submission stage, although it took some debugging to workout that was what was happening.

jonseymour avatar May 25 '24 07:05 jonseymour

Started getting the issue today again. Can confirm that d363cad solves it for me.

fyodor-roam avatar May 26 '24 23:05 fyodor-roam

Guys I applied the recent fix from @mestuddtc's branch and I am getting a different error now

Google Password: 
Please visit the following URL to view your CAPTCHA: <captcha-url>
Captcha (case insensitive):
Something went wrong - Could not find SAML response, check your credentials or use --save-failure-html to debug.

Can anyone be kind enough to create a docker image with the fix and share the public image here? That would be very helpful

syedkumail95 avatar Jun 03 '24 11:06 syedkumail95

Guys I applied the recent fix from @mestuddtc's branch and I am getting a different error now

Google Password: 
Please visit the following URL to view your CAPTCHA: <captcha-url>
Captcha (case insensitive):
Something went wrong - Could not find SAML response, check your credentials or use --save-failure-html to debug.

Can anyone be kind enough to create a docker image with the fix and share the public image here? That would be very helpful

I used the --save-failure-html flag to see what is the issue and this is what the html was like (attaching screenshot) google-html-image

syedkumail95 avatar Jun 03 '24 14:06 syedkumail95

Guys I applied the recent fix from @mestuddtc's branch and I am getting a different error now

Google Password: 
Please visit the following URL to view your CAPTCHA: <captcha-url>
Captcha (case insensitive):
Something went wrong - Could not find SAML response, check your credentials or use --save-failure-html to debug.

@syedkumail95, independently of this issue, months ago, I experienced this prompt unexpectedly. If I initiated the login manually from an incognito window and selected the option I normally use, it didn't happen again when aws-google-auth was next called and I changed nothing else, indicating that this manual intervention caused a change of state on the Google side. So, worth trying that. It might be that now that Google has been rolling out passkey support, it is providing users with an opportunity to select it, though, of course, aws-google-auth may or not support it directly (IDK).

jonseymour avatar Jun 03 '24 23:06 jonseymour

Guys I applied the recent fix from @mestuddtc's branch and I am getting a different error now

Google Password: 
Please visit the following URL to view your CAPTCHA: <captcha-url>
Captcha (case insensitive):
Something went wrong - Could not find SAML response, check your credentials or use --save-failure-html to debug.

@syedkumail95, independently of this issue, months ago, I experienced this prompt unexpectedly. If I initiated the login manually from an incognito window and selected the option I normally use, it didn't happen again when aws-google-auth was next called and I changed nothing else, indicating that this manual intervention caused a change of state on the Google side. So, worth trying that. It might be that now that Google has been rolling out passkey support, it is providing users with an opportunity to select it, though, of course, aws-google-auth may or not support it directly (IDK).

@jonseymour thanks alot man. It worked :D

syedkumail95 avatar Jun 04 '24 07:06 syedkumail95

Unfortunately, the fix from #281 stopped helping for me just today. It helped before, but now I'm getting a saml issue:

Captcha (case insensitive): courazie
ERROR:root:SAML lookup failed, storing failure page to 'saml.html' to assist with debugging.
Something went wrong - Could not find SAML response, check your credentials or use --save-failure-html to debug.

Saml response image

ezloj avatar Jun 06 '24 08:06 ezloj

Hm never mind. Looks like this suggestion (just above my comment) fixes it: https://github.com/cevoaustralia/aws-google-auth/issues/280#issuecomment-2146282333

ezloj avatar Jun 06 '24 09:06 ezloj

It seems Google has changed the workflow and all the pages are different. I have hacked together enough to successfully login with captcha and TOTP MFA. See #281

Thanks @mestuddtc , It's worked for me

manhtu57 avatar Jun 13 '24 07:06 manhtu57