authentication error

[vzlgdu]

Hi, starting from this morning I'm receiving authentication errore in my python scritp. It's running with no problem starting from many months ago...

What's happening?

Thanks

Guido

[twrecked]

Which version are you on?

[vzlgdu]

0.8.0a5

[twrecked]

Can you try the latest release. Arlo changed their back end and broke our login code.

[vzlgdu]

With the latest release I receive this errors:

body-error=JSONDecodeError body-error=JSONDecodeError body-error=JSONDecodeError authentication failed

[twrecked]

Can you try these 2 things:

• Add mqtt_hostname_check=False to your PyArlo options.
• Remove your aarlo pickle files.

If that doesn't work can you turn on verbose debug, I need more information on where it's failing.

[njaouen]

Hi vzlgdu, Any update on your side ?

[m0urs]

Seems that I do have the same issue here also. Can no longer login with latest code. Enclosed my encrypted debug.log. It would be great if you have any chance to look at it and maybe find out what the issue could be. Let me know if I should test something. Thanks a lot. P.S: I already set "mqtt_hostname_check=False" but with no success.

Looks a bit like a Cloudflare issue? Think we had something similar also in the past?

debug.log

[twrecked]

It's definitely a cloud flare thing. If you leave for a few hours it might recover. I think the old code made cloudflare angry and they blocked it temporarily.

[twrecked]

You connection is stopping straight away. A couple of things to try:

• change the user agent, try using user_agent='linux'
• try logging in from a website using the pyaarlo account

I did notice I'm sending in an extra header - the transaction id one - but I don't think that is causing the issue, I can still connect from here with it.

[vzlgdu]

Hi, sorry for the delay... with your suggest (mqtt_hostname_check=False) it was ok for some days. Today I received the same error.

Adding "user_agent='linux'" the result is the same.

Accessing from web interface doesn't work and ends with "timeout request"

Accessing from ios app everything work.

[m0urs]

Even after waiting some days I am still unable to login from pyarlo. I can login via browser from the same IP address. Maybe we need to set some more values toegether with the request. Here is a sample what my browser is sending:

:authority: ocapi-app.arlo.com :method: POST :path: /api/startAuth :scheme: https accept: application/json, text/plain, / accept-encoding: gzip, deflate, br accept-language: de authorization:XXXXXXXXXXXXXXXXXXXXXXX content-length: 184 content-type: application/json cookie: _XXXXXXXXXXXXXXXXXXXXXXXXXXX dnt: 1 origin: https://my.arlo.com referer: https://my.arlo.com/ sec-ch-ua: "Microsoft Edge";v="113", "Chromium";v="113", "Not-A.Brand";v="24" sec-ch-ua-mobile: ?0 sec-ch-ua-platform: "Windows" sec-fetch-dest: empty sec-fetch-mode: cors sec-fetch-site: same-site source: arloCamWeb user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.42 x-user-device-automation-name: QlJPV1NFUg== x-user-device-id: 615cdfd1-0a03-4a91-91b3-12d21aeba713 x-user-device-type: BROWSER

What about these ones:

sec-ch-ua: "Microsoft Edge";v="113", "Chromium";v="113", "Not-A.Brand";v="24"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-site

[vzlgdu]

Hi, I'm trying some test (now accessing using amail as 2fa) and I'm receiving this error:

/usr/local/lib/python3.7/dist-packages/urllib3/connection.py:362: SubjectAltNameWarning: Certificate for myapi.arlo.com has no subjectAltName, falling back to check for a commonName for now. This feature is being removed by major browsers and deprecated by RFC 2818. (See https://github.com/shazow/urllib3/issues/497 for details.) SubjectAltNameWarning request-error=ValueError session start failed body-error=JSONDecodeError body-error=JSONDecodeError body-error=JSONDecodeError authentication failed

I'm using this in my script:

import ssl ssl.SSLContext.verify_mode = ssl.VerifyMode.CERT_OPTIONAL

This are my logon string:

arlo = pyaarlo.PyArlo( username='xxx',password='yyy', tfa_type='email',tfa_source='imap', synchronous_mode=True,save_state=False, tfa_host='imap.gmail.com', tfa_username='xxx', tfa_password='zzz')

Thanks

[m0urs]

Do we have any new information? Is it working again for somebody? I still have the same issue with Cloudflare although I did not try it now for severral days. So it does not seem to be reset by its own :-(

[twrecked]

Sorry, it's working for me here - both standalone and on home assistant.

Did you remove the session files? Or are you still stuck at the cloud flare piece?

[m0urs]

Yes, the error seems still to be the Cloudflare error as far as I can see in the debug file. Where should I find the session / pickle files? Did not find anything which looks like that? And, @vzlgdu , is it working again for you?

[twrecked]

Look for a file with this name session.pickle. It's where we store the authentication credentials.

[m0urs]

There is no such file. Maybe because I am not currently authenticated?

 $ grep -ir "session.pickle" .
./pyaarlo/cfg.py:        return self.storage_dir + "/session.pickle"
Binary file ./pyaarlo/__pycache__/cfg.cpython-38.pyc matches
Binary file ./pyaarlo/backup/__pycache__/cfg.cpython-38.pyc matches

[twrecked]

So. A few more things to try/check:

• Are you on the latest cloudscraper release?

pip3 install --update cloudscaper

• You could try a custom user_agent. Add this to the PyArlo, note the ! at the front. (I copied this from above, make sure it's correct.)

  user_agent='!Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.42'

[ninjafella]

Hello there,

I'm having the same issue of having cloudflare blocking me. I've tried using different 'user_agents' and making sure my cloudscraper was up to date but no luck. I've attached my logs bellow.

Whats perculiar is that home assistant using aarlo can connect fine and i'm having no issues with it on there.

arlo_debug.log

[m0urs]

 user_agent='!Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.42'

I updated cloudscraper and tried this user agent, but with no luck :-( I am using pyarlo with my own script, not sure if there is an issue with my script using pyarlo (although it worked fine since some weeks ago): https://github.com/m0urs/arlo-fhem/blob/master/arlo-fhem.py

[shissam]

 user_agent='!Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36 Edg/113.0.1774.42'

I updated cloudscraper and tried this user agent, but with no luck :-( I am using pyarlo with my own script, not sure if there is an issue with my script using pyarlo (although it worked fine since some weeks ago): https://github.com/m0urs/arlo-fhem/blob/master/arlo-fhem.py

overnight - my script continued to fail for approx 7 hours, after manual intervention (removed session.pickle and aarlo.pickle) it failed once, and then was successful.

I was comparing my script to your script and noted that ours are pretty much the same (args to PyArlo specifically using the sse backend and my agent is still linux). There is one other thing that my system has... in /etc/hosts (based on recommendations from past issues dealing with cloudfare) i have this entry:

104.18.30.98 ocapi-app.arlo.com
#104.18.31.98 ocapi-app.arlo.com

also cloudscraper is

pip list|grep -i scrap
cloudscraper      1.2.58 

[m0urs]

"ocapi-app.arlo.com" is already resolved here to 104.18.30.98 so an additional hosts entry should not do anything else. My cloudscraper is now updated to 1.2.71. I think it was 1.2.58 before updating this morning.

However I still have no pickle files. Where should these files be created??

[shissam]

"ocapi-app.arlo.com" is already resolved here to 104.18.30.98 so an additional hosts entry should not do anything else. My cloudscraper is now updated to 1.2.71. I think it was 1.2.58 before updating this morning.

However I still have no pickle files. Where should these files be created??

the command you want to run to manually find those pickle files (and remove them) is:

sudo find / -mount -iname \*.pickle -type f

these files are created in a default location relative to the current working dir for the process unless specified on the PyArlo call (see storage_dir=)

[shissam]

"ocapi-app.arlo.com" is already resolved here to 104.18.30.98 so an additional hosts entry should not do anything else. My cloudscraper is now updated to 1.2.71. I think it was 1.2.58 before updating this morning. However I still have no pickle files. Where should these files be created??

the command you want to run to manually find those pickle files (and remove them) is:

sudo find / -mount -iname \*.pickle -type f

these files are created in a default location relative to the current working dir for the process unless specified on the PyArlo call (see storage_dir=)

to be clear remove both pickle files, mine are:

~/iot/aarlo/session.pickle
~/iot/aarlo/aarlo.pickle

[m0urs]

I am unable to find these files ... so I guess it will not be created as Cloudflare does not let me go so far and refuses with the error before?

I did a downgrade to cloudscraper 1.2.58 but this did not change anything.

I still get this Cloudflare issue:

You do not have access to ocapi-app.arlo.com.

The site owner may have set restrictions that prevent you from accessing the site.

...

Error code: 1020

Ray ID: 7cff7a4e2b7a37d8

Country: DE

Data center: fra12

IP: 176.199.xx.yy

Timestamp: 2023-05-31 13:14:03 UTC

[twrecked]

Also look in /tmp/.aarlo but I suspect you are seeing other problems.

[m0urs]

Also look in /tmp/.aarlo but I suspect you are seeing other problems.

Thanks, that was the right location :-) I did delete both files there but still no login possible. And the files are NOT recreated ....

[shissam]

in my world... those files persist between logins, so i'd expect those files would be around from the last successful login. for those files to not be on your system anywhere would lead me to think there was never a successful login -- but that cannot certainly be the case for you. so "strange"

one other thought would be to move back to the mqtt backend - but perhaps that would not make a difference.

if it were me, i'd first try the mqtt backend, if that failed, i'd jump to another ocapi-app.arlo.com, and if that failed, I'd remove the ocapi-app.arlo.com completely from my /etc/hosts.

i'll also say this - i had this same problem on 18 MAY, and my resolution on that day was to git pull from pyaarlo which brought me up to the current version of pyaarlo. So far, in all cases, removing those pickle files was the common denominator in resolving my issues then and this morning.

[twrecked]

@shissam That's pretty much what happened to me!

The pickle files mean we can skip the whole Cloud Flare authentication and continue with a previously authenticated session, the they are good for up to 14 days, after which we log in again.

They work until they don't. And I think (but I don't have much evidence to back this up) when Arlo makes server changes they will invalidate the current ones and the cache sessions will fail. I thought the code would recover but I'll double check the logic, it could also be at that point we've made too many attempts and tripped something.

@m0urs If you're comfortable with an editor you can add the missing header lines and try that. Just look for x-user-device-type in the backend.py file. It appears twice (the code is in need of a tidy up).

[m0urs]

If you're comfortable with an editor you can add the missing header lines and try that. Just look for x-user-device-type in the backend.py file. It appears twice (the code is in need of a tidy up).

You mean I should add these headers?

sec-ch-ua: "Microsoft Edge";v="113", "Chromium";v="113", "Not-A.Brand";v="24" sec-ch-ua-mobile: ?0 sec-ch-ua-platform: "Windows" sec-fetch-dest: empty sec-fetch-mode: cors sec-fetch-site: same-site

If yes: I tried it but I got some issues because there are " and ; characters inside these header values which I guess, need to be masked? I did it this way:

            "x-user-device-type": "BROWSER",
            "sec-ch-ua": "\"Microsoft Edge"\;v=\"113\", \"Chromium\"\;v=\"113\", \"Not-A.Brand\"\;v=\"24\"",
            "sec-ch-ua-mobile": "?0",
            "sec-ch-ua-platform": "\"Windows\"",
            "sec-fetch-dest": "empty",
            "sec-fetch-mode": "cors",
            "sec-fetch-site": "same-site",
        }

So I tried this but I do get the following error:

May 31 16:12:29 raspberrypi02 arlo-fhem[6133]:   File "/opt/fhem/arlo-fhem/pyaarlo/backend.py", line 681
May 31 16:12:29 raspberrypi02 arlo-fhem[6133]:     "sec-ch-ua": "\"Microsoft Edge"\;v=\"113\", \"Chromium\"\;v=\"113\", \"Not-A.Brand\"\;v=\"24\"",
May 31 16:12:29 raspberrypi02 arlo-fhem[6133]:                                                                                                    ^
May 31 16:12:29 raspberrypi02 arlo-fhem[6133]: SyntaxError: unexpected character after line continuation character

If I do not mask the ; but only the " characters then I do get this error:

May 31 16:11:09 raspberrypi02 arlo-fhem[5837]:   File "/opt/fhem/arlo-fhem/pyaarlo/backend.py", line 681
May 31 16:11:09 raspberrypi02 arlo-fhem[5837]:     "sec-ch-ua": "\"Microsoft Edge";v=\"113\", \"Chromium\";v=\"113\", \"Not-A.Brand\";v=\"24\"",
May 31 16:11:09 raspberrypi02 arlo-fhem[5837]:                                    ^
May 31 16:11:09 raspberrypi02 arlo-fhem[5837]: SyntaxError: invalid syntax

What would be the correct syntax here?