forum icon indicating copy to clipboard operation
forum copied to clipboard

macOS session refresh loop when self hosting

Open alan-hkpanorama opened this issue 1 year ago • 28 comments

Describe the bug Stuck in a login loop on macOS desktop app, unable to sync with self hosted server. Error messages:

  • "Unable to Sync"
  • "Sync error: invalid login credentials"

To Reproduce Steps to reproduce the behavior: Login with username and password on macOS desktop app, after selecting custom sync server.

Expected behavior Sync notes without non-stop prompts to refresh session.

Screenshots Screenshot 2024-06-21 at 1 48 30 PM

Desktop:

  • OS: macOS Sonoma 14.5
  • Version: 3.194.0

Additional context I've done a fresh install of the self host sync server using Docker, with connections through Cloudflare. Fresh install of Standard Notes on macOS as well. iOS app works no problem and syncs well. Issue is similar to #3513 and the error seems to be server side, with the relevant logs here:

query: SELECT user.uuid AS user_uuid, user.version AS user_version, user.email AS user_email, user.pw_nonce AS user_pw_nonce, user.encrypted_server_key AS user_encrypted_server_key, user.server_encryption_version AS user_server_encryption_version, user.kp_created AS user_kp_created, user.kp_origination AS user_kp_origination, user.pw_cost AS user_pw_cost, user.pw_key_size AS user_pw_key_size, user.pw_salt AS user_pw_salt, user.pw_alg AS user_pw_alg, user.pw_func AS user_pw_func, user.encrypted_password AS user_encrypted_password, user.created_at AS user_created_at, user.updated_at AS user_updated_at, user.locked_until AS user_locked_until, user.num_failed_attempts AS user_num_failed_attempts FROM users user WHERE user.email = ? -- PARAMETERS: ["EMAIL_HIDDEN"]

[...]

-- PARAMETERS: ["f67e5c16cbc7886b7ff15c21390d5b49"]
query: SELECT session.uuid AS session_uuid, session.user_uuid AS session_user_uuid, session.private_identifier AS session_private_identifier, session.hashed_access_token AS session_hashed_access_token, session.hashed_refresh_token AS session_hashed_refresh_token, session.access_expiration AS session_access_expiration, session.refresh_expiration AS session_refresh_expiration, session.api_version AS session_api_version, session.user_agent AS session_user_agent, session.created_at AS session_created_at, session.updated_at AS session_updated_at, session.readonly_access AS session_readonly_access, session.version AS session_version, session.application AS session_application, session.snjs AS session_snjs FROM sessions session WHERE session.private_identifier = ? -- PARAMETERS: ["f67e5c16cbc7886b7ff15c21390d5b49"]
{"application":"Desktop-3.194.0","level":"error","message":"No cookies provided for cookie-based session token.","method":"POST","secChUa":""Not=A?Brand";v="99", "Chromium";v="118"","service":"auth:server","sessionUuid":"d38a1c66-c770-44a6-bf28-c1962e266bc6","snjs":"2.209.3","url":"/v1/items","userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) StandardNotes/3.194.0 Chrome/118.0.5993.54 Electron/27.0.0 Safari/537.36","userId":"986db3db-b74c-4c72-b232-3ff4ad78ce7f"}
query: SELECT revoked_session.uuid AS revoked_session_uuid, revoked_session.private_identifier AS revoked_session_private_identifier, revoked_session.user_uuid AS revoked_session_user_uuid, revoked_session.received AS revoked_session_received, revoked_session.created_at AS revoked_session_created_at, revoked_session.received_at AS revoked_session_received_at, revoked_session.user_agent AS revoked_session_user_agent, revoked_session.api_version AS revoked_session_api_version FROM revoked_sessions revoked_session WHERE revoked_session.private_identifier = ? -- PARAMETERS: ["f67e5c16cbc7886b7ff15c21390d5b49"]

alan-hkpanorama avatar Jun 21 '24 06:06 alan-hkpanorama

I also have same issue. Exactly the same steps to reproduce

ilfarpro avatar Jun 23 '24 21:06 ilfarpro

I just tested on Windows in a VM as well - same error messages appearing.

alan-hkpanorama avatar Jun 26 '24 11:06 alan-hkpanorama

I get this on Arch Linux using the AppImage as well.

self-hosted AppImage: 3.194.0

New Install: After logging in this infinite loop. Tested copying over ~/.config/Standard Notes from another machine that is working. Asks to reset key on initial launch. After it works fine on "new" install but now get this loop on "old" install.

taryujiai avatar Jun 30 '24 17:06 taryujiai

Turns out the workaround for now is to use the following version of standard notes server:

image: standardnotes/server:5c02435ee478b893747d3f9e41062aae12d7ff10

Worked for me, all is well again!

alan-hkpanorama avatar Jul 02 '24 02:07 alan-hkpanorama

Hi, there was an issue discussed on our Discord group regarding the authorization tokens for the docker image, so we apologize for the trouble. Indeed a temporary workaround is to make use of the older image mentioned by @alan-hkpanorama, though I've also raised the authorization token issue with our back end developer, and we'll do our best to keep you posted 🙏

effieeee avatar Jul 02 '24 15:07 effieeee

Turns out the workaround for now is to use the following version of standard notes server:

image: standardnotes/server:5c02435ee478b893747d3f9e41062aae12d7ff10

Worked for me, all is well again!

I confirm that this mitigation works. Thanks for pointing it out.

joaofl avatar Jul 02 '24 22:07 joaofl

I have the same issue and can also confirm that it works with this image: standardnotes/server:5c02435ee478b893747d3f9e41062aae12d7ff10

micudaj avatar Jul 09 '24 10:07 micudaj

First time standard notes user. I ran into this issue on my Windows desktop app when accessing a freshly-installed self-hosted standard notes instance installed on a Rocky Linux 9 server.

I edited the .env file and replaced:

image: standardnotes/server with image: standardnotes/server:5c02435ee478b893747d3f9e41062aae12d7ff10

Then I restarted the self-hosted instance by running:

docker compose down
docker compose pull && docker compose up -d

Then I logged out of my standard notes workspace on my Windows desktop app, then logged back in. Everything worked normally after that.

vlasky avatar Jul 22 '24 03:07 vlasky

Same thing happened to and I had some notes that I usually create on the go on my phone. But all of them are lost after syncing with the server after it as up again by using image 5c02435ee478b893747d3f9e41062aae12d7ff10.

As notes were quite important is there are way to restore them? And any suggestion how we can handle similar thing just to be safe if note is not syncing with server maybe we could take a backup locally and sync it with server afterwards?

Kirito70 avatar Aug 11 '24 19:08 Kirito70

After almost 3 months this issue is still happening. I was considering self hosting standard notes to move away from google keep, but this is not a great start.

ilteoood avatar Sep 07 '24 08:09 ilteoood

I was experiencing the same issue running the docker web app container connecting to my self-hosted server, but I resolved it by adding the appropriate COOKIE_DOMAIN parameter to my docker-compose.yml:

services:
  server:
    image: standardnotes/server
    env_file: .env
    container_name: server_self_hosted
    restart: unless-stopped
    ports:
      - 3000:3000
      - 3125:3104
    volumes:
      - ./logs:/var/lib/server/logs
      - ./uploads:/opt/server/packages/files/dist/uploads
    networks:
      - standardnotes_self_hosted
    environment:
      - COOKIE_DOMAIN=my.domain.com

If you don't have a domain name for your server, try localhost.

pete3n avatar Sep 10 '24 11:09 pete3n

Thanks for the fix , updated my .env file and no more logon issues running the latest docker image (Appimage , IOS & Android)

bobcqld53 avatar Sep 10 '24 20:09 bobcqld53

Adding the COOKIE_DOMAIN seems to have fixed of for me on both Fedora and Android....thank you so much!! Happily running the latest docker image again!!

cnyrick avatar Sep 15 '24 13:09 cnyrick

@pete3n Thank you, that environment variable fixed it for me.

This should make it into some documentation asap. I just freshly set up the selfhosted version and was seconds before giving up everything.

aeonoea avatar Sep 17 '24 22:09 aeonoea

Thanks for Your solution @pete3n! I think this should in particular be added to the .env.sample- and/or docker-compose.example.yml-file.

jelkose avatar Sep 18 '24 15:09 jelkose

I can confirm that the login loop is present on the Linux desktop app (flatpak) 3.195.1 and on the self hosted web app 3.195.3. You can see on the top right corner a little red circle with "unable to sync", and you get a login prompt every time it attempts to sync again --- which is very very often!

ALL is fixed when you add the appropriate COOKIE_DOMAIN env var. Thanks!!!

RamonAbudAlcala avatar Sep 24 '24 13:09 RamonAbudAlcala

The team needs to start treating the self-hosted setup as a first-class citizen instead of an afterthought

1fexd avatar Sep 27 '24 16:09 1fexd

Oh my, days and days trying to solve this. Why is this not in the documentation? Thanks for the solution!

Jeief73 avatar Oct 02 '24 17:10 Jeief73

Fixed for me too. I added COOKIE_DOMAIN=app.mydomain.com in .env app, docker-compose down && docker-compose up -d, and it's running ! Thank you. Maybe it should be fixed in the documentation ant in the .env template ?

badrow avatar Oct 28 '24 12:10 badrow

I'm getting the same error on my just installed self hosted server, and the proposed COOKIE_DOMAIN fix is not working. My issue is with the desktop macOS app only, I'm able to login successfully with the iOS app. I was running desktop version 3.195.12, which is the download link version as of today on the main Standard Notes download page at https://standardnotes.com/download, but apparently that is a pre-release version according to the GitHub releases page. I noticed the current iOS version is 3.194.11, so I downloaded the same desktop version but the issue persists. I even removed the COOKIE_DOMAIN entry, and the iOS still works, and desktop does not.

Robotronic26 avatar Nov 08 '24 03:11 Robotronic26

Found my own solution: updated the compose file from the default of image: standardnotes/server to image: standardnotes/server:latest along with the COOKIE_DOMAIN value and the desktop app works now.

Robotronic26 avatar Nov 08 '24 03:11 Robotronic26

Tried using this method and I still have the same problem.

From the Desktop app, when I try to create the user, after confirming the password, it returns a 500 error for http://myhost:3000/v1/users: Unfortunately, we couldn't handle your request. Please try again or contact our support if the error persists.

If I try to confirm the password again, it returns a 400 for http://myhost:3000/v1/users with the message: This email is already registered.

If I try to login if this user, it passes the login screen and then goes immediately to screen of the beginning of this post in a loop, returning a 401 this time for http://myhost:3000/v1/items.

Yes, I'm able to access this URL: http://myhost:3000/ No, there are no errors in the logs of all containers, they are all up and running.

hknoener avatar Jan 21 '25 23:01 hknoener

@hknoener I guess the cookie domain is still not correct in your install.

  1. What sync server address are you using when logging in?
  2. What did you set COOKIE_DOMAIN to?
  3. Can you check the response for the /login request when logging in? It will have a few set-cookie headers and you'll be able to see the cookie domains there. They need to match the sync server hostname, or its domain (so if your sync server is "api.domain.tld", then "api.domain.tld" or "domain.tld" will work for COOKIE_DOMAIN)

Furthermore, the HTTP 500 error indicates an error in the backend - you should check the server logs (logs/*.err).

piit79 avatar Feb 07 '25 14:02 piit79

@piit79 I'm inclined to believe you're right, even though I've tried everything that crossed my mind already.

  1. What sync server address are you using when logging in?

http://portainer-main:3000

  1. What did you set COOKIE_DOMAIN to?

- COOKIE_DOMAIN=portainer-main

  1. Can you check the response for the /login request when logging in? It will have a few set-cookie headers and you'll be able to see the cookie domains there. They need to match the sync server hostname, or its domain (so if your sync server is "api.domain.tld", then "api.domain.tld" or "domain.tld" will work for COOKIE_DOMAIN)

It's the same as I set:

Set-Cookie:
access_token_3eefe17f-190e-4c9d-a875-6cc55f241a26=YjQ4N2E3YTg5YWI0; HttpOnly;Secure; Path=/;Partitioned; SameSite=None; Domain=portainer-main; Expires=Thu, 12 Feb 2026 22:50:08 GMT;
Set-Cookie:
refresh_token_3eefe17f-190e-4c9d-a875-6cc55f241a26=MWE4NGIxZTYwZGZi; HttpOnly;Secure; Path=/v1/sessions/refresh;Partitioned; SameSite=None; Domain=portainer-main; Expires=Thu, 12 Feb 2026 22:50:08 GMT;

Furthermore, the HTTP 500 error indicates an error in the backend - you should check the server logs (logs/*.err).

I also checked all logs in all 4 containers running in the stack, found no errors.

hknoener avatar Feb 12 '25 17:02 hknoener

@hknoener Thanks for providing all the detail. You are right that the cookie domain is set correctly.

However, you are accessing the API via an unencrypted HTTP connection (not https), and the cookie is set with the Secure flag. According to the documentation, that means that

...the cookie is sent to the server only when a request is made with the https: scheme (except on localhost)...

Therefore, the cookie is set, but it's not sent back to the API server on subsequent requests. You will most probably need to set up a reverse proxy with HTTPS.

piit79 avatar Feb 12 '25 22:02 piit79

Sorry, @piit79 , being it a homelab, local only environment, going through all this trouble just to host this app is not worth it.

Is there a way to disable HTTPS requirements to run it locally? Else I just run it directly on my main machine and drop using Docker, it doesn't make sense to me.

hknoener avatar Feb 12 '25 22:02 hknoener

That's for you to decide, @hknoener. I don't have any involvement with the project, I'm just another guy who likes to self-host stuff, trying to help other similarly-inclined folk :)

I don't know if it's possible to disable the secure cookies, but I doubt it.

On the other hand, since you're already using docker to host Standard Notes, spinning up a reverse proxy container (e.g. nginx) should be a matter of minutes. You could even implement a more sophisticated solution like Pangolin that would make future reverse proxy configurations a breeze and give you some extra features like secure remote access via Wireguard.

piit79 avatar Feb 13 '25 10:02 piit79

@piit79 before anything, sorry for the wording. Your input has been helpful, it was not my intention to downplay it whatsoever.

Dealing with HTTPS/Reverse Proxy/External access is still a step in the future for my Homelab. To be honest, given my use, I don't even know if I will ever have to expose it to the outside thus having to deal with HTTPS for this app is not worth the trouble. I wanted to test it and for that I can just install it on my main machine and use the local environment already provided. If I feel inclined to use it, despite the alternatives I already use, your suggestion will be my first steps.

One thing that's not clear to me though: I've read this whole discussion already and there was no mention to HTTPS. Maybe this was already a given, but it was not clear to me that it was the case.

From the ones that solved this problem, is there anyone not using HTTPS?

hknoener avatar Feb 13 '25 11:02 hknoener

@hknoener
Hi! Did you manage to solve the issue with not using HTTPS. I am in the same boat with my homelab

bahnsterian avatar Jun 17 '25 23:06 bahnsterian

@hknoener Hi! Did you manage to solve the issue with not using HTTPS. I am in the same boat with my homelab

Hi, @bahnsterian . No, I didn't, unfortunately.

hknoener avatar Aug 08 '25 21:08 hknoener