Installation in docker?

[WAdama]

Hi all,

I tried to install syncstorage-rs via docker, but I can't get my head around it with this documentation.

For one it seems the prerequisite mozilla-rust-sdk is now google-cloud-rust, is that correct?

Maybe someone has already got this to run and can provide me with a more step by step documentation...

Thanks Ingo

┆Issue is synchronized with this Jira Task

[ictabc]

Same here, managed to compile the syncstorage-rs with a bit of a strugle, but after that just a big black hole. No good documentation, missing a lot of parts in the documentation to get things working. Guess this documentation is build by people that forgot the parts that other people need to get things running. Don't want to use the antiquated syncserver code again. And I'm not allowed to sync my data to the cloud, specially if it's in the US.

Found a docker image in the docker hub, but absolutely no data on how to use it. https://hub.docker.com/r/mozilla/syncstorage-rs

As nobody has documented anything yet on how to use it with docker compile or docker cli, totally lost on what to do. I'm not a noob, but but with current documentation it's impossible to do anything.

Maybe someone from the community can write something, how to use the docker container. Or point me to a page where it is described, at least better than the non information that's in the readme posted in this repository.

[jakobkukla]

I managed to get it working a few days ago. See my docker-compose.yml file down below. Don't forget to change the secrets, mysql credentials/urls and paths.

version: "3.8"

services:
  firefox-sync:
    image: mozilla/syncstorage-rs:0.12.4
    container_name: firefox-sync
    depends_on:
      - firefox-sync-syncstorage-db
      - firefox-sync-tokenserver-db
    environment:
      SYNC_HOST: 0.0.0.0
      SYNC_HUMAN_LOGS: 1
      SYNC_MASTER_SECRET: MY_SECRET
      SYNC_DATABASE_URL: mysql://MY_SYNC_MYSQL_USER:MY_SYNC_MYSQL_USER_PASSWORD@firefox-sync-syncstorage-db:3306/syncstorage
      SYNC_TOKENSERVER__ENABLED: "true"
      SYNC_TOKENSERVER__RUN_MIGRATIONS: "true"
      SYNC_TOKENSERVER__NODE_TYPE: mysql
      SYNC_TOKENSERVER__DATABASE_URL: mysql://MY_TOKEN_MYSQL_USER:MY_TOKEN_MYSQL_USER_PASSWORD@firefox-sync-tokenserver-db:3306/tokenserver
      SYNC_TOKENSERVER__FXA_EMAIL_DOMAIN: api.accounts.firefox.com
      SYNC_TOKENSERVER__FXA_OAUTH_SERVER_URL: https://oauth.accounts.firefox.com/v1
      SYNC_TOKENSERVER__FXA_METRICS_HASH_SECRET: MY_OTHER_SECRET
      # I don't really know what this is doing
      SYNC_TOKENSERVER__ADDITIONAL_BLOCKING_THREADS_FOR_FXA_REQUESTS: 2
    ports:
      - 5000:8000
    restart: always
  firefox-sync-syncstorage-db:
    image: mysql:5.7
    container_name: firefox-sync-syncstorage-db
    environment:
      MYSQL_ROOT_PASSWORD: MY_SYNC_MYSQL_ROOT_PASSWORD
      MYSQL_DATABASE: syncstorage
      MYSQL_USER: MY_SYNC_MYSQL_USER
      MYSQL_PASSWORD: MY_SYNC_MYSQL_USER_PASSWORD
    volumes:
      - path/to/appdata/firefox-sync/syncstorage-db:/var/lib/mysql
    ports:
      - 3306
    restart: always
  firefox-sync-tokenserver-db:
    image: mysql:5.7
    container_name: firefox-sync-tokenserver-db
    environment:
      MYSQL_ROOT_PASSWORD: MY_TOKEN_MYSQL_ROOT_PASSWORD
      MYSQL_DATABASE: tokenserver
      MYSQL_USER: MY_TOKEN_MYSQL_USER
      MYSQL_PASSWORD: MY_TOKEN_MYSQL_USER_PASSWORD
    volumes:
      - path/to/appdata/firefox-sync/tokenserver-db:/var/lib/mysql
    ports:
      - 3306
    restart: always

After running docker-compose up to let mysql set itself up, you need to insert the rows below into the tokenserver db as described here. Change the mydomain.tld to your domain. IP address with port also works fine but afaik it must be the same domain that you intend to use for identity.sync.tokenserver.uri in firefox later (Only the domain and protocol as seen below though, not the entire url).

INSERT INTO `services` (`id`, `service`, `pattern`) VALUES ('1', 'sync-1.5', '{node}/1.5/{uid}');
INSERT INTO `nodes` (`id`, `service`, `node`, `available`, `current_load`, `capacity`, `downed`, `backoff`) VALUES ('1', '1', 'https://mydomain.tld', '1', '0', '1', '0', '0');

Since the port configuration in the docker-compose file is not explicit, you will need to look up the tokenserver-db container port with docker-compose ps when connecting to the db to insert the sql.

[WAdama]

Hi jakobkukla,

thanks very much for that!

[jakobkukla]

Keep in mind that I have no idea what I'm doing, so not sure if this configuration is suitable/save to be used in public. Maybe someone from the dev team could chime in :).

Especially the value for SYNC_TOKENSERVER__ADDITIONAL_BLOCKING_THREADS_FOR_FXA_REQUESTS is just some random number. I don't really know what the setting is doing...

[WAdama]

Just good to know it's really working.

I will also try something a little different as I have already a working instance of MariaDB.

[jakobkukla]

@WAdama MariaDB was not working for me for some reason. Had to specifically use mysql 5.7.

But if you can get it to run with MariaDB, I'd like to know how :). It should be working in theory.

[jrconlin]

Thanks @jakobkukla!

I think that's got most of the args folk should need. @ethowitz can say definitively, but I believe this comment describes what SYNC_TOKENSERVER__ADDITIONAL_BLOCKING_THREADS_FOR_FXA_REQUESTS does.

In short, it's a bit of extra thread count buffer that the TokenServer needs to talk to the FxA servers. A low count should be fine for small, stand alone installations.

[jakobkukla]

@jrconlin Thanks for the quick reply!

I have one more question. Would it be possible to create the service and node entry at first startup? Using something like tokenserver.node_domain as a setting. That would greatly improve the setup experience for self hosting. Or is there some technical reason that's currently not possible?

And maybe providing a reasonable default to tokenserver.additional_blocking_threads_for_fxa_requests would be a good idea? I think it's kind of an odd setting to leave uninitialized.

[WAdama]

If I try your compose file I get for the MySQL containers the following error: mysqld: Can't create directory '/var/lib/mysql/' (Errcode: 17 - File exists)

I use - of course - a folder which exists...

Edit: Found it. Folder has to be set to executable.. (chmod +x ...)

[jrconlin]

@jakobkukla: I think @ethowitz might be able to provide better guidance about creating the service and node entry at first start-up, since he's responsible for that code. He's currently very heads down on the crate re-org which will help a good deal in making the stand-alone side easier to build and maintain, so he may have that as a TODO item.

I suspect that both of these items might be on his task list.

[WAdama]

@jakobkukla I now have the containers up. But I get sync errors in about:sync-log

Did you add something else in the environment of the Sync container?

How is your Token Server in Firefox itself formatted?

[jakobkukla]

@WAdama No, I didn't need anything else. What kind of error are you getting?

My identity.sync.tokenserver.uri setting is set to https://mydomain.tld/1.0/sync/1.5.

[WAdama]

@jakobkukla I should have tried first without my reverse proxy inbetween...

Adressing my docker instance directly worked.

By the way I changed the installation to using only one mysql container. Of course I had to make the changes manually, but at least I need only one database container...

Edit: The problem with the reverse proxy is also solved - more or less. I have tried a subfolder in domain. If I don't use a subfolder but only the domain it works.

[WAdama]

By the way, is there a reason why you used version 0.12.4 and not latest?

[jakobkukla]

Yes, because latest is for some reason an image from 3 years ago.

[WAdama]

Ah ok, that's a reason.. ;-)

Don't want to bother you again, but after running with a test user and a test profile I now tried to change my existing Firefox profile to the new sync server and I got errors again.

Have attached two error logs

error-sync-1666896951311.txt error-sync-1666897119826.txt

[ictabc]

This helps a lot, but I still have some issues, although I'm close. (Running this on my synology docker until I get it working, then converting it to my Kubernetes cluster (just started with k8s), but want to get it work work first with docker.)

I'm also using an Apache as reverse proxy to handle the SSL part. But connecting directly to docker gives the same error.

My first mistake was to use :latest, because there is an issue where the latest version on docker hub isn't updated to the latest version available. Been open since Juli 15th, not fixed yet. https://github.com/mozilla-services/syncstorage-rs/issues/1362

Second mistake was not feeding the correct variables to docker, for some reason my node_type was set to MySQL URL. Finally figured that out. And everything seems to be working.

Now with the the correct docker images I get the following error. And can't seem to figure out what goes wrong.

docker-0.12.4-sync-error-last-lines.txt

1666953384868 Sync.Resource DEBUG GET fail 401 https://sync.my.domain/1.5/4/info/collections 1666953384868 Sync.Resource WARN GET request to https://sync.my.domain/1.5/4/info/collections failed with status 401 1666953384868 Sync.Service WARN 401: login failed.

For some reason I get an 401 when accessing my collections.

User data is filled in the database when logging in.

Using an newer docker images (Newer then 0.12.4), always gives an database error, doesn't seem to get the database_url settings for tokenstorage.

When I try to sync again I get the following errors in sync-log, still 401's

docker-0.12.4-sync-error.txt

[WAdama]

@jakobkukla It may be a dumb question, but you're using more than one user with your sync server I guess?

When I try a second user, the user doesn't register to my sync server, I tried even a totally new one. The first user register to the server and works, the second one not.

[jdarmetzki]

I have the same issues as @ictabc , but i am still looking into my setup. In the meantime i overhauled the compose-file so that the credentials are stored in a separate file, there is only one mysql-instance needed, and you don't need to manually insert stuff into the database.

version: "3.8"

services:
  firefox-sync:
    image: mozilla/syncstorage-rs:0.12.4
    container_name: firefox-sync
    depends_on:
      - firefox-sync-db
    environment:
      SYNC_HOST: 0.0.0.0
      SYNC_HUMAN_LOGS: 1
      SYNC_MASTER_SECRET: ${SYNC_MASTER_SECRET}
      SYNC_DATABASE_URL: mysql://${MYSQL_USER}:${MYSQL_PASS}@firefox-sync-db:3306/syncstorage
      SYNC_TOKENSERVER__ENABLED: "true"
      SYNC_TOKENSERVER__RUN_MIGRATIONS: "true"
      SYNC_TOKENSERVER__NODE_TYPE: mysql
      SYNC_TOKENSERVER__DATABASE_URL: mysql://${MYSQL_USER}:${MYSQL_PASS}@firefox-sync-db:3306/tokenserver
      SYNC_TOKENSERVER__FXA_EMAIL_DOMAIN: api.accounts.firefox.com
      SYNC_TOKENSERVER__FXA_OAUTH_SERVER_URL: https://oauth.accounts.firefox.com/v1
      SYNC_TOKENSERVER__FXA_METRICS_HASH_SECRET: ${METRICS_HASH_SECRET}
      # I don't really know what this is doing
      SYNC_TOKENSERVER__ADDITIONAL_BLOCKING_THREADS_FOR_FXA_REQUESTS: 2
    ports:
      - 5000:8000
    restart: always
  firefox-sync-db:
    image: mysql:5.7
    container_name: firefox-sync-db
    environment:
      MYSQL_ROOT_PASSWORD: ${MSYQL_SYNC_ROOT_PASS}
      MYSQL_DATABASE: syncstorage
      MYSQL_USER: ${MYSQL_USER}
      MYSQL_PASSWORD: ${MYSQL_PASS}
    entrypoint:
      sh -c "
      echo 'CREATE DATABASE IF NOT EXISTS syncstorage; CREATE DATABASE IF NOT EXISTS tokenserver;' > /docker-entrypoint-initdb.d/init.sql;
      echo 'GRANT ALL PRIVILEGES ON syncstorage.* TO `${MYSQL_USER}`@`%`;' >> /docker-entrypoint-initdb.d/init.sql;
      echo 'GRANT ALL PRIVILEGES ON tokenserver.* TO `${MYSQL_USER}`@`%`;' >> /docker-entrypoint-initdb.d/init.sql;
      /usr/local/bin/docker-entrypoint.sh --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
      "
    volumes:
      - ./syncstorage-db:/var/lib/mysql
    ports:
      - 3306
    restart: always
  db-setup-sidecar:
    image: mysql:5.7
    depends_on:
      - firefox-sync-db
      - firefox-sync
    entrypoint:
      bash -c " 
      IS_DONE=10;      
      while [ $$IS_DONE -gt 0 ]; do
        echo \"USE tokenserver; INSERT IGNORE INTO services (id, service, pattern) VALUES ('1', 'sync-1.5', '{node}/1.5/{uid}'); 
        INSERT INTO nodes (id, service, node, available, current_load, capacity, downed, backoff)  
        VALUES ('1', '1', '${DOMAIN}', '1', '0', '1', '0', '0') ON DUPLICATE KEY UPDATE node='${DOMAIN}';\"|/usr/bin/mysql -h firefox-sync-db --user=${MYSQL_USER} -p${MYSQL_PASS} ;
        RC=$$?;
        echo \"mysql return code was $$RC\";
        if [ $$RC == 0 ] ; then
          IS_DONE=0;
          echo 'Done!';
          exit 0;
        else
          echo 'Waiting for tables...';
          sleep 5;
          ((IS_DONE--));                  
        fi;
      done;
      echo 'Giving up, sorry';
      exit 42;
      "
    restart: "no"

For the config you just need a simple .env-File next to docker-compose.yml

MYSQL_USER=sync
MYSQL_PASS=<your_password>
SYNC_MASTER_SECRET=<your_master_password>
METRICS_HASH_SECRET=<your_hash_secret>
MSYQL_SYNC_ROOT_PASS=<your_mysql_root_password>

DOMAIN=https://<example.your.domain>

EDIT: Works for me now - DOMAIN in .env has to be prefixed with https://

[jakobkukla]

@WAdama No I've only tried with one user. Could the capacity field in the nodes table be the problem?

[WAdama]

@jakobkukla I will have a look and test it.

[WAdama]

@jakobkukla You're my hero... That was the solution. To be on the sure side I have set it to 5 and - shazam - the second user could attach and was created in the database, too...

[WAdama]

@jdarmetzki I used your compose file, it worked like a charm, great work. Thanks.

After having a working instance I will try to use the MariaDB on my Synology NAS.

[WAdama]

I got it running with MariaDB and Docker on my Synology NAS. I used jdarmetzki's work as blueprint.

First I connect on command line to the database instance: mysql -u root -p I then created the user, the dabases and give the user the rights: `CREATE USER sync_rs@"172.%" IDENTIFIED BY '';

CREATE DATABASE IF NOT EXISTS syncstorage_rs; CREATE DATABASE IF NOT EXISTS tokenserver_rs;

GRANT ALL PRIVILEGES ON syncstorage_rs.* TO sync_rs@"172.%"; GRANT ALL PRIVILEGES ON tokenserver_rs.* TO sync_rs@"172.%";`

For the container I used this compose file: `version: "3.8"

services: firefox-sync: image: mozilla/syncstorage-rs:0.12.5 network_mode: bridge container_name: FirefoxSync_RS environment: SYNC_HOST: 0.0.0.0 SYNC_HUMAN_LOGS: 1 SYNC_MASTER_SECRET: ${SYNC_MASTER_SECRET} SYNC_DATABASE_URL: mysql://${MYSQL_USER}:${MYSQL_PASS}@${DATABASE_SERVER}:${DATABASE_PORT}/syncstorage_rs SYNC_TOKENSERVER__ENABLED: "true" SYNC_TOKENSERVER__RUN_MIGRATIONS: "true" SYNC_TOKENSERVER__NODE_TYPE: mysql SYNC_TOKENSERVER__DATABASE_URL: mysql://${MYSQL_USER}:${MYSQL_PASS}@${DATABASE_SERVER}:${DATABASE_PORT}/tokenserver_rs SYNC_TOKENSERVER__FXA_EMAIL_DOMAIN: api.accounts.firefox.com SYNC_TOKENSERVER__FXA_OAUTH_SERVER_URL: https://oauth.accounts.firefox.com/v1 SYNC_TOKENSERVER__FXA_METRICS_HASH_SECRET: ${METRICS_HASH_SECRET} SYNC_TOKENSERVER__ADDITIONAL_BLOCKING_THREADS_FOR_FXA_REQUESTS: 2 ports: - ${EXTERNAL_PORT}:8000 restart: always and this .env file:COMPOSE_PROJECT_NAME=ffsync_rs MYSQL_USER=sync_rs MYSQL_PASS= SYNC_MASTER_SECRET= METRICS_HASH_SECRET= DATABASE_SERVER=<ip of your DB server> DATABASE_PORT=<port of your DB> EXTERNAL_PORT=`

After created and started the container attach again to the database and create the nodes and service entries: USE tokenserver_rs; INSERT IGNORE INTO services (id, service, pattern) VALUES ('1', 'sync-1.5', '{node}/1.5/{uid}'); INSERT INTO nodes (id, service, node, available, current_load, capacity, downed, backoff) VALUES ('1', '1', 'https://<your_server>:<port>', '1', '0', '5', '0', '0');

I've set the capacity entry to 5 in my case.

The sync was running at once. There's only one problem shown in the log. I've got several entries like Nov 03 18:14:51.367 ERRO Lost connection to MySQL server during query. But the logs in "about:sync-logs" are all success logs. I'm still investigating this.

The container was created with 0.12.4 but as you see I have already updated it with 0.12.5.

[ictabc]

Running basically the same config settings.

Usually I do the following to start cleanly Stopped, the docker container, dropped every table in current database. Start docker container again, database is refilled. Then add the services and node configuration. And start the sync, tokens part goes ok, collections keep getting an 401.

Keep getting:

1667554182564 Sync.Resource DEBUG GET fail 401 https://sync.my.domain/1.5/4/info/collections 1667554182564 Sync.Resource WARN GET request to https://sync.my.domain/1.5/4/info/collections failed with status 401 1667554182564 Sync.Service WARN 401: login failed.

Very strange that it works for some and not for others.

[WAdama]

Hi @ictabc,

The domain in Firefox and the database are the same? For example https://sync.mydomain.de/1.0/sync/1.5 in Firefox and https://sync.mydomain.de in database?

[ictabc]

Hi WAdama,

Yup, those are the same. Do have an Apache reverse proxy config in between. But even without the reverse proxy config, it doesn't work.

Don't get the 401 now, but still an auth error, when going directly to the server. But that is without SSL, so prefer the Apache reverse proxy option.

================================================= 1667562062183 Sync.SyncAuthManager ERROR Non-authentication error in _fetchTokenForUser: TokenServerClientNetworkError({"error":{}})(resource://services-common/tokenserverclient.js:39:36) JS Stack trace: [email protected]:62:16 [email protected]:241:13 1667562062183 Sync.Status DEBUG Status.login: success.status_ok => error.login.reason.network 1667562062183 Sync.Status DEBUG Status.service: error.login.failed => error.login.failed 1667562062184 Sync.SyncAuthManager INFO Failed to fetch the cluster URL: TokenServerClientNetworkError({"error":{}})(resource://services-common/tokenserverclient.js:39:36) JS Stack trace: [email protected]:62:16 [email protected]:241:13 1667562062184 Sync.Service DEBUG verifyLogin failed: TokenServerClientNetworkError({"error":{}})(resource://services-common/tokenserverclient.js:39:36) JS Stack trace: [email protected]:62:16 [email protected]:241:13 1667562062184 Sync.Status DEBUG Status.login: error.login.reason.network => error.login.reason.network 1667562062184 Sync.Status DEBUG Status.service: error.login.failed => error.login.failed 1667562062184 Sync.ErrorHandler ERROR Sync encountered a login error 1667562062184 Sync.SyncScheduler DEBUG Clearing sync triggers and the global score. 1667562062185 Sync.SyncScheduler DEBUG Next sync in 3600000 ms. (why=schedule) 1667562062186 FirefoxAccounts TRACE not checking freshness of profile as it remains recent 1667562062186 Sync.Service DEBUG Exception calling WrappedLock: Error: Login failed: error.login.reason.network(resource://services-sync/service.js:1039:15) JS Stack trace: [email protected]:1039:15 1667562062187 Sync.Service DEBUG Not syncing: login returned false. 1667562062187 FirefoxAccounts TRACE not checking freshness of profile as it remains recent

[ictabc]

Changed both to :

https://sync.mydomain.de:8000/1.0/sync/1.5 in Firefox and https://sync.mydomain.de:8000/ in database?

As port 5000 is not available on a synology.

[WAdama]

I know, have running it on a Syno myself. Using myself a port in the higher region (xxxxx)..

Did you check if the server is running correct with https://sync.mydomain.de:8000/heartbeat?

[ictabc]

{"status":"Ok","tokenserver":{"database":"Ok","status":"Ok"},"quota":{"enabled":false,"size":0},"database":"Ok","version":"0.12.5"}

Looks ok.

The token server works, but the collection part fails.