Directory /vendor/mozilla-rust-sdk does not exist

[ln-12]

The docker section in the readme states This requires access to the mozilla-rust-sdk which is now available at /vendor/mozilla-rust-adk(I guess adk is a typo and should be sdkinstead?). However, this directory does not exist (anymore). Are there updated instructions or an example docker-compose.yml available?

[jrconlin]

Ah, yeah. This was replaced with the official release google-cloud-rust, which is the published version of the vendor library.

We'll clean up that compose file soon.

[jackyzy823]

@jrconlin Another related question.

I see golang-go build dependency is introduced with mozilla-rust-sdk . commit

since mozilla-rust-sdk is gone , https://github.com/mozilla-services/google-cloud-rust and https://github.com/tikv/grpc-rs do not contains go code.

Also I build and test Dockerfile without golang-go, every thing seems works normally.

So is golang-go build dependency still necessary?

[ln-12]

Hey @jrconlin, are the compose files ready to be tested? I am not sure ybout the state as the readme still metions the old sdk.

[jrconlin]

Ah, sorry. I'll do a quick PR to clean those up today.

[jrconlin]

Sorry, a bunch of other things came up and I wasn't able to get this done.

I think that working off of the docker-compose files may not be the best idea because those are more for CI contract testing. They do a lot of bad things like use an insecure mysql instance, bad master secret, etc. Fine for testing, REALLY BAD IDEA for personal deployments.

I have a script I run to launch syncserver locally that might be useful to build off of:

#!/bin/bash
# Run this script in the project root
args=$*
LOG=${LOG:-trace}
# RUST_LOG=debug
#FEATURES="--features no_auth"
SYNC_CONF=${SYNC_CONF:-config/test.toml}
RUN_CMD=
#RUN_CMD= target/debug/syncstorage
FEATURES="--features grpcio/openssl"
RUST_LOG="syncstorage=$LOG,syncstorage-common=$LOG,syncstorage-db-common=$LOG,tokenserver-common=$LOG" \
    SYNC_HOST=0.0.0.0 \
    SYNC_DATABASE_URL=${SYNC_DATABASE_URL:-mysql://test:test@localhost/test} \
    SYNC_MASTER_SECRET=${SYNC_MASTER_SECRET:-secret0} \
    SYNC_DATABASE_POOL_MAX_SIZE=30 \
    SYNC_DATABASE_POOL_TIMEOUT=10 \
    cargo run $FEATURES -- --config $SYNC_CONF $args

(for folks not familiar with bash: ${FOO:-Default value} will use the env var $FOO, or "Default value" if it's not set.)

IIRC, you may want to run this in a python virtualenv (see requirements.txt for the python stuff). The recently integrated TokenServer code uses python in rust to do some functions and will call for libraries.

$ python3 -m venv venv
$ . venv/bin/activate
$ pip install -r requirements.txt
$ . run_mysql.bash

I'll try to poke at this a bit next week, but I'll be traveling and not sure how much time I'll have.

[ln-12]

Thanks for the instructions. I had to figure out the dependencies on macOS:

• I had to point the rust compiler to my openssl installation: OPENSSL_ROOT_DIR=/opt/homebrew/opt/openssl@3/
• After that, linking failed, so I had to install mysql-client and point to it: LIBRARY_PATH=$LIBRARY_PATH:/opt/homebrew/opt/mysql-client/lib

Now the binary is building and can be executed. When I use the existing local.example.toml, there is a missing property:

If the Tokenserver OAuth JWK is not cached, additional blocking threads must be used to handle the requests to FxA.
Error: configuration property "tokenserver.additional_blocking_threads_for_fxa_requests" not found

I simply added the line tokenserver.additional_blocking_threads_for_fxa_requests = 1 (without really knowing what it is doing) to the toml file which now leads to a connection error:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: ApiError { kind: Db(DbError { kind: DieselConnection(BadConnection("Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)")), status: 500, backtrace:    0: backtrace::backtrace::libunwind::trace

This makes sense as I did not setup any mysql server on my machine yet (and don't really want to). I will probably just wait until the process is a bit more convenient/there is a recommended docker file available before I self-host the sync server. Nevertheless, thanks for sharing your script!

[ln-12]

Hey @jrconlin, is your script still the recommended way of self hosting the sync service or is there a better approach?

In the meantime I found https://github.com/jeena/fxsync-docker but just wanted to check if I missed something here.

[jrconlin]

I'd suggest that you go with @jeena's repo.

My script could be whittled down to basically cargo run -- --config path/to/config.toml. I believe grpcio fixed the bug which required openssl for some configurations, so you probably don't need that, but if some exceptionally kind soul has taken the effort to build out a docker image and you don't want to have to have all the weird config crap live in your head all the time, I'd just use that.

[ln-12]

While jeena's setup works fine for me, it is no longer compatible with the latest version of this repo (see https://github.com/jeena/fxsync-docker/issues/3). So if someone find this issue and wants to use that as well, I would recommend to not update the syncstorage-rs version until https://github.com/mozilla-services/syncstorage-rs/issues/1482 is solved.