Self hosting docker-compose.yml

[airtonix]

Trying to get this running in our rancher server, bit of trial and error but I've started with this docker-compose:

version: '2'
services:
  elasticsearch:
    image: nickstenning/elasticsearch-icu
    stdin_open: true
    tty: true
    labels:
      io.rancher.container.pull_image: always
  web:
    image: hypothesis/hypothesis
    environment:
      BROKER_URL: amqp://guest:guest@rabbit:5672//
      ELASTICSEARCH_HOST: http://elasticsearch:9200
      DATABASE_URL: postgresql://postgres@postgres:5432/htest
      SECRET_KEY: SECRET_KEY
    stdin_open: true
    tty: true
    labels:
      io.rancher.container.pull_image: always
  rabbit:
    image: rabbitmq:3.6-management-alpine
    stdin_open: true
    tty: true
    labels:
      io.rancher.container.pull_image: always
  postgres:
    image: postgres:9.4-alpine
    stdin_open: true
    tty: true
    labels:
      io.rancher.container.pull_image: always

and rancher automatically creates this rancher-compose (this isn't required):

version: '2'
services:
  elasticsearch:
    scale: 1
    start_on_create: true
  web:
    scale: 1
    start_on_create: true
  rabbit:
    scale: 1
    start_on_create: true
  postgres:
    scale: 1
    start_on_create: true

and in our load balancer i have a rule like :

host: h.*
ingress port: 80
container: hypothesis.web
containerPort: 5000

but all I get when visting http://h.coreos.lan:80 is this :

Logs from the web container show this:

3/23/2018 10:00:17 AM2018-03-22 23:30:17,951 INFO supervisord started with pid 1
3/23/2018 10:00:18 AM2018-03-22 23:30:18,954 INFO spawned: 'logger' with pid 17
3/23/2018 10:00:18 AM2018-03-22 23:30:18,956 INFO spawned: 'web' with pid 18
3/23/2018 10:00:18 AM2018-03-22 23:30:18,958 INFO spawned: 'websocket' with pid 19
3/23/2018 10:00:18 AM2018-03-22 23:30:18,960 INFO spawned: 'worker' with pid 20
3/23/2018 10:00:18 AM2018-03-22 23:30:18,962 INFO spawned: 'collectd' with pid 21
3/23/2018 10:00:18 AM2018-03-22 23:30:18,964 INFO spawned: 'nginx' with pid 22
3/23/2018 10:00:18 AM2018-03-22 23:30:18,974 INFO success: collectd entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
3/23/2018 10:00:18 AM2018-03-22 23:30:18,975 INFO exited: collectd (exit status 0; expected)
3/23/2018 10:00:20 AM2018-03-22 23:30:20,166 INFO success: logger entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
3/23/2018 10:00:20 AM2018-03-22 23:30:20,166 INFO success: web entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
3/23/2018 10:00:20 AM2018-03-22 23:30:20,166 INFO success: websocket entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
3/23/2018 10:00:20 AM2018-03-22 23:30:20,166 INFO success: worker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
3/23/2018 10:00:20 AM2018-03-22 23:30:20,166 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
3/23/2018 10:00:20 AMcollectd             | INFO: GRAPHITE_HOST not provided so collectd will not be started
3/23/2018 10:00:20 AMwebsocket (stderr)   | 2018-03-22 23:30:19,716 [19] [gunicorn.error:INFO] Starting gunicorn 19.6.0
3/23/2018 10:00:20 AMwebsocket (stderr)   | 2018-03-22 23:30:19,717 [19] [gunicorn.error:INFO] Listening at: unix:/tmp/gunicorn-websocket.sock (19)
3/23/2018 10:00:20 AMwebsocket (stderr)   | 2018-03-22 23:30:19,717 [19] [gunicorn.error:INFO] Using worker: h.websocket.Worker
3/23/2018 10:00:20 AMwebsocket (stderr)   | 2018-03-22 23:30:19,723 [34] [gunicorn.error:INFO] Booting worker with pid: 34
3/23/2018 10:00:20 AMwebsocket (stderr)   | 2018-03-22 23:30:19,734 [34] [gunicorn.error:INFO] Made psycopg green
3/23/2018 10:00:20 AMweb (stderr)         | 2018-03-22 23:30:20,166 [18] [gunicorn.error:INFO] Starting gunicorn 19.6.0
3/23/2018 10:00:20 AMweb (stderr)         | 2018-03-22 23:30:20,166 [18] [gunicorn.error:INFO] Listening at: unix:/tmp/gunicorn-web.sock (18)
3/23/2018 10:00:20 AMweb (stderr)         | 2018-03-22 23:30:20,167 [18] [gunicorn.error:INFO] Using worker: sync
3/23/2018 10:00:20 AMweb (stderr)         | 2018-03-22 23:30:20,171 [39] [gunicorn.error:INFO] Booting worker with pid: 39
3/23/2018 10:00:21 AMworker               |
3/23/2018 10:00:21 AMworker               |  -------------- celery@255e900287e5 v4.1.0 (latentcall)
3/23/2018 10:00:21 AMworker               | ---- **** -----
3/23/2018 10:00:21 AMworker               | --- * ***  * -- Linux-4.12.10-coreos-x86_64-with 2018-03-22 23:30:21
3/23/2018 10:00:21 AMworker               | -- * - **** ---
3/23/2018 10:00:21 AMworker               | - ** ---------- [config]
3/23/2018 10:00:21 AMworker               | - ** ---------- .> app:         h:0x7f691f547310
3/23/2018 10:00:21 AMworker               | - ** ---------- .> transport:   amqp://guest:**@rabbit:5672//
3/23/2018 10:00:21 AMworker               | - ** ---------- .> results:     disabled://
3/23/2018 10:00:21 AMworker               | - *** --- * --- .> concurrency: 6 (prefork)
3/23/2018 10:00:21 AMworker               | -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
3/23/2018 10:00:21 AMworker               | --- ***** -----
3/23/2018 10:00:21 AMworker               |  -------------- [queues]
3/23/2018 10:00:21 AMworker               |                 .> celery           exchange=celery(direct) key=celery
3/23/2018 10:00:21 AMworker               |                 .> indexer          exchange=indexer(direct) key=indexer
3/23/2018 10:00:21 AMworker               |

[jayzhen521]

@airtonix I encountered similar problems, have you been fixed it?

[airtonix]

Nope.

[jayzhen521]

@airtonix My error is because docker run but lost the step of "bin/hypothesis init". db is not initialized. docker logs containerID is useful to check error. docker-compose exec postgres psql -U postgres is useful to see if data is match.

[adrianmace]

Here's mine - it doesn't work - attempting to run a self-hosted instance of the H service for my web app to integrate into and avoid cloud storage of visitor's accounts / comments.

version: '3'
services:
  postgres:
    image: postgres:9.4-alpine
    ports:
      - '127.0.0.1:5432:5432'
  elasticsearch:
    image: hypothesis/elasticsearch:latest
    ports:
      - '127.0.0.1:9200:9200'
    environment:
      - discovery.type=single-node
  rabbit:
    image: rabbitmq:3.6-management-alpine
    ports:
      - '127.0.0.1:5672:5672'
      - '127.0.0.1:15672:15672'
  hypothesis:
    image: hypothesis/hypothesis:latest
    ports:
      - '80:80'
      - '443:443'
      - '5000:5000'
    depends_on:
      - postgres
      - elasticsearch
      - rabbit
      - redis
    environment:
      - BROKER_URL=amqp://guest:guest@rabbit:5672//
      - CLIENT_ID=612e7a10-722a-435c-84fe-a5707a181dbf
      - CLIENT_SECRET=02zA9R_N1UeOuY3DlzxJ-50_m16hZIUjAggi1DLQQpLi4b2hoYOjHUtwww566yFJ
      - DATABASE_URL=postgres://postgres@postgres:5432/h
      - ELASTICSEARCH_URL=http://elasticsearch:9200
      - REDIS_HOST=redis
      - SECRET_KEY=tnXHWMEE5BqCXEfaNRuht38LK-fEvUaZf5v8u2rTOE1yeUHQDIKvdug2v9o4gYPt
  redis:
    image: redis:latest
    ports:
      - '127.0.0.1:6379:6379'

[jewome62]

To create automaticly database you can add env variables on db

  postgres:
    image: postgres:9.4-alpine
    ports:
      - '5432:5432'
    environment:
      - POSTGRES_DB=h
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres

And for hypothesis service : DATABASE_URL=postgres://postgres:postgres@postgres:5432/h

[cyphercider]

Has anyone been able to create a working, comprehensive docker-compose.yml for hosting hypothesis? I tried multiple configs. The one that has gotten me the furthest is below, but I'm getting an error that seems to reflect that postgresql migrations aren't being run.

postgres_1       | ERROR:  relation "group" does not exist at character 43
postgres_1       | STATEMENT:  SELECT "group".pubid AS group_pubid 
postgres_1       |      FROM "group" 
postgres_1       |      WHERE "group".readable_by = 'world'

Docker config:

version: '3'
services:
  postgres:
    image: postgres:9.4-alpine
    restart: always
    ports:
      - '127.0.0.1:5432:5432'
    environment:
      - POSTGRES_DB=h
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
  elasticsearch:
    image: hypothesis/elasticsearch:latest
    restart: always
    ports:
      - '127.0.0.1:9200:9200'
    environment:
      - discovery.type=single-node
  rabbit:
    image: rabbitmq:3.6-management-alpine
    restart: always
    ports:
      - '127.0.0.1:5672:5672'
      - '127.0.0.1:15672:15672'
    environment:
      - RABBITMQ_DEFAULT_USER=guest
      - RABBITMQ_DEFAULT_PASS=guest
  web:
    image: hypothesis/hypothesis
    restart: always
    ports:
      - '8090:80'
      - '8443:443'
      - '5000:5000'
    environment:
      - APP_URL=http://localhost:5000
      - AUTHORITY=localhost
      - BROKER_URL=amqp://guest:guest@rabbit:5672//
      - DATABASE_URL=postgres://postgres:postgres@postgres:5432/h
      - ELASTICSEARCH_URL=http://elasticsearch:9200
      - NEW_RELIC_APP_NAME=h (dev)
      - NEW_RELIC_LICENSE_KEY=
      - SECRET_KEY=SECRET_KEY
    stdin_open: true
    tty: true
    labels:
      io.rancher.container.pull_image: always

[cyphercider]

Hi All,

After mashing up some solutions from other issues plus using the 'make run-docker' command, I was able to run a full hypothesis deployment out of the box using the below docker-compose file. One thing I'm a bit concerned about is the fact I had to use the ':dev' tag for the hypothesis image rather than the latest from dockerhub...but I guess this is good enough to start using the service. I'll report back here with any version issues.

Working docker-compose.yml:

version: '3'
services:
  postgres:
    image: postgres:9.4-alpine
    ports:
      - '127.0.0.1:5432:5432'
  elasticsearch:
    image: hypothesis/elasticsearch:latest
    ports:
      - '127.0.0.1:9200:9200'
    environment:
      - discovery.type=single-node
  rabbit:
    image: rabbitmq:3.6-management-alpine
    ports:
      - '127.0.0.1:5672:5672'
      - '127.0.0.1:15672:15672'
  redis:
    image: redis:latest
    ports:
      - '127.0.0.1:6379:6379'
  web:
    image: hypothesis/hypothesis:dev
    restart: always
    ports:
      - '5000:5000'
    environment:
      - APP_URL=http://localhost:5000
      - AUTHORITY=localhost
      - BROKER_URL=amqp://guest:guest@rabbit:5672//
      - DATABASE_URL=postgresql://postgres@postgres/postgres
      - ELASTICSEARCH_URL=http://elasticsearch:9200
      - NEW_RELIC_APP_NAME=h (dev)
      - NEW_RELIC_LICENSE_KEY
      - SECRET_KEY=notasecret
      - MODEL_CREATE_ALL=true
      - REDIS_HOST=redis

[malteos]

Is there any progress on this? With the help of @jeffkhull's docker-compose.yml I managed to get everything running. However, I failed creating a user to login because verification mails aren't sent.

In general it would be great to have a tutorial on how to setup Hypothesis as self-hosted service.

[zeigerpuppy]

looks like the mail settings are here: https://github.com/hypothesis/h/blob/fd943f9daa495daf402d33867fa118526d2007e0/h/config.py

it would be nice to replace these with environment variables in the docker-compose.yml

    if "MANDRILL_USERNAME" in environ and "MANDRILL_APIKEY" in environ:
        settings_manager.set("mail.username", "MANDRILL_USERNAME")
        settings_manager.set("mail.password", "MANDRILL_APIKEY")
        settings_manager.set(
            "mail.host", "MANDRILL_HOST", default="smtp.mandrillapp.com"
        )
        settings_manager.set("mail.port", "MANDRILL_PORT", default=587)
settings_manager.set("mail.tls", "MANDRILL_TLS", default=True)

[zeigerpuppy]

I have tried testing with filling in these environment variables, but I think in dev mode emails are not sent at all (there are no errors or notifications in the logs). So I'm stuck at the same issue as @malteos, can create users and even make them admin with bin/hypothesis command but can't activate accounts as no email is sent.

I think we'll have to dig around in the config files to find a method to EITHER:

1. send emails from dev install
2. get normal hypothesis:latest to work in docker-compose
3. bypass or manually enable activation

[zeigerpuppy]

it looks like if we could get to the admin screen, then users can be activated by an admin: https://github.com/hypothesis/h/pull/3015

The best solution would just to work out why emails are not sending, seems strange that this would be disabled in dev but I guess we should be able to work it out from the Dockerfiles.

[zeigerpuppy]

I have noticed that our builds from hypothesis/hypothesis:dev have a different file in /h/config.py. The mail settings are different there. In fact, it looks like it may be designed for an internal test mailer (makes sense):

this is in the dev version but not in the production version

    DockerSetting('mail.host', 'mail',
                  pattern='{port_25_tcp_addr}'),
    DockerSetting('mail.port', 'mail', pattern='{port_25_tcp_port}'),

and this is missing from the dev:

    if "MANDRILL_USERNAME" in environ and "MANDRILL_APIKEY" in environ:
        settings_manager.set("mail.username", "MANDRILL_USERNAME")
        settings_manager.set("mail.password", "MANDRILL_APIKEY")
        settings_manager.set(
            "mail.host", "MANDRILL_HOST", default="smtp.mandrillapp.com"
        )
        settings_manager.set("mail.port", "MANDRILL_PORT", default=587)
        settings_manager.set("mail.tls", "MANDRILL_TLS", default=True)

I don't think it'll be as simple as adding these lines back in, but worth trying.

Also, does anyone have access to the Dockerfiles used to build the dev version, it'd be really helpful in working out what are the differences that are preventing getting it running...

[zeigerpuppy]

I note some more info regarding emails in dev mode from this issue: https://github.com/hypothesis/h/issues/2870

@nickstenning wrote:

For running in development I'd suggest using something like python -m smtpd -n -c DebuggingServer localhost:25 running in a container to print mails to stdout for inspection.

So I guess this confirms that there is no SMTP options configured in dev mode. It would be great to get this going so that it's possible to test a full deployment. In my case it's for a specific use case where we can't have comments stored on a third party system.

Any tips from devs on how to re-enable mail functionality to use a specified SMTP server would be greatly appreciated!

[robmoss-fp]

@zeigerpuppy

Just a note on the email issue above. Doesn't fix it, but I was able to get the initial user activated by grabbing the activation code and user ID (which will be "1" if you have created no other users, otherwise get from the "user" table) from the "activation" table in SQL and manually combining it in my browser thusly (replace localhost with another URL as needed):

http://localhost:5000/activate/[USER ID]/[ACTIVATION CODE]

Was then able to make the user an admin via these instructions. That user can then manually activate other accounts as needed. Just in case that helps anyone.

[zeigerpuppy]

great pickup @robmoss-fp, I'll give it a go. It at least lets us fully test an installation until we can get email working properly

[danielgrant-penten]

Has any body had any luck integrating a self hosting instance with the browser plugin? I have managed to created registered users and administration accounts on a self hosted instance following @jeffkhull docker above. Unfortunately a build of the browser extension does not allow the user to login (authorisation is accepted, except the extensions remains logged out). I also ran into issues creating the admin through the tox process and instead had to do an update within the Postgres docker "user" table. The Oauth authorisation from the extension is logging in to the self hosted instance (localhost:5000) but the actual extension itself remains logged out. Nothing is presented within the chrome console.

[zeigerpuppy]

I haven't had a chance to pick up my testing again yet. I feel that it should be possible to work this out... But a little help or pointers from the main dev's would be greatly appreciated!

[Goddywu]

esis init". db is not initialized. docker logs containerID is useful to check error.

@airtonix My error is because docker run but lost the step of "bin/hypothesis init". db is not initialized. docker logs containerID is useful to check error. docker-compose exec postgres psql -U postgres is useful to see if data is match.

how to init admin account ?

[Goddywu]

esis init". db is not initialized. docker logs containerID is useful to check error.

@airtonix My error is because docker run but lost the step of "bin/hypothesis init". db is not initialized. docker logs containerID is useful to check error. docker-compose exec postgres psql -U postgres is useful to see if data is match.

how to init admin account ?

thx the author. I have got my answer in my issue: https://github.com/hypothesis/h/issues/6014

[Stvad]

@zeigerpuppy I was curious if you ever got to a complete setup (or anyone else really)?

[zeigerpuppy]

@Stvad I didn't get a chance to return to this yet, but still keen to get it going. My efforts are focused on hardware builds at the moment, but would like to give this another try.

[lyz-code]

@lyzadanger @dwhly @robertknight , can you give us some insights on how to proceed? I'm sure you have fought with these same issues in the production site.

Thanks

[bpow]

The mail host and port can be overridden with MAIL_HOST and MAIL_PORT environment variables.

Modifying the docker-compose from @cyphercider to demonstrate this (using maildev to verify that the mail overrides are working, and including things like the POSTGRES_PASSWORD and SECRET_KEY from the enclosing environment).

version: '3'
services:
  postgres:
    image: postgres:13.4-alpine
    environment:
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD?need to specify POSTGRES_PASSWORD}
    expose:
      - '5432'
  elasticsearch:
    image: hypothesis/elasticsearch:latest
    expose:
      - '9200'
    environment:
      - discovery.type=single-node
  rabbit:
    image: rabbitmq:3.6-management-alpine
    expose:
      - '5672'
      - '15672'
  maildev:
    image: maildev/maildev
    expose:
      - '25'
    ports:
      - '8888:80'
  web:
    image: hypothesis/hypothesis
    restart: always
    ports:
      - '5000:5000'
    environment:
      - APP_URL=http://localhost:5000
      - AUTHORITY=localhost
      - BROKER_URL=amqp://guest:guest@rabbit:5672//
      - DATABASE_URL=postgresql://postgres:${POSTGRES_PASSWORD?need to specify POSTGRES_PASSWORD}@postgres/postgres
      - ELASTICSEARCH_URL=http://elasticsearch:9200
      - SECRET_KEY=${SECRET_KEY?need to specify SECRET_KEY that is really secret}
      - MODEL_CREATE_ALL=true
      - MAIL_HOST=maildev
      - MAIL_PORT=25

After exporting in your shell to set POSTGRES_PASSWORD and SECRET_KEY, you should be able to docker-compose up -d.

The system is not quite ready to go yet at that point, because the db still needs to be initialized. To do this:

docker-compose exec web bin/hypothesis init
docker-compose exec web bin/hypothesis migrate upgrade head

Then you can add a user with:

docker-compose exec web bin/hypothesis user add --username burdell --authority localhost --email [email protected]
docker-compose exec web bin/hypothesis user admin burdell

(The user add command will prompt for a password, which you could specify with --password at the command line if you wanted to script this.)

If you go to maildev after setting this up (it should be running on localhost:8888 from the docker-compose), then you'll see the mails that are being sent from the hypothes.is web interface (e.g., from the mailer test

I think this gets a fair bit further towards a production-like self-hostable docker-compose. Other things that could be considered TODOs:

• Make customized passwords for elasticsearch and rabbitMQ, parameterized in environment variables like POSTGRES_PASSWORD
• Obviously, specify a real smtpd server
• Disable newrelic if undesired? I can't figure out how to do this without replacing/overlaying conf/supervisord.conf in the hypothesis docker image. I don't see a way to disable this by environment variable
• Consider splitting the nginx, web, websocket and worker tasks to separate containers, avoiding supervisord entirely, to be more idiomatic for container-based systems.
• Automate/script the setup of a oauth clients. There is an authclient subcommand of bin/hypothesis, but it does not allow specification of grant_type or redirect_uri (see #7009).
• use volumes to store persistent data (especially for postgres/elasticsearch).

[lukaszgiez]

Hi,

I am trying to use h for annotating documentation in our internal project so I am running a local h server and client, and I have run into these issues:

1. When saving an annotation the client reports an error but on page refresh the annotations do get saved.
2. the client reports error while deleting annotations but on page refresh they do get deleted. I checked if the annotations get saved to the database via the make sql command and they are there. Also they seem to be present in elasticsearch. To find out I ran the curl 'http://localhost:9200/_search'
3. In server browser window I cannot create users nor groups. This is only possible via the terminal. elsticsearch log looks error-free postgres log reports ERROR: relation "alembic_version" does not exist at character 15 and STATEMENT: select 1 from alembic_version I am not sure what to do with that.

Now, I am rather new to docker and setting up a project like that so I have a lot to catch up on. I enclose my docker-compose.yml with my local ip and ID replaced with myIP and myID:

version: '3' services: postgres: image: postgres:11.5-alpine volumes: - h-db:/var/lib/postgresql/data ports: - '127.0.0.1:5432:5432' healthcheck: test: ["CMD", "pg_isready", "-U", "postgres"] interval: 1s elasticsearch: image: hypothesis/elasticsearch:latest ports: - '127.0.0.1:9200:9200' environment: - discovery.type=single-node rabbit: image: rabbitmq:3.6-management-alpine ports: - '127.0.0.1:5672:5672' - '127.0.0.1:15672:15672' mail: image: schickling/mailcatcher ports: - "1080:1080" - "25:1025" hypothesis: image: hypothesis/hypothesis ports: - '5000:5000' environment: - APP_URL=http://localhost:5000 - AUTHORITY=localhost - BROKER_URL=amqp://guest:guest@rabbitmq:5672 - DATABASE_URL=postgresql://postgres@postgres/postgres - ELASTICSEARCH_URL=http://elasticsearch:9200 - SECRET_KEY=notasecret - CLIENT_URL=http://myIP:3001/hypothesis - CLIENT_OAUTH_ID=myId - MODEL_CREATE_ALL=true - MAIL_PORT_25_TCP_ADDR=myIP - MAIL_PORT_25_TCP_PORT=25 volumes: h-db:

Is there sth amiss/undefined in the yml file? I am also totally new to databases and am not sure if postgres environment is correctly configured above. Nothing is obvious to me, so even if it's simple to you, it may be where I am totally ignorant. I hope you can assist! lukasz

[jcarpi]

Just pinging this issue. I am also interested in self-hosting a Hypothesis server for organization-internal documents and annotations.

[sergioSHKLR]

I would also love it if self-hosting was an official feature, rather than trying to reverse engineer it.

[webysther]

This project is great, works for me when using the client inside the calibre-web, but, have ES and PG as require for using to self-host is really to much, with sqlite + FTS give the same UX, waiting an alternative to do the same, anyone have a alternative? 5 years waiting is really hard to keep going.

[robertknight]

I am not aware of any other server implementations for the Hypothesis client at this time, but there is nothing to stop someone from creating one. This project is designed to serve the needs and scale of Hypothesis's (free) public service and customers, so we are not going to change the database any time soon.

[webysther]

Hi @robertknight there an expectation to change the API any time soon? Because create a fork to make a api alternative maybe is a viable option.

[zeigerpuppy]

I agree @webysther, a friendly-fork is probably the best option. The core functionality is good, so the main parts to work on would be the auth pathway and email settings.

It may be easier just to have a handoff to OAuth2 in the login process. That way it would be a useful addition to the platform while avoiding a lot of code conflicts with upstream.