boltstream

boltstream copied to clipboard

Hi, this looks great! Do you have any plans to provide support for Docker?

I would love to provide Docker support! But unfortunately I don't know anything about Docker. Would you be able to provide support for Docker? PRs are welcome!

in my branch here i was able to get the application running in Docker "successfully" (see: a boltstream.me homepage with "no live streams" text)... but only with some questionable changes:

1. because of a pip error [0], i downgraded importlib-metadata. i've no idea what the implications of this are, but i at least got to the home page without it complaining...
2. i made DB_URL configurable (in the immediate case, to point it to a Docker mysql host). i think the preferred method in Django for this type of thing is to override the settings file, but i think the preferred sysadmin approach is to configure it via environment. given that this project intends to handle infrastructure as well, i think the latter is a better option. i could see it getting messy if there are a dozen environment variables to configure though.

not married to either of these changes, just did them to "make it work" for now.

all that said, i'm more of an apps guy (never used Ansible or Terraform)... so it's not really clear to me where Docker fits into this whole project. i'm a little lost trying to reconcile how Docker sits side by side with Ansible or Terraform. assuming there is a lot of overlap, i think answering this question might get a little hairy?

so just to get the ball rolling, i've opted for strictly an "app development container" type deal... but Docker can definitely sit at a higher level than this, i'm just not sure exactly how.

[0]

ERROR: Cannot install -r requirements.txt (line 27) and importlib-metadata==3.1.1 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested importlib-metadata==3.1.1
    django-bootstrap4 2.3.1 depends on importlib-metadata<3; python_version < "3.8"

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies

also interested inj docker-compose.yml

Docker'ing the application shouldn't be that hard. The real issue is going to be how nginx is configured. The way ansible is installing/running is convoluted and not docker friendly. I will make some attempts.

Why MySQL? Django default is postgresql?

Why MySQL? Django default is PostgreSQL?

PostgreSQL is not the default, but the most mature and featured of the DB engine's implementation. That is why most Django developers recommend it. Personally, I would love to keep seeing PostgreSQL here, but from what I saw, switching engines at this stage on your local settings will not be a problem.

However, could we make a double bet? Support jsonb type for metadata and PG Notify for light queues. It could be interesting.

cc: @kn0ll

@mariocesar Right, it's SQLite, but I was thinking recommended for production usage. I agree we should switch early if there is no specific use case for mysql.

Can the three nginx servers be reduced to a single one? Not sure why they are separate. I can work on a single dockerfile for that.

We need a .env-template file that lists all the required environment variables.

Three Nginx server can be three processes running in a single container, or the image can run different CMD, like:

FROM bolstream/base:latest
CMD /bin/run_nginx-web

FROM bolstream/base:latest
CMD /bin/run_nginx-rtmp

FROM bolstream/base:latest
CMD /bin/run_nginx-nchan

EDIT: Outdated, see below.

Ok here is it dockerized: PR https://github.com/benwilber/boltstream/pull/5

git clone https://github.com/nitrag/boltstream.git
git checkout dockerize
docker-compose build
docker-compose up -d
docker-compose stop boltstream
docker-compose run boltstream python manage.py migrate 
# collect static?
docker-compose run boltstream python manage.py createsuperuser
# bug: you'll have to enter a UUID
docker-compose up -d
# now all are up and running

That's as far as I got. Lots of work to do on the app side. When I sign-in and click stream in just asks me to sign-in again.

Thanks a lot for working on this! It really looks great!

Can the three nginx servers be reduced to a single one? Not sure why they are separate. I can work on a single dockerfile for that.

The reason there are 3 separate nginx instances is because they're technically 3 completely different services. nginx-rtmp is the RTMP ingest + HLS packager. nginx-nchan is the websocket server for live chat + synchronized metadata, and then just regular nginx is the webserver handling the Django app and serving the MPEG-TS HLS streams. They need to be deployed independent of each other just because they have different responsibilities and scaling characteristics. (You don't want to kill all the live streams being ingested by nginx-rtmp if the nchan service gets overloaded, for instance. Also, for some technical reasons that I don't fully understand, nginx-rtmp can't run in multi-worker mode otherwise directives like rtmp_control and rtmp_stat don't work.

So running them all as separate nginx services allows regular nginx + nginx-nchan to run in multi-worker mode, and nginx-rtmp to just run in single worker-per-server.

@benwilber awesome project! I was looking for an open source platform exactly like this. Once the docker support branch is merged, I can help with automating the docker builds(CI/CD), creating the kubernetes manifests and deploying a sample version of this (if we want to)

Good job with the docker PR Is this for local dev including db? Having issues when doing createsuperuser. Cannot find boltstream_steams table

This PR is still a work in progress. I will have more time to spend on this towards the end of the month.

On Sun, Dec 13, 2020 at 1:21 PM JohnSDevs [email protected] wrote:

Good job with the docker PR Is this for local dev including db? Having issues when doing createsuperuser. Cannot find boltstream_steams table

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/benwilber/boltstream/issues/1#issuecomment-744047538, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABQDEQ2D6VIPFLAH65JOSTSUUA23ANCNFSM4UPZ5LGQ .

This is an awesome project. what is the status of it? I would love to it in action.