Plex Cluster: Beta Testers Needed

[Fmstrat]

Hi everyone,

You've been mentioned because at some point you have posted an issue here or participated via PR. I am migrating away from plex-db-sync to a new "instant sync" software platform I've developed called Plex Cluster.

I am looking for beta testers for the below platform. Reply here if you are interested!

Features

• Syncs watched status on-demand, right away
• Uses standard Plex APIs without accessing the database
• Can work across multiple Plex servers
• Scheduled "full update" can completely sync user's watched status for any user that has been active on each Plex server since running Plex Cluster
• Syncs media that is contained on both (or multiple) servers without erroring when media does not exist
• Works for Plex.tv or Managed users

Architecture

Plex Cluster contains two applications:

• Plex Cluster Proxy, which is installed alongside every Plex server and acts as a proxy (using nginx) between Plex Media Server and the internet. This application's job is to pass any requests that come to it along to the Plex server while catching any requests that mark watched status and also forwarding them on the Plex Cluster Manager. There is one instance of Plex Cluster Proxy for each Plex server.
• Plex Cluster Manager is then used to synchronize the status between Plex servers. There is only one instance of Plex Cluster Manager which is used by all instances of Plex Cluster Proxy.

An example use case:

1. Plex client requests to watch a show with GIUD (unique identifier) ABC from the TV Shows library on Plex Server 1
2. The request goes through Plex Cluster Proxy 1 and is passed along to Plex Server 1
3. The user stops the show midway.
4. The request goes through Plex Cluster Proxy 1 and is passed along to Plex Server 1, but is also passed along to Plex Cluster Manager
5. Plex Cluster Manager takes the request and checks for any other Plex server with a library named TV Shows and GUID ABC
6. For any other servers it finds, such as Plex Server 2, it forwards the request on to them and the show is instantly marked as watched up to that midway point.
7. User switches over to Plex Server 2 and sees that they are midway through show ABC

@ChaosZero112 @daxxi13 @Malcolmest @seertenedos @skinnyzaz @sbcrumb @goofballtech @onezero1010101 @ddurdle @dpeet

[Fmstrat]

I should note, while this solution can be run "raw,", it is designed to be a true cluster setup. This means supporting more than 2 servers, but also means it focuses on containerization (Docker or K8 if you really wanted to).

If you desire to beta test, but do not wish to run a container platform such as Docker, I would still be interested in your participation and assistance in updating the setup README.

Thank you.

[seertenedos]

I am interested in testing it and i am actually in the process of trying to sync 2 servers as the minute all running in docker

[sbcrumb]

Count me in.

[lusky3]

I could assist. I have a 3-instance container setup ready to go.

[RXWatcher]

I have multiple servers on different continents and would love this. Please count me in. -RX

[goofballtech]

running 2 servers via docker. Im in.

On Tue, Feb 4, 2020, 10:25 Ben Curtis [email protected] wrote:

Hi everyone,

You've been mentioned because at some point you have posted an issue here or participated via PR. I am migrating away from plex-db-sync to a new "instant sync" software platform I've developed called Plex Cluster.

I am looking for beta testers for the below platform. Reply here if you are interested! Features

• Syncs watched status on-demand, right away
• Uses standard Plex APIs without accessing the database
• Can work across multiple Plex servers
• Scheduled "full update" can completely sync user's watched status for any user that has been active on each Plex server since running Plex Cluster
• Syncs media that is contained on both (or multiple) servers without erroring when media does not exist
• Works for Plex.tv or Managed users

Architecture

Plex Cluster contains two applications:

• Plex Cluster Proxy, which is installed alongside every Plex server and acts as a proxy (using nginx) between Plex Media Server and the internet. This application's job is to pass any requests that come to it along to the Plex server while catching any requests that mark watched status and also forwarding them on the Plex Cluster Manager. There is one instance of Plex Cluster Proxy for each Plex server.
• Plex Cluster Manager is then used to synchronize the status between Plex servers. There is only one instance of Plex Cluster Manager which is used by all instances of Plex Cluster Proxy.

An example use case:

1. Plex client requests to watch a show with GIUD (unique identifier) ABC from the TV Shows library on Plex Server 1
2. The request goes through Plex Cluster Proxy 1 and is passed along to Plex Server 1
3. The user stops the show midway.
4. The request goes through Plex Cluster Proxy 1 and is passed along to Plex Server 1, but is also passed along to Plex Cluster Manager
5. Plex Cluster Manager takes the request and checks for any other Plex server with a library named TV Shows and GUID ABC
6. For any other servers it finds, such as Plex Server 2, it forwards the request on to them and the show is instantly marked as watched up to that midway point.
7. User switches over to Plex Server 2 and sees that they are midway through show ABC

@ChaosZero112 https://github.com/ChaosZero112 @daxxi13 https://github.com/daxxi13 @Malcolmest https://github.com/Malcolmest @seertenedos https://github.com/seertenedos @skinnyzaz https://github.com/skinnyzaz @sbcrumb https://github.com/sbcrumb @goofballtech https://github.com/goofballtech @onezero1010101 https://github.com/onezero1010101 @ddurdle https://github.com/ddurdle @dpeet https://github.com/dpeet

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Fmstrat/plex-db-sync/issues/21?email_source=notifications&email_token=AB6W4BMGDB5RFDTVSNLYAMTRBDGZZA5CNFSM4KPPOG72YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IKYP6ZQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB6W4BOFRFID2PD2UXTUT5TRBDGZZANCNFSM4KPPOG7Q .

[seertenedos]

If something is watched on both servers will it update both servers to the oldest watch date? That is one issue I currently have that has caused old watched items on one server to show as newly watched on my second server when it synced.

[fredtbrown]

I would love to test however I am running on windows and no containers.

[zerodayz1]

Running Plex on docker on two servers. This is lit, as the kids say. I'm in.

[ddurdle]

I'm in for testing. I have 4 instances to test with.

On Tue, Feb 4, 2020 at 4:47 AM zerodayz1 [email protected] wrote:

Running Plex on docker on two servers. This is lit, as the kids say. I'm in.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Fmstrat/plex-db-sync/issues/21?email_source=notifications&email_token=AAGP4NGF5TNVUYDTLVMMT4DRBE2RVA5CNFSM4KPPOG72YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKW7IPY#issuecomment-581825599, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGP4NA5QTLWMDRIFROE5U3RBE2RVANCNFSM4KPPOG7Q .

[Fmstrat]

@seertenedos @sbcrumb @ChaosZero112 @RXWatcher @goofballtech @zerodayz1 @ddurdle At some point in the near future you should expect to get an invite to participate.

@fredtbrown As stated you could run this natively, but you would either need to do things like install bash and nginx. If you are interested, I would recommend installing Docker for Windows, and running the Plex Cluster services in containers, and Plex as a system application (like you do now). Let me know if you would like to try, but during the beta I'll really be looking for issues, and won't have a lot of time for support.

@seertenedos As for "oldest watched" that was part of the reason I created plex-db-sync. The "instant" sync feature of Plex Cluster should remove that issue, as the marked times will be seconds apart unless something goes down. The "full sync" feature is really just a catch all, or a way to sync up users at the very beginning. My suggestion would actually be to sync everything with plex-db-sync, the switch to Plex Cluster. That way the times are all lined up.

[dpeet]

@Fmstrat I'm in as well!

[seertenedos]

@Fmstrat you have content on both servers covered when you watch them and http call come in which is the main one but there is 2 issues missed I think.

1. I watched something on server 1 but server 2 does not have that show at the time. Later on server 2 gets the show or episode. (Being able to trigger full update would be fine but ideally being able to set older watch dates on it would be ideal.) I am getting this as I slowly back fill new servers. (I also have a third server I have offline and just want to keep the watch status and sync them in as other servers add the shows if they add them)

2. Will you capture watch status change via sync and other non direct stream over http protocols?

3. More an very cool to have would be for the central server to have the watch status from all servers stored so if a server does or gets corrupted that is the only one with a show that watch status is not lost. I have lost watched status from Plex a few times over the years and even sometimes have some of my older watch status disappear for some unknown reason.

Your idea of interception is fantastic and using the API to set watched status is also smart. I assume for full sync you may read the database file to speed it up but write will likely be via API. I am guessing you can't set the date you watched something via API but maybe that is a special option to sync watch dates to oldest date but actually updates to database files and restarts Plex. It would not write new watch status but instead just update the ones in the database.

If you can't do all that it is fine just some ideas as the interception was a smart idea.

[daxxi13]

@Fmstrat I'd like to help too

[onezero1010101]

@Fmstrat I'm interesting in testing as well.

[hergonoway]

@Fmstrat : count me in too. Plex db sync is working great between my 2 servers, but a non-async solution is definitly what I seek for a long time :)

quick question : does the Plex Cluster Manager cache requests for a certain amount of time in case of non availibility of one or multiple node ?

[simssj]

Sign me up. I have a cluster of three plex instances, all remote from one another. One is on MacOS and the other two are on Raspberry Pis. All are running "bare metal" (without any containerization) but Docker is installed and running on each node for other housekeeping tasks. Our family uses these nodes as a distributed media center and family movie library; keeping tabs on what's been watched, where, is a real challenge. I came here to PR a modification into the current plex-db-sync script (to add a small delay while stopping the Plex servers, to allow for some settle time) but I'm prefer to get on the new Plex Cluster bandwagon.

[edrock200]

I'm interested! I have one docker instance and 5 standalones. One docker on Ubuntu, 3 centos 7 minimals and 2 centos 8 minimals.

[skinnyzaz]

I am interested in testing

[onezero1010101]

@Fmstrat Have you closed beta testing? Or are you going to release a repo that anyone willing can test? Would be nice to let everyone know if you are going to be accepting any new testers or if we just need to wait until its made public. I know I'm holding off on setting up a new cluster for this, rather then move my existing sync over. Thanks

[Deadly-Bytes]

I would like to test if possible also

[TheImpossible21]

Is it still possible to join the beta testing?

[seertenedos]

I know I still want to join as well

On Wed, 1 Apr 2020, 11:26 pm TheImpossible21, [email protected] wrote:

Is it still possible to join the beta testing?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Fmstrat/plex-db-sync/issues/21#issuecomment-607219034, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADB44LIQE4MKIIR4YXG7GTRKMXIVANCNFSM4KPPOG7Q .

[sbcrumb]

@seertenedos @sbcrumb @ChaosZero112 @RXWatcher @goofballtech @zerodayz1 @ddurdle At some point in the near future you should expect to get an invite to participate.

@fredtbrown As stated you could run this natively, but you would either need to do things like install bash and nginx. If you are interested, I would recommend installing Docker for Windows, and running the Plex Cluster services in containers, and Plex as a system application (like you do now). Let me know if you would like to try, but during the beta I'll really be looking for issues, and won't have a lot of time for support.

@seertenedos As for "oldest watched" that was part of the reason I created plex-db-sync. The "instant" sync feature of Plex Cluster should remove that issue, as the marked times will be seconds apart unless something goes down. The "full sync" feature is really just a catch all, or a way to sync up users at the very beginning. My suggestion would actually be to sync everything with plex-db-sync, the switch to Plex Cluster. That way the times are all lined up.

Any status on this? Are we still getting an invite? or has this been put on the back burner ?

[Fmstrat]

@sbcrumb Yes, an invite will come. I've been working through a launch of a government contract I work on in the midst of my company having some changes now, so timing has been tight. I haven't been able to work on the final two issues yet (Roku session variables and reconnection attempts).

[ninthwalker]

@Fmstrat Add me to your list to test when you get around to finishing things up. Very familiar with containers and clustering in general. Thanks.

[notlimahrelyt]

Very interested. I have 4 servers running docker.

[rjblake]

I'd like to participate - have 3x Plex Server (1x Synology, 1x Linux, 1x Mac OS)

[geek-baba]

I just requested similar feature, I would love to test this, I have bare metal primary plex, kube-plex on k8s cluster and 3rd one one as plain docker on OMV NAS.

[pmow]

Interested. Local/Cloud dockers with local/cloud storage. What a fantastic idea.