motioneye
motioneye copied to clipboard
motioneye-project
Plan for continuing with this project
@cclauss @Mictronics @jmichault and everyone else, let's make a plan how to continue with this project, now moved to the new @motioneye-project GitHub organisation.
First of all, everyone who has contributed to motionEye before, or wants to do so, please feel free to join in, also I think we can add everyone who is interested to the organisation.
I think we agree that Python 3 support is the first thing to achieve, given that Python 2 has reached EOL since over 2 years already, @jmichault and @Mictronics have Python 3 compatible forks from where we can commit back, and on this repo we have the python3 branch and a bunch of open pull requests for further related fixes.
I personally would go this way:
- [x] Merge the
python3branch: #1572 - [x] Rebase and merge Python 3 related pull requests
- [x] Rebase and merge https://github.com/Mictronics/motioneye
- Ah, and that one is here as PR already: #2258
- Cannot be merged as of too many conflicts. Partly resolved in #2318 and I found the standalone PRs for some of those changes as well, which we should merge preferably then.
- [x] Rebase and merge https://github.com/motioneye-project/motioneye.eo
- A you did already start, so it's now python3-intl branch 🙂.
Approximately from simple to difficult, regarding possible conflicts 😉.
I would like to continue contributing in my way (docs and support, rather than code) if you'd like, as a collaborator.
I just invited you to the organisation. This way you should be able to create branches right here on the main repo, but I think this was already the case before, since you were a collaborator already.
I don't know anything about programming, especially in the github environment, I was able to edit the wiki documents, and assist end users as needed, and marking issues with the various flags (which hadn't been done prior to my being given collaborator status. I am a script-kiddie in python2 & 3, pretty good with bash and dos command (win) scripting, and so forth. jasaw (video/motion issues) and jawsper (brought an image of motionEyeOS) are both very good in what they do. I am a professional beta tester, too, since back in the Zoom Modem voicemail days (for the C64 & Early PCs), I can break things quite easily. I am an Enterprise (Level 2 & 3) Senior Help Desk, for the past 25+ years. I do translations between end users and developers. I have a lot of resources available for testing (VBox VM Server Ryzen7 CPU 32GB RAM/10TB Storage, Gbit network, Pi1, 2, 3B, 3B+, 4B-4GB and 4B-8GB, PiZeroW, 8 Network IP cams, 2 USB Cams, 1 CSI PiCam) and used them regularly for helping out here. I do docker some, too. If you go through the WiKi, I've done a lot of updates to a lot of the pages. If someone uses an esoteric (read: non-Debian based) OS, I'm willing to try to get it to work, and then document it... Let me know if you can continue to use my 'services'....
That is very valuable, always good to have someone with some focus on the docs, issues/bug report review and testing side. Also Docker experience is good, which I lack, while I see the Dockerfile here.
I haven't had a look into motionEyeOS yet. I personally would concentrate on getting motionEye itself up and ready for Python 3 first, and then update Docker images and motionEyeOS in a second step. But those two definitely need others with more insights and experience with Docker and BuildRoot.
I would like to contribute, too, if my spare time allows me to do that. I am a long time developer and familiar with Python 3, Docker, Kubernetes.
My preference would be to add #2308 as step 0. given that it is now green and puts in place some important guardrails.
@cclauss Okay, it doesn't hurt anyway. I made some comments as minor enhancements, otherwise good to be merged from my end.
I just invited you both as members to the orga. I'll send out invitations to all motionEye collaborators, which have their permissions preserved anyway. That way we also see who of those is still active and interested to continue contributing to the project 🙂.
Hello, my python3-intl branch is ready to be merged : https://github.com/motioneye-project/motioneye/pull/2313
Awesome, many thanks for your work. From my end, I'd merge python3 first into dev and then rebase and merge PRs with python3 as base onto dev. It practically may not make any difference, after I resolved conflicts between those branches with the result that python is now up-to-date with dev, but it somehow feels more natural to me 😅.
@kleini, Hello, I tried to modify Dockerfile to take into account all the changes made for python3 and internationalization. Can you take a look to see what's left to do?
From what I see it contains all changes which were applied to dev/python3 branches before, so should be fine. Only thing which went inside as well is that python3-pip is not purged for the final image, while python-pip was purged before. If I'm not mistaken, motionEye does not require pip, so to reduce the container size a bit, this could be re-added to the list of purged packages. But as always, also since you switched (reasonably!) to Bullseye, it simply needs to be tested 🙂.
Just had a quick look on the "Install in Docker" wiki page and the Dockerfile. Building the container image works and the container works. Getting a picture from my network cameras works as well. So for a first step everything is fine. General questions from my side would be:
- Where is CI gone? I would like to build container images in CI and publish to docker hub or into one of the other container image registries.
- Today I would not build applications inside containers any more. I would build the outside and copy over the build artifacts into the container to reduce container size. But looking at the Dockerfile, a small container size was not a primary target before. Instead the container image should be able to do anything.
- Initial configuration file /etc/motioneye/motion.conf does not work.
- Help for configuration options does not work.
- Is there some Python 3 base container in some registry? Would that be the better base for a container image?
- How does motionEye "talk" to motion? I think, it would be usefull to separate motionEye und motion into separate containers if communication is possible across container boundaries.
- Move away from Docker and use buildah, podman, cri-o, containerd to get rid of no real open source dependencies.
Where is CI gone?
https://github.com/motioneye-project/motioneye/tree/dev/.github/workflows
Initial configuration file /etc/motioneye/motion.conf does not work.
Because motion is controlled by motionEye, and the motion.conf settings needed are in /etc/motioneye/motioneye.conf
I think, it would be usefull to separate motionEye und motion into separate containers if communication is possible across container boundaries.
Why? Why increase the complexity? Multiple points of failure? Diagnostics issues when something does break? Different versions / builds / new dependencies when something is updated? Have one force update the other? When motion updates and breaks the 'current' version of motionEye (it has happened) how do you control either? With fixed versions in a single container, you update as a package. The Docker IMO brings everything into the package needed to run to assist with the above questions.
From what I see it contains all changes which were applied to dev/python3 branches before, so should be fine. Only thing which went inside as well is that
python3-pipis not purged for the final image, whilepython-pipwas purged before. If I'm not mistaken, motionEye does not requirepip, so to reduce the container size a bit, this could be re-added to the list of purged packages. But as always, also since you switched (reasonably!) to Bullseye, it simply needs to be tested 🙂.
pip is purged (and the ability to exec in the container) to prevent people from making changes (similar to the RO file systems in motionEyeOS), from what I've discovered. Pretty effective. motionEye doesn't need pip after installed. Pip is only needed after install in a 'normal' install to do an upgrade:
pip install motioneye --upgrade
I think, it would be usefull to separate motionEye und motion into separate containers if communication is possible across container boundaries.
Why? Why increase the complexity? Multiple points of failure? Diagnostics issues when something does break? Different versions / builds / new dependencies when something is updated? Have one force update the other? When motion updates and breaks the 'current' version of motionEye (it has happened) how do you control either? With fixed versions in a single container, you update as a package. The Docker IMO brings everything into the package needed to run to assist with the above questions.
Because this large container needs to be rebuild completely for every little change. Maintaining a deployment is much easier, if different tools - motionEye and motion in this case - are in separate containers. The correct combination of containers can be controlled with various tools like docker-compose, kubernetes, Helm charts. And furthermore motion requires a lot of CPU resources. Having those split across multiple containers improves overall performance. An ideal case would be to have every process running in its own container. A container based on some normal OS like Debian Buster in this case, is often a problem with containers. Containers should not run an OS, making those containers error prone and vulnerable. As previously stated, a container just runs a single process. Having tools separated in this way, allows to have a motionEye container image, which only contains some Python runtime and motionEye, nothing else. If motionEye has any vulnerabilities the underlying container allows nearly no operations except running motionEye.
An incompatibility problem between motionEye and motion is a problem of pinning versions of both tools. This is not solved by a single container image. This must be solved by pinning these versions and it does not matter, whether they are in the same container image or in separated ones. And the motionproject already does provide a container image, so we don't need to download some built motion any more and include that in our container image.
Are you going to support cross host setups too, when people using clusters install motion on one host, and motionEye on another host?
Are you going to support cross host setups too, when people using clusters install motion on one host, and motionEye on another host?
This would be an interesting approach.
Because this large container needs to be rebuilt completely for every little change.
This is not really true. There is tons of documentation about the order of commands in a Dockerfile to reduce long builds.
The advantage, IMO, of a complete container, is you don't care what the docker host is running. The same docker for motionEye runs on all the debians, the RHs, the Arch, whatever. (Theoretically, Windows, too) without consideration of what is needed vs what is already installed on the host, and no communications issues between the packages (if the sandbox system WORKED, they would not be able to easily talk to each other, or possibly other containers...)
Because this large container needs to be rebuilt completely for every little change.
This is not really true. There is tons of documentation about the order of commands in a Dockerfile to reduce long builds.
If you read the option 3 in Install In Docker it pulls existing base 'images' of the OS, installs only what motionEye needs, and doesn't need 'get a cup of coffee' break. This was modified because of the age of software included in the existing version, and documented in the spirit of open source / open documentation on how to 'make your own mods'. If it goes the way of 'single process per container, and the complexities of managing many (possibly many, many?) packages and containers, I may fork the project (I will for me) because I deal with average newby non-programmers and developers who want to learn how to make changes or fit their situation without the need to go through a 4 year IS college degree to understand it.
IMO motionEye is not a corporate project, but for the average street Joe who plays with RPis and SBCs or is getting into Linux on PCs and has a few IP or PiCams laying around. Corporate developers and programmers seem to not think about how a product is used, they write for a corporate environment, not a home environment.
Maybe a bad analogy, but think about a dishwasher. What if you had to have a BUNCH of devices to do the same job as the single dishwasher? A machine to do the pre-rinse, a device to do the primary wash cycle, a device to do a final rinse, a device to do a sanitizing rinse? Would you buy all those devices or a single device?
My contribution to the previous version of motionEye was to support the end user (mostly street Joe), with testing to determine if their issue was Bug or PEBKAC, bad documentation, or something else. If I wasn't able to tell up front, I tried to reproduce the issue, and point it in the right direction. In the process, some things were simplified and better documented. I found ways around the pip2 issue and included them to extend the longevity of the project.
I've been digging into the docker image issue to get it out of Debian 9 and up to Debian 10 (and soon Debian 11, maybe) and into motion 4.3.2. because I think this is a better product than most of the other options out there.
IMO split containers is overkill here and causes more issues (complexity for users) than what it may solve (the ability to update it independently of motionEye). Also AFAIK motionEye executes motion directly and accesses its config file, so it isn't possible.
Y'know, I just found a reason to possibly split motionEye from motion from motionEye, however, I still do not think it's a "Good Idea"(tm) for Joe Average-user, and that is the Hub Only functionality (no motion detection, but to control and view other motionEye Cameras. There isn't any motion detection capability, or transmogrifying video capability... Again, I don't think this is a needed option for motionEye... https://github.com/motioneye-project/motioneye/wiki/Installation#the-motion-daemon
Not a CCrisan project, or motionEye project. There are a number of unofficial apps for phones, too...
the Hub Only functionality
If one wants that, then one obviously wants things to be split explicitly for some reason, and in this case Docker may not be the preferred solution anyway. IMHO our Docker image should cover the average case and address especially the users who want to spin up motionEye as fast and easy as possible, which is a single container image so you can do docker run are things are working.
I reviewed all open pull requests now. Some became obsolete in the meantime (closed), some partly (asked for rebase), some are quite nice and seem safe to apply (asked for rebase and in case asked for review from some of you guys), some merged already, some require further work (converted to draft).
I hope we get outstanding replies soon, so rebases are done and we can review and in case merge those. If you guys find time to answer the review requests, that would be great, so in case we can merge what is ready from my end.
I'm quite happy to see, that there it nothing really depending or affected much by the move to Python 3. Most conflicts are due to extra moved to motioneye sub directory and utils.py + handlers.py being split into dedicated files, but on first view it doesn't look too complicated to migrate the changes over.