jaruba
Headless Version
Due to the fact PowderWeb is dependent on xdg/gui for intial login. Containerising such and amazing tool had become a nightmare. Is there any plan on releasing a docker container ?
This may help with head start
FROM ubuntu:18.04
RUN apt update && \
apt install -y git curl && \
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.34.0/install.sh | bash && \
. ~/.bashrc && \
nvm install 8.10.0 && \
. ~/.profile && \
git clone https://github.com/jaruba/PowderWeb.git && \
cd PowderWeb && \
npm install && \
cd public && \
npm install -g bower && \
bower install --allow-root
WORKDIR /PowderWeb/public
ENTRYPOINT ["/bin/bash", "-c"]
CMD ["npm", "start"]
A docker container would be cool. The UI could work as Web only, so a version of Powder Web without Electron should be possible.
But there is a big drawback with this, Electron isn't used just for the Windowed App, it's also used for select folder / file boxes (which Node.js has absolutely no native method for). But selecting folder / file can only be done locally anyway, so it could be changed to a string input that should point to folders / files..
My biggest limitation right now is finding the required time to work on this..
I've renamed this to "Headless Version" and tagging it as a priority task. There might be many other good reasons to have a headless version. It will stay as a priority task unless I can find good enough reasons why Powder Web would dependent on Electron.
@tekicat I've made a separate branch for the headless version. And did get a version of Powder Web to work completely without Electron.
I never tested it with Docker though, it would sure help if you could test it..
If the lines you previously posted are correct, to use the headless branch you should use:
FROM ubuntu:18.04
RUN apt update && \
apt install -y git curl && \
curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.34.0/install.sh | bash && \
. ~/.bashrc && \
nvm install 8.10.0 && \
. ~/.profile && \
git clone -b headless [email protected]:jaruba/PowderWeb.git && \
cd PowderWeb && \
npm install && \
cd public && \
npm install -g bower && \
bower install --allow-root
WORKDIR /PowderWeb/public
ENTRYPOINT ["/bin/bash", "-c"]
CMD ["npm", "run", "start-headless"]
@jaruba With the below docker file
FROM node:8-alpine3.10
RUN apk add git && \
npm install -g bower && \
addgroup -S powderweb && adduser -S powderweb -G powderweb && \
mkdir /app && \
chown -R powderweb:powderweb /app/
USER powderweb
WORKDIR /app/
RUN git clone -b headless https://www.github.com/jaruba/PowderWeb.git && \
cd PowderWeb && \
npm install && \
cd public && \
bower install
WORKDIR /app/PowderWeb/public
ENTRYPOINT ["/bin/sh", "-c"]
CMD ["npm run start-headless"]
I get a 404 on opening http://localhost:3000/auth?token=<master key>
workspace/PowderWeb/docker# docker run -p 3000:3000 powderweb:18.04
> [email protected] start-headless /app/PowderWeb
> node main.js
[ '/usr/local/bin/node', '/app/PowderWeb/main.js' ]
master key: b1b506746eb647bdee953b35fe1abe13
connect ECONNREFUSED 127.0.0.1:80
Am I missing something here ?
connect ECONNREFUSED 127.0.0.1:80
This seems to be the only error, and it says that something is trying to connect to localhost:80, everything should work on port 3000 so maybe this docker command is wrong somehow?
docker run -p 3000:3000 powderweb:18.04
You could also try using the LAN IP instead of localhost..
Same here, even before trying to open any browser. Just launching "npm run start-headless" get those 2 lines of output (master key + connect ECONNREFUSED 127.0.0.1:80)
Then if I try to open a browser (through internet in my case), I get "404 File Not Found" in the browser and on the server I get something like:
(node:249) TimeoutOverflowWarning: 9007199254740991 does not fit into a 32-bit signed integer.
Timer duration was truncated to 2147483647.
(Use node --trace-warnings ... to show where the warning was created)
(node:249) TimeoutOverflowWarning: 9007199254740991 does not fit into a 32-bit signed integer.
Timer duration was truncated to 2147483647.
I used a totally different DockerFile (to test) I doubt this is linked to Docker
FYI, I fiigured out this issue... After the npm install, I had to run a "npm run build" to build the dist folder....
Coming back again for updates on the topic... So after the build + removing this warning which was annoying (but not an issue) There was an issue with video-js. The version present in 0.9 (which is basically commenting some code) is working fine with this headless version.
So now I confirm, that the headless is working in docker. Need to merge all the new changes, and try to keep this version up to date in the future :)
@Tekka90 Wow, I'm glad to hear u got it working, but the headless branch still needs to be updated to the latest version, and I should release it separately too.
Sorry for the late response, I was on vacation. I'll look into it a bit more too when I have some time, and I always wanted to make an official release of the headless version available.
@Tekka90 I updated the headless branch to the latest version, but did not test it much, please test it if possible, and also paste the Dockerfile you used that worked for you so I can look into making an official Docker image for Powder Web.
@Tekka90 if all that was needed was the dist folder, then we should also be able to use npm run build-front instead of npm run build, which is much more lightweight.
Hi @jaruba
- Here is my DockerFile FROM node:alpine3.12
RUN apk add git &&
npm install -g bower &&
addgroup -S powderweb && adduser -S powderweb -G powderweb &&
mkdir /app &&
chown -R powderweb:powderweb /app/
USER powderweb
WORKDIR /app/
RUN git clone -b headless https://www.github.com/jaruba/PowderWeb.git &&
cd PowderWeb &&
npm install &&
cd public &&
bower install &&
sed -i 's/Number.MAX_SAFE_INTEGER/2147483647/g' /app/PowderWeb/server/index.js &&
npm run build
WORKDIR /app/PowderWeb
EXPOSE 3000 CMD npm run start-headless
I did the sed to get readable logs otherwise am flooded with warnings...
-
Thanks a lot for the merge to 0.9.0 !!
-
It is working for the navigation.However, the bin folder is not present, so for example when trying to add a Torrent, I get «/bin/sh: /app/PowderWeb/bin/torrent-thread/torrent-thread: not found » I m really not familiar with node.js, but I assume something is wrong in the build process ?
I can add that the bin folder with its content is there .. In the container if I try to launch torrent-thread I get the same « not found » even though it is there... I try to launch ffmpeg, no problem there, so something is not working with torrent-thread
@Tekka90
well, i made torrent-thread too, so i'm sure it can be solved
what OS and what architecture are you running your Docker installation on?
@jaruba The host machine should not matter... the OS torrent-thread is running on in minimalist (alpine, node version as you can see in the Dockerfile. In case it can matter, the host is a Debian Stretch. Nothing much running there outside of docker... Would be useful to get more logs :) can we enable more verbosity in torrent thread ?? BTW if you want to debug, this is one the huge advantage of docker, you should be able to reproduce the issue just by running the container on your machine.
@Tekka90 i'll try to answer as best i can:
- the warning you were getting:
(node:249) TimeoutOverflowWarning: 9007199254740991 does not fit into a 32-bit signed integer. Timer duration was truncated to 2147483647.appears only if you use a newer version of node instead of node v8.12.0 (which is recommended for this project, as the readme states: Node v8.12.0 and NPM v6.4.1 were used for this project, it should work with newer versions too, but i can't vouch for stability on them) torrent-thread(along with all other binaries needed for this project, other binaries are:subtitles-thread,ffmpegandyoutube-dl) are all downloaded for the currently running OS and architecture duringnpm install, my guess is that the download failed for one of the next reasons: unsupported OS / architecture (doubtful), newer node / npm version used (possible), random download failure (possible: download server unreachable, download interrupted, etc.) or some other unexpected reason
There's a chance that torrent-thread might be there if you just rebuild the Docker image, but I strongly recommend that we use the expected Node and NPM versions for this in order to guarantee stability of the installation and app.
@jaruba Sorry I missed the part about needing a specific version of Nodejs sorry :( So now I did use 2 different images with Node 8.12.0 + rpm 6.4.1. One based on Alpine, one based on Debian In both cases, now I have an error at launch:
/app/PowderWeb/node_modules/youtube-dl/lib/youtube-dl.js:33 if (!fs.existsSync(ytdlBinary)) { ^
ReferenceError: ytdlBinary is not defined
at Object.node main.js
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start-headless script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
I assume something is missing in my Image :(
This is a certificate issue while npm install
Error: certificate has expired
at TLSSocket.
I know: https://github.com/przemyslawpluta/node-youtube-dl/issues/310
And the error you got from youtube-dl when running the app is because of the same reason, that error shows when the binary was not downloaded.
I'm more curious why it worked for you to install youtube-dl 7 days ago, as this certificate issue is pretty old, and should of happened then too.
I think this is why torrent-thread didn't download either, because youtube-dl failed during npm install and all the install scripts after it were never started.
I think i may be able to fix this by disabling certificate validation in my fork of node-youtube-dl by setting strictSSL to false here:
https://github.com/jaruba/node-youtube-dl/blob/master/lib/downloader.js#L18
I'll try it a bit later tonight.
@Tekka90 I made the change, but did not test it. I think this has a good chance of fixing everything, please try to build the docker image again at your earliest convenience.
@jaruba Good news: Your last fix is working, now I am able to get the YouTube-dl installed correctly in alpine os Bad news: back to error : torrent child process exited with code 127 /bin/sh: /app/PowderWeb/bin/torrent-thread/torrent-thread: not found
Again, the file /app/PowderWeb/bin/torrent-thread/torrent-thread does exist in the container, and if I try to launch it using command line, I get the same result !
For reference, this is the new Dockerfile: ` FROM node:8.12.0-alpine
RUN apk add git &&
npm install -g bower &&
addgroup -S powderweb && adduser -S powderweb -G powderweb &&
mkdir /app &&
chown -R powderweb:powderweb /app/
USER powderweb
WORKDIR /app/
RUN git clone -b headless https://www.github.com/jaruba/PowderWeb.git &&
cd PowderWeb &&
npm install &&
cd public &&
bower install &&
cd .. &&
npm run build
WORKDIR /app/PowderWeb
EXPOSE 3000 CMD npm run start-headless `
Could be linked to another issue I see at build time :
[email protected] postinstall /app/PowderWeb/node_modules/window-prebuilt node install.js
(node:17) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 SIGINT listeners added. Use emitter.setMaxListeners() to increase limit
FYI, linked to my previous message, the issue with torrent-thread is not linked to the warning... I fixed the warning (npm install -g npm@next) And still get the same issue :( FYI: ls -la /app/PowderWeb/bin/torrent-thread total 44120 drwxr-xr-x 2 powderwe powderwe 4096 Jul 15 08:01 . drwxr-xr-x 6 powderwe powderwe 4096 Jul 15 08:01 .. -rwxr-xr-x 1 powderwe powderwe 45170269 Jul 15 08:01 torrent-thread
/app/PowderWeb $ /app/PowderWeb/bin/torrent-thread/torrent-thread /bin/sh: /app/PowderWeb/bin/torrent-thread/torrent-thread: not found
@Tekka90 the "not found" error can also happen if a html page / http error leaked into the torrent-thread file when it downloaded, is it possible to open /app/PowderWeb/bin/torrent-thread/torrent-thread with a text editor? and if it includes plain text instead of a binary, that could lead to a clue of what's going on
Yes I can open it, and the content is binary, I also tried to change rights (777), same issue...
I will try to build it from source and see if I have the same issue or not
@Tekka90 building from source is more trouble then it's worth, i'll add some logs to the install scripts of torrent-thread and subtitles-thread so we can understand what's going on better.
@Tekka90 Another reason why linux shows that error (presuming that the downloaded file is not corrupted in any way, and presuming it has the executable flag chmod +x <filename>) is if the binary is the incorrect architecture. For example, if the system is x64 and the file is x86 or vice versa.
It should be possible to see the architecture of the file with file <filename> command
@jaruba The issue was with alpine... you need to install some glibc libraries.... I preparing a clean Dockerfile and share it there, but I got it to work now :)