Problem running linux scripts

[susobaco]

Describe the bug I have some scripts programmed to raise and lower volumes, run vlc, etc, etc, etc. After a while, the scripts stop executing and the error message: QProcessPrivate::createPipe: Cannot create pipe 0x7f0320139030 (Too many open files)

Expected behavior that the scripts work

-Linux

• Q Light Controller Plus version 4.12.4 and Q Light Controller Plus version 4.12.3
• Debian bullseye

Additional context Add any other context about the problem here.

[susobaco]

I don't know if this problem can be solved with what is described in this page: https://www.baeldung.com/linux/error-too-many-open-files

[sbenejam]

Hello Suso, can you post in the QLC+ forums details of your system, operating system, projects and scripts, etc. as you can provide. There's a lot of users there who can help you solve them or find out what's wrong.

Missatge de Suso Comesaña @.***> del dia dc., 1 de juny 2022 a les 13:54:

I don't know if this problem can be solved with what is described in this page: https://www.baeldung.com/linux/error-too-many-open-files

— Reply to this email directly, view it on GitHub https://github.com/mcallegari/qlcplus/issues/1339#issuecomment-1143508938, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO5H44TGBXHS3PI3S6V4Q3VM5FNPANCNFSM5XQOSE7A . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[susobaco]

Thanks, as soon as I have some time, I will visit the forum.

[mcallegari]

So basically you opened an issue report regarding a misterious script you haven't shared, without describing the actual communication channel with QLC+ (web?? other??) and you want QLC+ to be fixed somehow. Sorry but this is not the way to report an issue. As suggested by Santiago, please use the forum like everybody else.

[susobaco]

Sorry, I have been trying to register in the forum for days and it is impossible. I keep trying.

[susobaco]

I launch the scripts from the qlc+ virtual console, with a button that executes it. Here is an example script.

#!/bin/sh xdotool set_desktop 3 player-control vlc play exit

When I start running qlc+ , the scripts work, but, after a while, they stop working and I get the error described above. It is as if the program opens many pipes and does not close them. I increased the limits according to the instructions in the link and for now they work, but the processes don't seem to close. Sorry about the forum, but I really am not able to register.

[mcallegari]

Does the script actually end or some process in it keeps it pending? You can probably inspect this with the 'ps' command.

Also, have you tried to "externalize" the script into a bash script and invoke it with &? Something like /path/to/script/myscript.sh &

[sbenejam]

I tested this command in Ubuntu 22.04. The command pal

Missatge de Massimo Callegari @.***> del dia dt., 7 de juny 2022 a les 15:50:

Does the script actually ends or some process in it keeps it pending? You can probably inspact this with the 'ps' command

— Reply to this email directly, view it on GitHub https://github.com/mcallegari/qlcplus/issues/1339#issuecomment-1148704241, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO5H46XFCLRL6MT2APIGQ3VN5HQRANCNFSM5XQOSE7A . You are receiving this because you commented.Message ID: @.***>

[sbenejam]

I used in a console playerctl -p vlc play playerctl -p vlc stop xdotool command is not needed to control vlc with playerctl command.

Missatge de Santiago Benejam Torres @.***> del dia dt., 7 de juny 2022 a les 19:12:

I tested your commands and I cannot run player-control. Have you installed anything that provides the player-control command?. In Ubuntu I found only playerctl to control vlc.

Missatge de Massimo Callegari @.***> del dia dt., 7 de juny 2022 a les 15:50:

Does the script actually ends or some process in it keeps it pending? You can probably inspact this with the 'ps' command

— Reply to this email directly, view it on GitHub https://github.com/mcallegari/qlcplus/issues/1339#issuecomment-1148704241, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO5H46XFCLRL6MT2APIGQ3VN5HQRANCNFSM5XQOSE7A . You are receiving this because you commented.Message ID: @.***>

[susobaco]

I believe that the scrip ends, since it carries the command "Exit 0". I have also tried running it from a script called "vlc_play.sh" , which contains those commands and it does the same. When I start qlc+ , the output from the command :

#lsof -p $(pidof qlcplus) .... qlcplus 1224 manteniment 47r a_inode 0,13 0 10093 inotify qlcplus 1224 manteniment 48u a_inode 0,13 0 10093 [eventfd]

After executing the command ( which is associated with a button that executes a syscommand ) the output is:

#lsof -p $(pidof qlcplus) .... lcplus 1224 manteniment 47r a_inode 0,13 0 10093 inotify qlcplus 1224 manteniment 48u a_inode 0,13 0 10093 [eventfd] qlcplus 1224 manteniment 50w FIFO 0,12 0t0 22078 pipe qlcplus 1224 manteniment 51r FIFO 0,12 0t0 22079 pipe qlcplus 1224 manteniment 53r FIFO 0,12 0t0 22080 pipe qlcplus 1224 manteniment 55r FIFO 0,12 0t0 22081 pipe qlcplus 1224 manteniment 57u a_inode 0,13 0 10093 [pidfd]

If I run another script again, the output is:

#lsof -p $(pidof qlcplus) .... qlcplus 1224 manteniment 50w FIFO 0,12 0t0 22078 pipe qlcplus 1224 manteniment 51r FIFO 0,12 0t0 22079 pipe qlcplus 1224 manteniment 52w FIFO 0,12 0t0 29072 pipe qlcplus 1224 manteniment 53r FIFO 0,12 0t0 22080 pipe qlcplus 1224 manteniment 54r FIFO 0,12 0t0 29073 pipe qlcplus 1224 manteniment 55r FIFO 0,12 0t0 22081 pipe qlcplus 1224 manteniment 57u a_inode 0,13 0 10093 [pidfd] qlcplus 1224 manteniment 58r FIFO 0,12 0t0 29074 pipe qlcplus 1224 manteniment 60r FIFO 0,12 0t0 29075 pipe qlcplus 1224 manteniment 62u a_inode 0,13 0 10093 [pidfd]

Debian is configured by default with a limit of 1024. The output of the command is:

$ulimit -n 1024

So, if I run the scripts a number of times, there comes a time when I get the error:

$QProcessPrivate::createPipe: Cannot create pipe 0x7f0320139030 (Too many open files)

If I increase the limit to 500000 as proposed in the link, I don't get the error.

I don't know if this is the solution or if something can be done to make qlc+ close these "pipes".

Note:

Qlc+ I use it to project presentations with libreoffice impress, pdf with impressive, play audios with audacious and project videos with vlc on an extended desktop that exits to a projector. I use the xdotool command to switch virtual desktops. Vlc only runs on desktop 4. So, if I run an impress presentation on desktop 1, when I switch desktops, with a good compton configuration, I get a fade of images on the extended desktop.

(P.S. I still can't register, is there a problem with the forum or am I just clumsy?)

[mcallegari]

BTW this is related to #1270

As for forum, new users are successfully registering daily. Perhaps try with another email address

[mcallegari]

I have pushed a change that fixes the issue on the procfix branch. Please help to test. Please note though that this changes the behavior of a system command execution. A Script must have a proper wait entry to keep the Script running for as long as the system command will last (e.g. play a 10 seconds video, wait 10 seconds) If no wait time is added, the Script will end immediately and the invoked process killed accordingly. I might need to fix the case where a Script is a Chaser step, then the duration is overridden.

@chille might be interested in this too.

[hjtappe]

Hello @mcallegari, I would object that the script needs to know the timing expected from the environment. Most scripts just trigger another program or remote entity (e.g. to update a screen or start something), but are not made to match the duration of the calling step, e.g. through a web hook to set the room lights on a house automation it will terminate before the new hous scene is done and the sequence in qlcplus continues. Maybe, I misunderstand your point, though.

Can we fork the qlcplus process and let the script perform whatever needs to be done and ignore how long it is running, otherwise? The script would then terminate and the forked process would as well whenever the script deems it time to terminate.

I might be able to look into this in a few weeks and try to join you in finding a solution.

[mcallegari]

The problem here is that QProcess leaves file descriptors open forever (at least on Linux), unless you explicitly tell Qt to close/terminate/delete them. Not knowing the duration of a Script would cause to terminate it too soon. How to solve this then?

[hjtappe]

I would dig into a solution with the QProcess::startDetached Architecture, but I have less research and experience than anyone in this chat how to manage the pipes closure yet. To me, that directs into what I would expect, but I may be wrong. It should be possible to detach and monitor or timeout in the detached process to clean things up because when calling a command from within qlcplus it's more a fire-and-forget with the baseline that if the called program or script is behaving well, terminating sometime and we don't clutter the system resources. Is my assumption wrong?

[mcallegari]

@sbenejam I'd ike to hear your opinion too on this since I know you use/used Scripts in a few projects. Thanks

[sbenejam]

Massimo, I tested with system scripts or commands long time ago to control libreoffice or pdf viewers to make presentations. From this case I have tested the version of GIT not the procfix yet. With the playerctl -p vlc command play the processes remain open until QLC+ stops. The player-control command that Suso uses in Debian don't exists in Ubuntu. I think it depends on whether a system command or script that can contain other system commands that can end up either in a good or bad way. For example mpv video.ogg plays a video with a duration that QLC+ does not know. But if you want to play the video for only 10s a wait 10 maybe enough, I don't know how it can be done. playerctl -p vlc does nothing if VLC is not running. Maybe per playerctl a wait 1 is enough, so the only thing that does is send an opportune signal, stop, pause, etc. to VLC. If I have some time this weekend I'll test the profix commit with different commands, playing vídeos and audio files. Regards

Missatge de Massimo Callegari @.***> del dia dj., 16 de juny 2022 a les 18:20:

@sbenejam https://github.com/sbenejam I'd ike to hear your opinion too on this since I know you use/used Scripts in a few projects. Thanks

— Reply to this email directly, view it on GitHub https://github.com/mcallegari/qlcplus/issues/1339#issuecomment-1157868962, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO5H45GHD7WQTPI4SJWKKLVPNH6TANCNFSM5XQOSE7A . You are receiving this because you were mentioned.Message ID: @.***>

[hjtappe]

@mcallegari, the changes on the branch seem to make sense, better than a detached start, I think now. But do they leave sufficient time for commands to execute?

If I run time playerctl -p vlc pause with an open player it takes up to 60ms on the commandline.

I wonder how we can reproduce this. When I add the systemcommand to engine/test/script/script_test.cpp, compile and execute it, it has no effect on the running VLC. Either it is not executed at all, it is terminated prematurely or there is another failure which I don't know of yet. Stderr seems to be silenced.

I see two kinds of systemcommands:

1. Forground programs started at that point in the script and terminated at the end of the script as they run in the foregound (like a VLC being fired up and stopped like in the documentation)
2. Background programs calling a hook on e.g. a DBUS (playerctl) or REST (curl) interface to remote-control another program, which may take some second until they should terminate.

As we know of programs which shall terminate at the end of the script and some which may take a second or more:

• Should we consider to add documentation in the manual to add a "wait 3s" after the systemcommands because they will be terminated
• or add the expected execution time (as an argument to the systemcmd) and track and wait before terminating them or make use of waitForFinished() before we delete the process
• or distinguish systemcmd into systemcmd and systemservice

How can we add a test for the behavior? I tried with bash -c "wait 3s && date > /tmp/script_test.log", but had no success in facilitating the script_test.cpp for this matter yet.

[mcallegari]

Fixed by #1358

[sbenejam]

I tested your commands and I cannot run player-control. Have you installed anything that provides the player-control command?. In Ubuntu I found only playerctl to control vlc.

Missatge de Massimo Callegari @.***> del dia dt., 7 de juny 2022 a les 15:50:

Does the script actually ends or some process in it keeps it pending? You can probably inspact this with the 'ps' command

— Reply to this email directly, view it on GitHub https://github.com/mcallegari/qlcplus/issues/1339#issuecomment-1148704241, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABO5H46XFCLRL6MT2APIGQ3VN5HQRANCNFSM5XQOSE7A . You are receiving this because you commented.Message ID: @.***>