jhuckaby
Log Watcher succesfully connected but no live log shown
Current situation:
- 1 Master node
- 1 Worker node
- Accessing cronicle via NGINX on the master node
The jobs running on the master node correctly show live logs but the ones running on the worker node do not.
It seems the connection is successfully established
Log Watcher: Connecting to server: /...
Log Watcher: Connected successfully!
but no content is shown. After the script has finished running I can indeed see the logs.
If I run the same script interactively (without Cronicle) I can indeed see stuff being printed to STDOUT, and I know the script is running in the background.
I'm not sure if this is a problem due to the connection between frontend and worker node, or just a mismatch in what the script is actually printing to STDOUT.
I have set "custom_live_log_socket_url": "/", on both serves conf.json. Rest of the configuration is standard.
Could be output buffer autoflush? See https://github.com/jhuckaby/Cronicle/issues/364
Could be output buffer autoflush? See #364
I already found #364 but suspect this is not the culprit, otherwise I'd see the same behaviour when running interactively. Plus, I tried different executables (all with the bash plugin) and still seeing no live log until the script is finished.
I did see this behaviour sometimes on the master node in the past, but only for corner cases where Python is using tqdm to show a progress bar.
I have the suspicion that this is related to the connection between master and slave because I haven't had to do any special setup to see the live log on the master node, which shares the same OS with the worker node, besides setting the custom_live_log_socket_url .
That should be a side effect of using reverse proxy. You should have a direct connection between your browser and the node where the job is running. I think you can make it work only with a single node using a custom socket URL option. Check old issues regarding reverse proxies, in general, that could be a pain. You can check this issue for a possible workaround ("centralized level log") for multi-node setup. https://github.com/jhuckaby/Cronicle/issues/391
That should be a side effect of using reverse proxy. You should have a direct connection between your browser and the node where the job is running. I think you can make it work only with a single node using a custom socket URL option. Check old issues regarding reverse proxies, in general, that could be a pain. You can check this issue for a possible workaround ("centralized level log") for multi-node setup. #391
That sounds more like it. I thought that for the worker nodes all requests, including the ones for the live logs, are proxied through the master node. Let's see if I can make it work with the workaround. What if I give the worker node a FQDN? Would it work then?
P.S. @mikeTWC1984 your fork looks amazing. Any way I could try it out without using docker by just applying your modifications on top of the standard Cronicle version that I have installed?
@guidocioni yes, you can run it without docker. It can be installed the same way as the original cronicle. You can also run "bundle.sh" script, which will generate "dist" folder (cronicle-edge home dir). To import jobs from the previous install, run bin/control.sh export (in the old cronicle folder), then go to edge UI -> schedule -> import, and paste it there. In theory, you can also copy or link "data" folder from the old install to the new one, but you likely face some issues
@guidocioni yes, you can run it without docker. It can be installed the same way as the original cronicle. You can also run "bundle.sh" script, which will generate "dist" folder (cronicle-edge home dir). To import jobs from the previous install, run
bin/control.sh export(in the old cronicle folder), then go to edge UI -> schedule -> import, and paste it there. In theory, you can also copy or link "data" folder from the old install to the new one, but you likely face some issues
ok thanks. means that basically they're two completely separated installations. I thought I could just apply the modifications of your brqnch over my version of cronicle and then use that.