girder_worker
girder_worker copied to clipboard
girder
Distributed task execution engine with Girder integration, developed by Kitware
There's no reason this couldn't start before the full refactor, at least some paragraphs outlining the vision of Girder-Worker and what are the major paths.
I'd want to be able to enable logging of the system service of girder_worker installed by the Ansible role.
see https://github.com/girder/girder_worker/pull/97 At least provide the option to set up cron jobs at some scheduled interval to call Docker GC.
Similar to #91, this would provide the JSON metadata for girder inputs to the processing task if requested.
This would just be an additional `metadata` field on a Girder output binding that would get set on the item after any data upload.
If we need to move away from RabbitMQ it might be possible to implement a results backend that puts the results directly into Girder. http://docs.celeryproject.org/en/latest/userguide/tasks.html#task-result-backends
We rate limit the flushing of output up to the job log, but we do so synchronously, so undesirable things can happen. Namely, if a bunch of output is written...
In order to construct workflows more easily from separate JSON components, enable `task_uri` as an alternative to the `task` key in workflow steps. These would be resolved to task specs...
Hi, Does the girder worker support parallel execution of tasks inside a workflow. For instance, lets say I have a workflow having tasks T1,T2,T3,T4... Is there a way I can...
Hi, I was interested to know if girder worker can use XML or JSON spec to populate and run a workflow. I haven't seen such so far in the doc....