cwl_log.json missing

[ifishlin]

@inutano I finally run the service up, Thanks you.

However, after my pipeline ended, I got an empty result like this.

(base) yuyulin@benchmark-cwl:~$ .cwlmetrics/bin/cwl-metrics fetch tsv
container_id	stepname	hostname	instance_type	cpu_total_percent	memory_max_usage	memory_cache	blkio_total_bytes	container_image	container_elapsed_sec	container_exit_code	tool_status	total_inputfile_size	workflow_id	workflow_name	workflow_elapsed_sec

I spent some time to debug, then I figured out the cwl-log-generator didn't create the cwl_log.json as expected. (The docker.info, docker.ps and cwltool.log all existed) Also, there was an error log file, cwl_log_generator.error.log , in result directory.

The content of the cwl_log_generator.error.log :

#
/cwl-log-generator/lib/cwllog/cwl/debuginfo.rb:65:in `get_inputs'
/cwl-log-generator/lib/cwllog/cwl/debuginfo.rb:45:in `generate'
/cwl-log-generator/lib/cwllog/cwl.rb:8:in `generate'
/cwl-log-generator/lib/cwllog.rb:21:in `logs'
/cwl-log-generator/lib/cwllog.rb:15:in `parse_logs'
/cwl-log-generator/lib/cwllog.rb:27:in `cwl_log'
/cwl-log-generator/lib/cwllog.rb:11:in `generate'
/cwl-log-generator/generate_cwl_log:41:in `
'

Can you tell me how to fix it? Thanks

cwltools version

3.0.20201113183607

docker images

quay.io/inutano/cwl-metrics-client   0.1.12.6            b5ab27b508a0        23 months ago       912MB
quay.io/inutano/cwl-log-generator    0.1.22              879ac49d1a26        23 months ago       1.27GB
quay.io/inutano/run-dmc              0.1.1               ee64e5b7700c        2 years ago         5.43MB
quay.io/inutano/fluentd              0.1.0               114366efa81f        2 years ago         42.8MB
sebp/elk                             622                 8b1ad728adaa        2 years ago         1.15GB
telegraf                             1.5.0-alpine        60c85e959f3d        2 years ago         40.7MB

[inutano]

Hi, thank you for reporting! The error indicates that the parser failed to parse the stderr from cwltool. I suppose this is because of the cwltool version difference since the cwl-metrics implementation has not been tested with the later version. I'm investigating the changes in the behavior of the runner and I will fix it soon. Thanks again and sorry for the inconvenience!

[ifishlin]

@inutano thanks, I am glad to help. Please let me know when the issue is fixed.

[inutano]

@ifishlin Hi, I've just released ver. 2 of the CWL-metrics, which deprecated the Perl daemon and provide cwl-metrics run command to run the workflow and collect its metrics. It also fixed the bug of cwl_log.json generation. Could you try the latest version and see if the problem is fixed?

The new version has the new interface. Here's a brief instruction:

// move to the directory of your choise
$ cd ~/work
// clone this repo
$ git clone https://github.com/inutano/cwl-metrics
// move to the cloned repo
$ cd cwl-metrics
// launch the system
$ ./cwl-metrics up
// run CWL workflow via cwl-metrics run command
$ ./cwl-metrics run <workflow.cwl> <job.yaml>
// fetch tsv data of metrics
$ ./cwl-metrics fetch tsv
// fetch json data of metrics
$ ./cwl-metrics fetch json
// Stop the system
$ ./cwl-metrics down

[ifishlin]

@inutano thanks, I will try it asap.