clearml icon indicating copy to clipboard operation
clearml copied to clipboard

Bug? default docker image not parsed correctly

Open MoPl90 opened this issue 3 years ago • 5 comments
trafficstars

Setting agent.docker_default.image has no effect on the 'Execution' view of the experiment, while the agent.docker_default.arguments are correctly parsed. Experiment queueing fails unless an image is specified.

MoPl90 avatar Jul 19 '22 11:07 MoPl90

Hi @MoPl90 ,

Where do you specify the image? in the UI?

jkhenning avatar Jul 19 '22 12:07 jkhenning

Hi, I specified both fields mentioned in the clearml.config file, but only the arguments are parsed when I clone and queue an experiment. The arguments then also show in the UI, but the image field stays empty.

MoPl90 avatar Jul 19 '22 12:07 MoPl90

Are you sure you used the correct names? See here, should be agent.default_docker.image and agent.default_docker.arguments

jkhenning avatar Jul 19 '22 13:07 jkhenning

Excerpt from my config file:

 default_docker: {
         # default docker image to use when running in docker mode
         image: "my_image:1.0"
         # optional arguments to pass to docker image
         arguments: ["--network=host", "--ipc=host", "--gpus all"]
     }

MoPl90 avatar Jul 19 '22 13:07 MoPl90

Hello again,

since I started using the hyper parameter optimisation functionality, this is getting more critical. The docker image name is also not copied to the children tasks of the optimisation task.

Thanks!

MoPl90 avatar Jul 21 '22 13:07 MoPl90

Just wanted to mention a similar problem. If I clone the task in the UI and its IMAGE and ARGUMENTS fields are empty on the execution page, docker arguments from the the clearml.conf file on the agent machine won't be used when I execute this task, however image name will be parsed correctly and will appear on the execution page in the IMAGE field. At the same time if I start new experiment not from the UI but from the code using Task.init() API both IMAGE and ARGUMENTS fields will be correctly parsed from the clearml.conf file and will appear in the UI.

romanvoeikov avatar Dec 06 '22 21:12 romanvoeikov

@romanvoeikov What clearml and clearml-agent versions are you using?

erezalg avatar Dec 07 '22 14:12 erezalg

Oh, sorry. I was going to mention it in the original message: clearml-agent version is 1.5.0, clearml version is 1.4.1. JFYI: just checked with clearml==1.8.3, still the same

romanvoeikov avatar Dec 07 '22 14:12 romanvoeikov