envd icon indicating copy to clipboard operation
envd copied to clipboard

feat(CLI): Generate Dockerfile from build.envd

Open gaocegege opened this issue 3 years ago • 9 comments

Description

As an ML infra engineer, I may want to get the Dockerfile from the envd, instead of building the image, to integrate envd into the internal existing infrastructures.

$ envd build --output type=dockerfile
envd.Dockerfile is generated

Message from the maintainers:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

gaocegege avatar Jun 28 '22 13:06 gaocegege

I am working on this. and the cli could better:

envd build --dryrun=dockerfile

aseaday avatar Jun 28 '22 16:06 aseaday

@aseaday Thanks for your contribution!

Would you mind me assigning this issue to you?

BTW, how do you plan to implement it? I think we need to make ir.Graph an interface, then implement the compile func for dryrun and buildkit llb.

gaocegege avatar Jun 29 '22 01:06 gaocegege

Assign it to me. I would give a plan first and any suggestions?

aseaday avatar Jun 29 '22 17:06 aseaday

I think we can make ir.Graph an interface instead of the struct. Then dryrun and buildkit llb generation are two different implementations for the interface.

gaocegege avatar Jun 30 '22 01:06 gaocegege

Hi, is there some progress?

gaocegege avatar Jul 13 '22 06:07 gaocegege

Hi, is there some progress?

I am still working on part rescontruct from those lines:

	root = llb.User("envd")(root)
	// Refer to https://github.com/moby/buildkit/blob/31054718bf775bf32d1376fe1f3611985f837584/frontend/dockerfile/dockerfile2llb/convert_runmount.go#L46
	cache := root.File(llb.Mkdir("/cache",
		0755, llb.WithParents(true), llb.WithUIDGID(g.uid, g.gid)),
		llb.WithCustomName("[internal] settings conda cache mount permissions"))
	run := root.
		Run(llb.Shlex(cmd), llb.WithCustomNamef("conda install %s",
			strings.Join(g.CondaPackages, " ")))
	run.AddMount(cacheDir, cache,
		llb.AsPersistentCacheDir(g.CacheID(cacheDir), llb.CacheMountShared), llb.SourcePath("/cache"))

I think I could finish it by the end of this month.

aseaday avatar Jul 13 '22 06:07 aseaday

I would decide the final solutions method this weekend:

  • Continue to convert the llb.state to Dockerfile
  • Constrcut the a structure as the interfaces of llb.state but in fact it is a dockerfile builder

aseaday avatar Jul 13 '22 06:07 aseaday