llvm
Using Docker builds for iterative / interactive development of Torch-MLIR
We recently added docker support to build CI images and Release images on your local system. It would be great to understand what the community would like to see in using the same for docker build process for your incremental development.
Quoting @silvasean : "incremental development" means a workflow suitable for developers to use as part of a tight edit/compile/test cycle. This may involve a dirty work tree or other weird in-flight situations, like a local hack to a generated file. This also involves running filtered tests etc. The set of tests that make sense to run for CI do not make sense to run for "incremental development".
I can see adding a way to run a smaller set of tests is currently lacking, but are there other such features you would like that you use in your regular edit/compile/test cycle. I'm sure we wont be able to cover every edit/compile/test cycle or every IDE integration but we can see how we can generally add more support for dockerized development.
Right now my default go to is:
time TM_PACKAGES="out-of-tree in-tree" ./build_tools/python_deploy/build_linux_packages.sh It builds and tests both out-of-tree and in-tree with ccache in the order of minutes. You can all the tests if you want and just build and it is almost the same time as me calling cmake --build build.
I can also think of pass through to drop into a command line in the docker and ability to execute any command.
For me, a critical first step would be to find a way for vscode to be able to connect into the source tree inside the docker image, use the compile_commands.json, etc.
This sounds like a lot of work though, and I'm not sure that it would be worth it. I think there's already a tremendous value-add just from being able to reliably replicate the CI after they are happy with their change. I'm not sure we want to rush into a full "docker-first development" process where everybody develops exclusively inside docker images.
yup agreed. just wanted a place to capture everyone's thoughts how it would look for whenever we get to it.
Ah I hadn't thought of compile_commands.json for vscode as I don't use it.
@NatashaKnk and I were attempting to use the docker build for iterative / interactive development, and the main blocker we ran into is that the Python files in the python_packages dir were symlinked to paths that only existed in the Docker image, so when using the package from outside, it didn't work. Perhaps we should have a way to log into the docker image and do work from there, but then there are questions related to IDE's etc.