Odin
Odin copied to clipboard
odin-lang
Slow Compile Times
Context
- Operating System & Odin Version:
odin version dev-2022-03:02475de7a - Please paste
odin reportoutput:
Odin: dev-2022-03:02475de7a
OS: Windows 10 Home Basic (version: 21H1), build 19043.1766
CPU: AMD Ryzen 9 5900HX with Radeon Graphics
RAM: 15774 MiB
Expected Behavior
Compile times would be short enough in length that one doesn't feel the need to start doing something else while waiting for them.
Current Behavior
On mid-sized projects compile times (on unoptimized debug builds) take a long time. A single line change in a small package shouldn't result in having to wait 10 seconds for the project to build.
Notes
Let me know if this is a duplicate of an existing issue -- I tried searching a few keywords and couldn't find anything
If you use -show-more-timings, might the case be that it spends an inordinate amount of time in LLVM?
You can use the experimental feature of -use-separate-modules to make builds faster.
Not sure if it currently works with -debug though. Might be worth an issue if there's not one already.
If you use
-show-more-timings, might the case be that it spends an inordinate amount of time in LLVM?
83% of the time is spent in LLVM; regardless this is still an issue effecting the language, we can't just blame LLVM and call it a day. If we need to use LLVM I would assume there are still things we could do (eg. incremental builds, only recompiling changed packages) to improve things.
You can use the experimental feature of
-use-separate-modulesto make builds faster. Not sure if it currently works with-debugthough. Might be worth an issue if there's not one already.
I was using this at some point but the compiler had some issues with it at the time which made it unusable; these issues seem to have been fixed now. Unfortunately, even when using this option, the compile times (despite taking 60% of the time as without) are still unacceptably long.
If you use
-show-more-timings, might the case be that it spends an inordinate amount of time in LLVM?83% of the time is spent in LLVM; regardless this is still an issue effecting the language, we can't just blame LLVM and call it a day. If we need to use LLVM I would assume there are still things we could do (eg. incremental builds, only recompiling changed packages) to improve things.
I don't disagree. To address the issue we first need to know where the majority of time is being spent, and it looks like my educated guess of LLVM was correct. It wasn't to dismiss the issue with "oh, it's LLVM", but a fact finding operation.
@Kelimion My bad if I read the tone wrong!
The simplest course of action feels like it would be caching the .obj files created when using -use-separate-modules such that only changed packages (and those package's dependants) are recompiled. It feels as if this would improve build times in the vast majority of cases.
Without knowing anything about how odin's compiler works, what are the reasons this wouldn't be as simple as hashing all the source files for a package and its dependencies and reusing the previous build's .obj file for that package if the hash is unchanged from the previous build?
@Kelimion My bad if I read the tone wrong! No problem.
Without knowing anything about how odin's compiler works, what are the reasons this wouldn't be as simple as hashing all the source files for a package and its dependencies and reusing the previous build's .obj file for that package if the hash is unchanged from the previous build?
It would also cause recompiles for whitespace changes or renaming a variable (which would affect the RTTI but not the object code), but I think that's probably an acceptable trade-off. At the very least for an initial stab at the issue.
The salient question to ask would be where we'd store these cached object files. Maybe a .object-cache directory in the project directory, next to the executable, with each of the objects named after the (tree)hash of the source files resulting in that object file? And then an odin clean . command to clean up the cache?
The salient question to ask would be where we'd store these cached object files. Maybe a .object-cache directory in the project directory, next to the executable, with each of the objects named after the (tree)hash of the source files resulting in that object file? And then an odin clean . command to clean up the cache?
Maybe .odin_object_cache so we know it's odin-specific. Regarding the caching I think an additional file in the object cache directory would make more sense than storing the object files under their hash. If we store a file that maps package name => object file, hash we know which object file to delete for a package when we compile a new one. If we just use the hash in the filename we're going to end up with an object file for every code change (which would add up quickly) and wouldn't know which ones we can delete.
Storing the files as package_name__hash.obj could also work, and just delete any .obj file starting with package_name__ when we compile a new .obj file for a given package.
Precompilation is a little more complicated than just the -use-separate-modules. Effectively it means you need to ignore most of the minimum dependency system as each package would need to precompile everything within, even stuff that isn't used. Most (not all) parapoly stuff would probably need to be rebuilt each time but that isn't much of an issue. Another is anything to do with RTTI since that requires everything to be in a specific order which means you still effectively require a pseudo-single translation unit for all of those aspects.
Odin's package system does allow for precompilation in theory but the main issue is purely the parapoly and RTTI stuff which will cock things up a little bit.
The actual solution is to just yeet LLVM, but not for the short period of time.
Given what I assume is the massive task of dropping LLVM, I think it would be overall beneficial to explore short-term solutions to at least improve upon the issue in the mean time.
... having to wait 10 seconds for the project to build ...
83% of the time is spent in LLVM; regardless this is still an issue effecting the language, we can't just blame LLVM and call it a day
Isn't 2 seconds still a long time?
mid-sized projects
How big is the project?
If you're to clone this repo (10K LOC or so) https://github.com/rui314/chibicc and run make on it (with mingw for example), how long does that take?
The actual solution is to just yeet LLVM, but not for the short period of time.
This sounds like a good spot to introduce the middle end and start working towards other backend options... :P
