c2rust icon indicating copy to clipboard operation
c2rust copied to clipboard

Published 20 hours ago • verified publisher icon immunant

Parallelize transpiling

Open rinon opened this issue 5 years ago • 3 comments

We should be able to easily transpile all translation units in parallel, either in separate processes or threads. Just needs to be implemented at some point.

rinon avatar Nov 04 '19 21:11 rinon

Running c2rust_transpile::transpile in parallel (with a .par_map) on completely separate compilation databases (the directories under tests/) actually causes panics regularly, and once I got a segfault, too. I'm guessing there's something in c2rust-ast-exporter or clang APIs that's stateful and causing lots of issues. This would be very nice to have, even if just for speeding up tests.

kkysen avatar Jun 15 '22 05:06 kkysen

clang does keep internal global state, and I don't think that's going to be fixable. I think this will need to be process parallelization, i.e. what build systems do.

rinon avatar Jun 15 '22 17:06 rinon

Yeah, that seems a lot safer to do. Would it be possible to process-parallelize the clang and c2rust-ast-exporter parts up until serialization, and then from deserialization on, which is all in Rust, do normal thread-parallelization?

kkysen avatar Jun 15 '22 20:06 kkysen