Extremely slow compilation.

[magnetophon]

I've seem to ran into a different bottleneck.

In other projects, the evaluation times can be quite long.

In the project I'm currently working on, a small change increased the compile time from +/- 7 minutes to over an hour! Most of the time is spent after end Constant propagation:

start parser

end parser (duration : 0.015902)

start evaluation

end evaluation (duration : 95.6962)

start propagation

end propagation (duration : 3.1008)

start generateCode

        start prepare

                start deBruijn2Sym

                end deBruijn2Sym (duration : 0.135595)

                start L1 typeAnnotation

                end L1 typeAnnotation (duration : 0.167431)

                start Cast and Promotion

                end Cast and Promotion (duration : 0.0610712)

                start simplification

                end simplification (duration : 0.516657)

                start Constant propagation

                end Constant propagation (duration : 0.0240948)

                start L5 typeAnnotation

                end L5 typeAnnotation (duration : 0.11369)

        end prepare (duration : 5850.16)

        start compileMultiSignal

                start prepare2

                end prepare2 (duration : 0.000438929)

        end compileMultiSignal (duration : 0.316341)

end generateCode (duration : 5850.95)

As always, a minimal example would be helpful, but how do I create one when the problem only crops up for complex DSPs?

[magnetophon]

My laptop just finished compiling this. It took 48 hours and 20 minutes!!

[grammaton]

Hi, here my compilation times on a Catalina MacBook Pro:

1. faust2caqt -t 0 -time -midi DigiDrie.dsp

start parser
end parser (duration : 0.013186)
start evaluation
end evaluation (duration : 73.3572)
start propagation
end propagation (duration : 2.4686)
start generateCode
	start prepare
		start deBruijn2Sym
		end deBruijn2Sym (duration : 0.093725)
		start L1 typeAnnotation
		end L1 typeAnnotation (duration : 0.16187)
		start Cast and Promotion
		end Cast and Promotion (duration : 0.0406661)
		start simplification
		end simplification (duration : 0.6043)
		start Constant propagation
		end Constant propagation (duration : 0.019213)
		start L5 typeAnnotation
		end L5 typeAnnotation (duration : 0.11191)
	end prepare (duration : 258.811)
	start compileMultiSignal
		start prepare2
		end prepare2 (duration : 0.000506163)
	end compileMultiSignal (duration : 0.230723)
end generateCode (duration : 259.355)

2. faust2jack -t 0 -time -midi DigiDrie.dsp

start parser
end parser (duration : 0.0149689)
start evaluation
end evaluation (duration : 68.5592)
start propagation
end propagation (duration : 2.50089)
start generateCode
	start prepare
		start deBruijn2Sym
		end deBruijn2Sym (duration : 0.108488)
		start L1 typeAnnotation
		end L1 typeAnnotation (duration : 0.167289)
		start Cast and Promotion
		end Cast and Promotion (duration : 0.0437441)
		start simplification
		end simplification (duration : 0.503554)
		start Constant propagation
		end Constant propagation (duration : 0.0160341)
		start L5 typeAnnotation
		end L5 typeAnnotation (duration : 0.140927)
	end prepare (duration : 320.971)
	start compileMultiSignal
		start prepare2
		end prepare2 (duration : 0.000260115)
	end compileMultiSignal (duration : 0.239967)
end generateCode (duration : 321.46)

[sletz]

• I've just added some more timing points to better see what happens here: https://github.com/grame-cncm/faust/commit/ae581bdc9a908a322dc5a484d6d7075747f75756
• then I see that the conditionAnnotation step seems to be the bottleneck. Since this step is only used in the experimental enable/control primitive (still to be improved !?), you can deactivate is with -es 0
• then faust -t 0 -time DigiDrie.dsp -es 0 now take 1m4.424s here on a MacBook pro

[magnetophon]

Thanks for looking in to this!

@grammaton That's very quick. Did you compile the LFO branch that I linked to? https://github.com/magnetophon/DigiDrie/tree/LFO

@sletz That is fantastic, thank you! I was using control, but this way I have a quick compile again, at least for development.

Do you think the compile without -es 0 might be fast one day?

The quick compile made me hopeful for vectorisation, so I tried faust2jack -t 0 -time -midi -vec -lv 0 -g -vs 8 DigiDrie.dsp -es 0 . That is so far stuck at end prepare (duration : 0.680866). I'll let you know if/when it finishes..

[sletz]

Thanks for looking in to this!

@grammaton That's very quick. Did you compile the LFO branch that I linked to? https://github.com/magnetophon/DigiDrie/tree/LFO

@sletz That is fantastic, thank you! I was using control, but this way I have a quick compile again, at least for development.

Do you think the compile without -es 0 might be fast one day?

Well Yann is working on the new ondemand primitive that should allow to implement control/enable in a cleaner way. We'll see what happens regarding compilation time then.

The quick compile made me hopeful for vectorisation, so I tried faust2jack -t 0 -time -midi -vec -lv 0 -g -vs 8 DigiDrie.dsp -es 0 . That is so far stuck at end prepare (duration : 0.680866). I'll let you know if/when it finishes..

[magnetophon]

Thanks. Sounds good!

For what it's worth: the -vec compile is still stuck at end prepare, after 2 hours.