Optimization efficacy and efficiency

[rybern]

No one has yet worked on:

• [ ] Tuning optimizations to improve performance for real Stan programs, including:
  • [ ] Optimization behavior (e.g., should lazy code motion extract constants?)
  • [ ] Optimization repetitions
  • [ ] Optimization ordering (the current order is my educated guess but is not empirical)
• [ ] Making sure that optimizations are applied whenever safe
• [ ] 'Optimizing' the optimization code (e.g. some large data structures are recomputed for each optimization)

I want to make it clear that #522 was about solving instances when optimizations produced non-compiling or non-correct code. I solved all issues that arose with our corpus and with all of the optimizations activated. I did not do any performance tuning. In addition, some of the changes I made which decreased the efficacy of the optimizations may have been unnecessarily conservative.

I'd be happy to help out anyone who wanted to work on these, and I'll work on them myself when I have time.

[SteveBronder]

Optimization ordering (the current order is my educated guess but is not empirical)

We should probably investigate the order and repetitions of how other compilers do these. Like I'm pretty sure gcc etc. runs multiple DCE passes after things like lazy code motion

[rybern]

@SteveBronder That sounds great. I glanced at GCC but it wasn't a big influence in my guesses. We'd probably want to get some benchmarking set up first.