pydm icon indicating copy to clipboard operation
pydm copied to clipboard

WIP/Discussion: Alternate macro loading for UI files

Open ZLLentz opened this issue 2 years ago • 1 comments

This is something I tried to do during the codeathon but it didn't quite work out. I thought it was interesting enough to put up a draft PR as a food for thought/discussion starting point.

This started from a look at the performance of the largest photon-side pydm application, https://github.com/pcdshub/pmps-ui. This application takes ~40s to start, so it seemed like a good case study for performance monitoring.

Upon profiling, it's clear that about half of the load time (~16s) is taken up by creating widgets from ui files. So, that leads us into an investigation: how does pydm handle the loading/templating of a ui file? Well, here are the steps:

  1. Read the file from disc
  2. Substitute macros into the file text
  3. Invoke the pyqt uic loader to create a widget class
  4. Graft the widgets defined by the widget class onto an empty Display object

Now, this repeats every time we re-use the same template. So even though we're using the same designer layout for the widget, we're repeating a lot of work- particularly the file reads and the uic loader.

Is there a way we can re-use the designer layout once? Yes, and that's what this PR does, bringing the ui-file-based load times down to ~12s for a 25% speedup. Unfortunately, this created some bad behavior with the other functionality of PyDM, namely, duplicating all of our PVs with identical template versions that get cleaned up and remade repeatedly as we instantiate all the template widgets, ultimately resulting in a net slowdown.

Here are the steps for loading/templating a ui file laid out in this PR:

  1. Load widget class from ui file cache if it exists, otherwise read the file
  2. Invoke the pyqt uic loader to create a widget class if it was not in the cache, otherwise this can be skipped
  3. Inspect the class for template strings in properties and cache these if needed, otherwise skip
  4. Graft the widget class onto an empty Display object
  5. If macros were provided, replace all the template string properties of the display as appropriate

Note how for each file we do steps 1-3 exactly once, and then for subsequent re-use we only do steps 4-5.

The unfortunate slowdown comes from the double-setting of channel properties, first with the template value, and second with the correct value. Maybe with a bit more thought and work this can be a useful performance boost for large displays with template repeaters.

ZLLentz avatar May 13 '22 17:05 ZLLentz

I finally have some time here and there to start looking into performance improvements. I've grabbed your profiler from pcdsutils have it up and running, and am going to start here. Will go through your suggestions/notes in the other performance PR again as well.

jbellister-slac avatar Dec 07 '22 00:12 jbellister-slac

closing in favor of #965

ZLLentz avatar Jun 27 '24 00:06 ZLLentz