[BUG] Unresponsive GUI with large files

Open oerc0122 opened this issue 10 months ago • 0 comments

Description of the error When selecting an analysis type with a "large" file GUI (note I am seeing this with a 3MiB file) will hang as it tries to read the whole file as part of the configurator. The "output prediction" for certain analyses also seems to take time.

Better profiling of configurator load times may be necessary to see where the issue actually lies.

Logs with timings

2025-04-23 11:58:34,447 - INFO - process[3009645] - ActionsTree 56 - tree: clicked on VelocityAutoCorrelationFunction
2025-04-23 11:58:34,453 - INFO - process[3009645] - Action 222 - Configuration OrderedDict([('trajectory', {}), ('frames', {}), ('interpolation_order', {}), ('projection', {}), ('normalize', {}), ('atom_selection', {}), ('atom_transmutation', {}), ('weights', {}), ('output_files', {}), ('running_mode', {})])
2025-04-23 11:58:47,074 - INFO - process[3009645] - Action 240 - Set up input trajectory
2025-04-23 11:58:47,075 - INFO - process[3009645] - Action 275 - Set up the right widget for frames
2025-04-23 11:58:47,076 - INFO - process[3009645] - Action 275 - Set up the right widget for interpolation_order
2025-04-23 11:58:47,077 - INFO - process[3009645] - Action 275 - Set up the right widget for projection
2025-04-23 11:58:47,077 - INFO - process[3009645] - Action 275 - Set up the right widget for normalize
2025-04-23 11:58:47,928 - INFO - process[3009645] - Action 275 - Set up the right widget for atom_selection
2025-04-23 11:58:48,810 - INFO - process[3009645] - Action 275 - Set up the right widget for atom_transmutation
2025-04-23 11:58:48,828 - INFO - process[3009645] - Action 275 - Set up the right widget for weights
2025-04-23 11:58:48,830 - INFO - process[3009645] - Action 275 - Set up the right widget for output_files
2025-04-23 11:58:48,831 - INFO - process[3009645] - Action 275 - Set up the right widget for running_mode
2025-04-23 11:58:48,832 - INFO - process[3009645] - Action 372 - Show output prediction
2025-04-23 11:59:01,779 - INFO - process[3009645] - Action 376 - Axes = dict_keys(['ps'])

Traceback during hang

Traceback (most recent call last):
  File "/home/jacob/MDANSE/MDANSE_GUI/Src/MDANSE_GUI/Tabs/Visualisers/Action.py", line 234, in update_panel
    input_widget = widget_class(parent=self, **ddict)
  File "/home/jacob/MDANSE/MDANSE_GUI/Src/MDANSE_GUI/InputWidgets/HDFTrajectoryWidget.py", line 33, in __init__
    self._configurator.configure(str(filename))
  File "/home/jacob/MDANSE/MDANSE/Src/MDANSE/Framework/Configurators/HDFTrajectoryConfigurator.py", line 50, in configure
    inputTraj = IInputData.create("HDFTrajectoryInputData", self["value"])
  File "/home/jacob/MDANSE/MDANSE/Src/MDANSE/Core/SubclassFactory.py", line 176, in create
    return specific_class(*args, **kwargs)
  File "/home/jacob/MDANSE/MDANSE/Src/MDANSE/Framework/InputData/InputFileData.py", line 31, in __init__
    self.load()
  File "/home/jacob/MDANSE/MDANSE/Src/MDANSE/Framework/InputData/HDFTrajectoryInputData.py", line 33, in load
    traj = Trajectory(self._name)
  File "/home/jacob/MDANSE/MDANSE/Src/MDANSE/MolecularDynamics/Trajectory.py", line 51, in __init__
    self._trajectory = self.open_trajectory(self._format)
  File "/home/jacob/MDANSE/MDANSE/Src/MDANSE/MolecularDynamics/Trajectory.py", line 66, in open_trajectory
    trajectory = trajectory_class(self._filename)
  File "/home/jacob/MDANSE/MDANSE/Src/MDANSE/Trajectory/MdanseTrajectory.py", line 56, in __init__
    self._chemical_system.load(self._h5_file)
  File "/home/jacob/MDANSE/MDANSE/Src/MDANSE/Chemistry/ChemicalSystem.py", line 305, in load
    self.initialise_atoms(atom_types, atom_names)
  File "/home/jacob/MDANSE/MDANSE/Src/MDANSE/Chemistry/ChemicalSystem.py", line 60, in initialise_atoms
    self._atom_indices = [
  File "/home/jacob/MDANSE/MDANSE/Src/MDANSE/Chemistry/ChemicalSystem.py", line 61, in <listcomp>
    self.add_atom(self._database.get_atom_property(symbol, "atomic_number"))
  File "/home/jacob/MDANSE/MDANSE/Src/MDANSE/Trajectory/MdanseTrajectory.py", line 462, in get_atom_property
    value = self._h5_file[f"/atom_database/{symbol}"][index]
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "/home/jacob/.pyenv/versions/3.10.14/lib/python3.10/site-packages/h5py/_hl/group.py", line 366, in __getitem__
    return dataset.Dataset(oid, readonly=(self.file.mode == 'r'))
  File "/home/jacob/.pyenv/versions/3.10.14/lib/python3.10/site-packages/h5py/_hl/base.py", line 285, in file
    from . import files
  File "<frozen importlib._bootstrap>", line 1071, in _handle_fromlist

Describe the expected result Smooth, responsive GUI which does not need to reload the file every time an action is selected.

Describe the actual result 15s pause when selecting an action.

Suggested fix Given the trajectory is actually selected in the previous tab (Trajectories), it should be loaded once there and simply passed into the appropriate tabs as necessary.

It may also be possible to put these into a background thread.

Additional details Convert MDANSE/Tests/UnitTests/Data/apoferritin.pdb & MDANSE/Tests/UnitTests/Data/apoferritin.dcd with DCD converter. Load and select and action.

Apr 23 '25 10:04 oerc0122