romerogroup
IndexError in pyprocar.scripts.dosplot when parsing QE DOS directory (atoms array empty)
When calling pyprocar.bandsdosplot for Quantum ESPRESSO outputs, the DOS parsing fails with an IndexError: tuple index out of range while constructing the Structure object, apparently because the atoms array is empty.
import matplotlib.pyplot as plt
import pyprocar
bands_settings = {'dirname':'Bands' , 'mode':'plain' , 'elimit':[-3.,3.]}
dos_settings = {'dirname':'Dos' , 'mode':'plain' , 'doslimit':[-3.,3.] }
fig , ax = plt.subplots(figsize = (10,5))
pyprocar.bandsdosplot(code = 'qe' ,
bands_settings = bands_settings ,
dos_settings = dos_settings ,
quiet_welcome=True,
show=False,
ax = ax)
Gives the following error:
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
Cell In[17], line 5
2 dos_settings = {'dirname':'Dos' , 'mode':'plain' , 'doslimit':[-3.,3.] }
4 fig , ax = plt.subplots(figsize = (10,5))
----> 5 pyprocar.bandsdosplot(code = 'qe' ,
6 bands_settings = bands_settings ,
7 dos_settings = dos_settings ,
8 quiet_welcome=True,
9 show=False,
10 ax = ax)
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/scripts/scriptBandsDosplot.py:122, in bandsdosplot(bands_settings, dos_settings, dos_limit, elimit, k_limit, grid, code, lobster, savefig, title, title_fontsize, discontinuities, draw_fermi, plot_color_bar, repair, show, dpi, figsize, **kwargs)
120 fig, axes = plt.subplots(1, 2, figsize=figsize, clear=True, dpi=dpi, sharey=True)
121 ebs_plot_fig, ebs_plot_ax = bandsplot(ax=axes[0], **bands_settings)
--> 122 edos_plot_fig, edos_plot_ax = dosplot(ax=axes[1], **dos_settings)
124 # # combines bandsplot and dos plot
125 # ax_ebs, ax_dos = combine_axes(
126 # ebs_plot_fig, edos_plot_fig, fig, plot_color_bar=plot_color_bar
127 # )
128 ax_ebs = axes[0]
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/scripts/scriptDosplot.py:322, in dosplot(code, dirname, mode, orientation, spins, atoms, orbitals, items, normalize_dos_mode, fermi, fermi_shift, elimit, dos_limit, savefig, labels, projection_mask, ax, show, print_plot_opts, export_data_file, export_append_mode, use_cache, verbose, **kwargs)
319 if not os.path.exists(dos_pkl_filepath):
320 logger.info(f"Parsing DOS from {dirname}")
--> 322 parser = io.Parser(code=code, dirpath=dirname)
323 dos = parser.dos
324 structure = parser.structure
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/io/parser.py:30, in Parser.__init__(self, code, dirpath, verbose)
27 self.dos = None
28 self.structure = None
---> 30 self._parse()
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/io/parser.py:47, in Parser._parse(self)
44 self._parse_bxsf()
46 elif self.code == "qe":
---> 47 self._parse_qe()
49 elif self.code == "siesta":
50 self._parse_siesta()
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/io/parser.py:191, in Parser._parse_qe(self)
189 self.ebs = parser.ebs
190 self.kpath = parser.kpath
--> 191 self.structure = parser.structure
192 self.dos = parser.dos
193 return None
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/functools.py:998, in cached_property.__get__(self, instance, owner)
996 val = cache.get(self.attrname, _NOT_FOUND)
997 if val is _NOT_FOUND:
--> 998 val = self.func(instance)
999 try:
1000 cache[self.attrname] = val
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/io/qe/core.py:672, in QEParser.structure(self)
670 @cached_property
671 def structure(self) -> Optional[Structure]:
--> 672 return Structure(
673 atoms=self.species,
674 lattice=self.direct_lattice,
675 fractional_coordinates=self.atomic_positions,
676 rotations=self.rotations,
677 )
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/core/structure.py:66, in Structure.__init__(self, atoms, cartesian_coordinates, fractional_coordinates, lattice, rotations)
63 self.atoms = np.array(atoms)
64 self.lattice = np.array(lattice)
---> 66 if self.atoms.shape[0] == 0:
67 raise ValueError("atoms must be a non-empty list")
68 if self.fractional_coordinates.shape[0] == 0:
IndexError: tuple index out of range
Thanks for reporting this issue and providing a reproducible example!
I looked into it, and the root cause was that in some Quantum ESPRESSO workflows, the Dos/ directory does not contain a pwscf.xml file, but instead provides a data-file-schema.xml. Previously, the parser was only looking for pwscf.xml, so when it didn’t find it, the atomic structure (atoms array) could not be built, which led to the IndexError: tuple index out of range you encountered.
I’ve now fixed this by adding a fallback:
- If
pwscf.xmlis not present, the parser will automatically usedata-file-schema.xml. - This ensures the structure is loaded correctly in both cases.
After fixing that, I also noticed a second issue: sometimes the pdos_atm files contain an inconsistent number of energy points between different atoms/orbitals. This caused misalignment and errors when constructing the DOS object. I resolved this by regularizing the DOS onto a consistent energy grid via interpolation.
Here’s an updated plot after applying the fixes (showing that bands and DOS now parse and plot correctly):
📈 [Attach your plot image here]
So in short:
- ✅ Added fallback to parse
data-file-schema.xmlifpwscf.xmlis missing - ✅ Fixed inconsistent DOS grids in
pdos_atmfiles by interpolation
Both issues should now be resolved in the latest commit. Please try again with the updated version and let me know if you run into any further problems!
Install these changes with
pip uninstall pyprocar
pip install --pre pyprocar
Hey, thanks for your attention to this issue. After installing the new version, I got this new error:
TypeError Traceback (most recent call last)
Cell In[4], line 5
2 dos_settings = {'dirname':'Dos' , 'mode':'plain' , 'doslimit':[-3.,3.] }
4 fig , ax = plt.subplots(figsize = (10,5))
----> 5 pyprocar.bandsdosplot(code = 'qe' ,
6 bands_settings = bands_settings ,
7 dos_settings = dos_settings ,
8 quiet_welcome=True,
9 show=False,
10 ax = ax)
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/scripts/scriptBandsDosplot.py:121, in bandsdosplot(bands_settings, dos_settings, dos_limit, elimit, k_limit, grid, code, lobster, savefig, title, title_fontsize, discontinuities, draw_fermi, plot_color_bar, repair, show, dpi, figsize, **kwargs)
119 # Make the two axes share the y-axis (energy axis)
120 fig, axes = plt.subplots(1, 2, figsize=figsize, clear=True, dpi=dpi, sharey=True)
--> 121 ebs_plot_fig, ebs_plot_ax = bandsplot(ax=axes[0], **bands_settings)
122 edos_plot_fig, edos_plot_ax = dosplot(ax=axes[1], **dos_settings)
124 # # combines bandsplot and dos plot
125 # ax_ebs, ax_dos = combine_axes(
126 # ebs_plot_fig, edos_plot_fig, fig, plot_color_bar=plot_color_bar
127 # )
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/scripts/scriptBandsplot.py:155, in bandsplot(code, dirname, mode, spins, atoms, orbitals, items, fermi, fermi_shift, interpolation_factor, interpolation_type, projection_mask, kticks, knames, kdirect, elimit, ax, show, savefig, print_plot_opts, export_data_file, export_append_mode, ktick_limit, x_limit, use_cache, quiet_welcome, **kwargs)
152 if not os.path.exists(ebs_pkl_filepath):
153 logger.info(f"Parsing EBS from {dirname}")
--> 155 parser = io.Parser(code=code, dirpath=dirname)
156 ebs = parser.ebs
157 structure = parser.structure
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/io/parser.py:30, in Parser.__init__(self, code, dirpath, verbose)
27 self.dos = None
28 self.structure = None
---> 30 self._parse()
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/io/parser.py:47, in Parser._parse(self)
44 self._parse_bxsf()
46 elif self.code == "qe":
---> 47 self._parse_qe()
49 elif self.code == "siesta":
50 self._parse_siesta()
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/io/parser.py:187, in Parser._parse_qe(self)
178 def _parse_qe(self):
179 """parses qe files
180
181 Returns
(...) 184 None
185 """
--> 187 parser = qe.QEParser(dirpath=self.dirpath)
189 self.ebs = parser.ebs
190 self.kpath = parser.kpath
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/io/qe/core.py:59, in QEParser.__init__(self, dirpath)
43 self._dirpath: Path = Path(dirpath)
44 self._detected: Dict[str, Union[Path, List[Path], None]] = {
45 "scf_in": None,
46 "scf_out": None,
(...) 57 "pw_xml": None,
58 }
---> 59 self.detect_files()
File ~/miniconda3/envs/pyprocar_env/lib/python3.12/site-packages/pyprocar/io/qe/core.py:75, in QEParser.detect_files(self)
65 return
67 # files: List[Path] = []
68 # for root, dirs, filenames in os.walk(self._dirpath, followlinks=True):
69 # for name in filenames:
(...) 72 # except Exception:
73 # pass
---> 75 files = [p for p in self._dirpath.rglob("*", recurse_symlinks=True) if p.is_file()]
78 # XMLs
79 atomic_proj_xml = [p for p in files if re.search(r"(?i)^atomic_proj\.xml$", p.name)]
TypeError: Path.rglob() got an unexpected keyword argument 'recurse_symlinks'
Thanks for following up!
This error is actually related to the same root cause discussed in the other issue. The problem comes from the fact that I originally used
Path.rglob("*", recurse_symlinks=True)
to detect QE files, but the recurse_symlinks argument is only available in Python 3.13+. That’s why you’re seeing the TypeError on Python 3.12.
I’ve now rewritten the file detection logic so it works consistently across all supported Python versions (no need for Python 3.13 anymore). So if you update to the latest dev release, this rglob(..., recurse_symlinks) error should also be resolved.
Hey!, The obtained density of states is noisy:
I had previously calculated the DOS using Elk and plotted it with xmgrace, and the result looks nothing like the one obtained here.
Thank you sincerely for following up on this process.