pega-datascientist-tools icon indicating copy to clipboard operation
pega-datascientist-tools copied to clipboard

_delete_temp_files in generateReport is _way_ too aggressive.

Open StijnKas opened this issue 1 year ago • 0 comments

pdstools version checks

  • [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest version of pdstools.

Issue description

The _delete_temp_files method in generateReport of the ADMDatamart class is way too aggressive.

https://github.com/pegasystems/pega-datascientist-tools/blob/master/python/pdstools/adm/ADMDatamart.py#L1657

By default, it currently deletes all [".yaml", ".qmd", ".ipynb", ".arrow"] files in the working directory.

Considering that, by default, the working directory is the current directory, this will remove all files with these types from the local directory, and since it's using OS commands they're not put in the thrash either. It already accidentally removed a few notebooks and other cached files that i had there locally.

Reproducible example

Have files in your current directory, then run generateReport

Expected behavior

We should make sure it only deletes the cached files that we write for the health check, and only the files used to generate this specific health check run.

Installed versions

Replace this line with the output of pdstools.show_versions(), leave the backticks in place

StijnKas avatar May 15 '24 18:05 StijnKas