OneTrainer icon indicating copy to clipboard operation
OneTrainer copied to clipboard

[Bug]: Adding a blank/space at the end of a concept directory produces an error message when preparing the dataset for training

Open gilga2024 opened this issue 8 months ago • 7 comments

What happened?

An error message that a concept directory could not be found was shown when preparing the dataset for training / caching.

Steps to reproduce (on a linux machine):

  1. create a directory that contains a blank/space at the end of the directory name -> "abc "
  2. create a concept and refer to the directory just created as the "path"
  3. start the training with the concept
  4. The error message is shown

Assumed problem/error in code: Somewhere in the pipeline that handles concept directory a "trim" method is called that removes the blanks/spaces at the end.

What did you expect would happen?

The training should start without an error message

Relevant log output

Traceback (most recent call last):
  File "/mypath/OneTrainer/modules/ui/TrainUI.py", line 528, in __training_thread_function
    trainer.train()
  File "/mypath/OneTrainer/modules/trainer/GenericTrainer.py", line 502, in train
    self.data_loader.get_data_set().start_next_epoch()
  File "/mypath/OneTrainer/venv/src/mgds/src/mgds/MGDS.py", line 49, in start_next_epoch
    self.loading_pipeline.start_next_epoch()
  File "/mypath/OneTrainer/venv/src/mgds/src/mgds/LoadingPipeline.py", line 75, in start_next_epoch
    module.start(self.__current_epoch)
  File "/mypath/OneTrainer/venv/src/mgds/src/mgds/pipelineModules/CollectPaths.py", line 62, in start
    file_names = sorted(self.__list_files(path, include_subdirectories))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mypath/OneTrainer/venv/src/mgds/src/mgds/pipelineModules/CollectPaths.py", line 45, in __list_files
    dir_list = [os.path.join(path, filename) for filename in os.listdir(path)]
                                                             ^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/mypath/trainingdata/abc'

Output of pip freeze

No response

gilga2024 avatar Jun 24 '24 00:06 gilga2024