PaddleSeg icon indicating copy to clipboard operation
PaddleSeg copied to clipboard

evaluate时显存不够用,怎么修改

Open wwma opened this issue 1 year ago • 9 comments

问题确认 Search before asking

  • [X] 我已经搜索过问题,但是没有找到解答。I have searched the question and found no related answer.

请提出你的问题 Please ask your question

使用自定义的数据集进行模型训练,配置文件信息如下:

batch_size: 1
iters: 20000

train_dataset:
  type: Dataset
  dataset_root: dataset
  train_path: dataset\train.txt
  num_classes: 5
  mode: train
  transforms:
    # 训练时的数据未增强
    - type: ResizeStepScaling #将原始图像和标注图像随机缩放为0.5~2.0倍
      min_scale_factor: 0.5
      max_scale_factor: 2.0
      scale_step_size: 0.25
    - type: RandomPaddingCrop
      crop_size: [2048, 1024]
    - type: RandomHorizontalFlip
    - type: RandomDistort
      brightness_range: 0.4
      contrast_range: 0.4
      saturation_range: 0.4
    - type: Normalize



val_dataset:
  type: Dataset
  dataset_root: dataset
  val_path: dataset\val.txt
  num_classes: 5
  mode: val
  transforms:
    - type: Normalize

在配置文件中,已经将batch_size设置为1,训练的时候可以正常运行。 但是在evaluate的时候,会提示错误信息

Out of memory error on GPU 0. Cannot allocate 38.106859GB memory on GPU 0, 8.777832GB memory has been allocated and available memory is 
only 1.221680GB.

Please check whether there is any other process using GPU 0.
1. If yes, please stop them, or start PaddlePaddle on another GPU.
2. If no, please decrease the batch size of your model.
 (at ..\paddle\fluid\memory\allocation\cuda_allocator.cc:86)

请问是因为val数据集中图像尺寸过大吗? train数据集和val 数据集需要统一成相同尺寸

wwma avatar Aug 14 '24 09:08 wwma