ValueError: Sample larger than population or is negative

[BaseMax]

Hello,

Running python script cause to an error:

reading validation images...
Traceback (most recent call last):
  File "brain-segmentation-pytorch/inference.py", line 184, in <module>
    main(args)
  File "brain-segmentation-pytorch/inference.py", line 22, in main
    loader = data_loader(args)
  File "brain-segmentation-pytorch/inference.py", line 80, in data_loader
    random_sampling=True,
  File "/content/brain-segmentation-pytorch/dataset.py", line 56, in _init_
    validation_patients = random.sample(self.patients, k=validation_cases)
  File "/usr/lib/python3.6/random.py", line 320, in sample
    raise ValueError("Sample larger than population or is negative")
ValueError: Sample larger than population or is negative

History of my commands in a Arch Linux system:

cd ~/OpenSource/
git clone https://github.com/mateuszbuda/brain-segmentation-pytorch
cd brain-segmentation-pytorch/
ls
pip3 install numpy
pip3 install torch
pip3 install matplotlib
python inference.py
pip3 install medpy
python inference.py
pip3 install skimage
pip3 install skit-image
pip3 install scikit-image
pip3 install tqdm
mkdir data
wget https://www.kaggle.com/kmader/mias-mammography && unzip...
python inference.py --images data --weights weights/unet.pt
python3 inference.py --images data --weights weights/unet.pt

[max@base brain-segmentation-pytorch]$ python -v

Python 3.8.5 (default, Sep  5 2020, 10:50:12) 

[max@base brain-segmentation-pytorch]$ python inference.py --images data --weights weights/unet.pt

reading validation images...
Traceback (most recent call last):
  File "inference.py", line 184, in <module>
    main(args)
  File "inference.py", line 22, in main
    loader = data_loader(args)
  File "inference.py", line 76, in data_loader
    dataset = Dataset(
  File "/home/max/OpenSource/brain-segmentation-pytorch/dataset.py", line 56, in __init__
    validation_patients = random.sample(self.patients, k=validation_cases)
  File "/usr/lib/python3.8/random.py", line 363, in sample
    raise ValueError("Sample larger than population or is negative")
ValueError: Sample larger than population or is negative

---

We face to same problem in Google CoLab:

---

• https://stackoverflow.com/questions/20861497/sample-larger-than-population-in-random-sample-python
• https://github.com/mateuszbuda/brain-segmentation-pytorch/blob/master/dataset.py#L56

Can you guide?

I change 56th line of dataset.py, from:

validation_patients = random.sample(self.patients, k=validation_cases)

to:

validation_patients = random.choices(self.patients, k=validation_cases)

Still, there is an error in the program:

[max@base brain-segmentation-pytorch]$ python3 inference.py --images data --weights weights/unet.pt

reading validation images...
Traceback (most recent call last):
  File "inference.py", line 184, in <module>
    main(args)
  File "inference.py", line 22, in main
    loader = data_loader(args)
  File "inference.py", line 76, in data_loader
    dataset = Dataset(
  File "/home/max/OpenSource/brain-segmentation-pytorch/dataset.py", line 57, in __init__
    validation_patients = random.choices(self.patients, k=validation_cases-1)
  File "/usr/lib/python3.8/random.py", line 399, in choices
    return [population[_int(random() * n)] for i in _repeat(None, k)]
  File "/usr/lib/python3.8/random.py", line 399, in <listcomp>
    return [population[_int(random() * n)] for i in _repeat(None, k)]
IndexError: list index out of range

Regards, Max

[mateuszbuda]

It simply means that validation_cases is larger than len(self.patients). patients is a dictionary mapping patient IDs to volumes. Did you run it on your own dataset? You can add this assert after setting self.patients variable: https://github.com/mateuszbuda/brain-segmentation-pytorch/blob/8ef2e2d423b67b53ec8113fc71a9b968bb0f66e7/dataset.py#L51

assert len(patients) >= validation_cases, f'The number of patients ({len(self.patients)}) is lower than the validation sample size ({validation_cases})'

[fidhaph]

I have solved " validation_patients = random.sample(self.patients, k=validation_cases) File "/usr/lib/python3.6/random.py", line 320, in sample raise ValueError("Sample larger than population or is negative") ValueError: Sample larger than population or is negative" issue by adding --images 'root folder with images' argument while running inference.py