fastbook icon indicating copy to clipboard operation
fastbook copied to clipboard

Shared memory error on DataLoader in first code cell of intro

Open mico-boje opened this issue 4 years ago • 1 comments

The following code: dls = ImageDataLoaders.from_name_func( path, get_image_files(path), valid_pct=0.2, seed=42, label_func=is_cat, item_tfms=Resize(224) Will sometimes cause a shared memory error, when the server does not have enough memory. This error can be rather offputting to new users (and this is the very first code cell in the book). I suggest adding num_workers=0 as an argument to amend this.

mico-boje avatar Mar 02 '21 20:03 mico-boje

What is the exact error you're seeing? I have seen an out-of-memory error before, which I ended up solving by reducing the size of the batch (bs parameter in the data loader).

If we're talking about the same error, this seems better than reducing concurrency completely with num_workers=0

zxul767 avatar Sep 29 '21 18:09 zxul767