cucim
cucim copied to clipboard
[BUG] - Info messages appearing as warnings in Jupyter notebooks
Describe the bug When running cucim - accelerated dataloading in Jupyter, a lot of pink warning style output can be generated that looks like an error.
Steps/Code to reproduce bug Run the following code in a notebook with a suitably large image:
from cucim import CuImage
from dask.distributed import as_completed
from dask.distributed import Client, LocalCluster
import os
import numpy as np
cluster = LocalCluster(dashboard_address= 8789, processes=True)
client = Client(cluster)
# iterate over a set of regions from which to threshold
def process_chunk(params):
start_loc_list = params[0]
inp_file = params[1]
patch_size = params[2]
slide = CuImage(inp_file)
res = []
for start_loc in start_loc_list:
region = np.array(slide.read_region(start_loc, [patch_size, patch_size], 0))
if region.flatten().var() > 100:
res.append(start_loc)
return res
# As the results are processed, put them into a list
def compile_results(futures):
patches = []
for future in as_completed(futures):
res1 = future.result()
if res1:
for patch in res1:
patches.append(patch)
return patches
input_file = "patient_100_node_0.tif"
wsi = CuImage(input_file)
sizes=wsi.metadata["cucim"]["resolutions"]
w = sizes["level_dimensions"][0][0]
h = sizes["level_dimensions"][0][1]
patch_size = 256
num_processes = os.cpu_count()
# compute the coordinates of the image patches
start_loc_data = [(sx, sy)
for sy in range(0, h, patch_size)
for sx in range(0, w, patch_size)]
# calculate the number of patches per process/thread
chunk_size = len(start_loc_data) // num_processes
# create list of patches to process
start_loc_list_iter = [(start_loc_data[i:i+chunk_size],input_file,patch_size) for i in range(0, len(start_loc_data), chunk_size)]
# Threshold each patch asynchronously and return a future
future_result1 = list(client.map(process_chunk, start_loc_list_iter))
patches = compile_results(future_result1)`
Expected behavior Ideally no such output should be produced unless the user specifies that they want info/warning output to be shown.
Environment details (please complete the following information): Jupyter notebook running NGC PyTorch container 21.07 cucim v21.8.2
Additional context Here is some example output when loading an image with 10 processes:
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)