python-bioformats icon indicating copy to clipboard operation
python-bioformats copied to clipboard

How do I turn off debug output? When I read the image, it produces a lot of output

Open CDPDisk opened this issue 1 year ago • 2 comments

Does this information affect my reading speed? I'm comparing the read speed of this library with openslide for the same whole slide image file. I think the text affects the results of the comparison. By the way, may I ask if you have compared the read speed of both? Thank you.

There are large debug information like this: 00:06:21.279 [Thread-0] DEBUG loci.common.NIOByteBufferProvider -- Using mapped byte buffer? false 00:06:21.296 [Thread-0] DEBUG loci.formats.ClassList -- Could not find loci.formats.in.URLReader java.lang.ClassNotFoundException: loci.formats.in.URLReader at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at loci.formats.ClassList.parseLine(ClassList.java:196) at loci.formats.ClassList.parseFile(ClassList.java:258) at loci.formats.ClassList.(ClassList.java:138)

.......................

00:06:27.945 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 12706 Offset 561612867 00:06:27.949 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 12012 Offset 562366606 00:06:27.952 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 12654 Offset 563101287 00:06:27.956 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 10309 Offset 563834629 00:06:27.959 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 13852 Offset 564582166 00:06:27.963 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 14513 Offset 565362621 00:06:27.966 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 14102 Offset 566153907 00:06:27.970 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 14210 Offset 566949636 00:06:27.976 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 5575 Offset 567734454 00:06:27.979 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 3502 Offset 568461339 00:06:27.982 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 5183 Offset 569159654 00:06:27.986 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 8849 Offset 569853110 00:06:27.989 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 9327 Offset 570551552 00:06:27.993 [Thread-0] DEBUG loci.formats.tiff.TiffParser -- Reading tile Length 9548 Offset 571258525

CDPDisk avatar Nov 28 '24 16:11 CDPDisk

And this is code, referenced this code: @https://github.com/tand826/wsiprocess/issues/49#issuecomment-1502857149


import os
import contextlib
# install openslide-bin when use openslide-python > 1.4.1
# with os.add_dll_directory('C:/Program Files/openslide-win64-20220811/bin'):
#     from openslide import OpenSlide
# import openslide
# from openslide import OpenSlide
import javabridge
import bioformats
import torch
from torch.utils.data import Dataset, DataLoader
from torchvision import transforms
import numpy as np
import time
import multiprocessing

class DatasetWithBioformats(Dataset):

    def __init__(self, wsi, tile_size):
        self.wsi = wsi
        self.tile_size = tile_size
        self.transform = transforms.Resize((self.tile_size, self.tile_size))

    def lazy_load(self):
        javabridge.start_vm(class_path=bioformats.JARS, run_headless=True)
        self.reader = bioformats.ImageReader(self.wsi)
        self.set_params()

    def __del__(self):
        if hasattr(self, "reader"):
            self.reader.close()
        javabridge.kill_vm()

    def __len__(self):
        return len(self.coords)

    def __getitem__(self, idx):
        if not hasattr(self, "reader"):
            self.lazy_load()
        x, y = np.random.randint(0, self.width-self.tile_size), np.random.randint(0, self.height-self.tile_size)
        w = min(self.tile_size, self.width - x)
        h = min(self.tile_size, self.height - y)
        w, h = min(self.tile_size, self.width - x), min(self.tile_size, self.height - y)
        tile = self.reader.read(c=0, rescale=False, XYWH=(x, y, w, h))
        tensor = torch.tensor(tile, dtype=torch.uint8).reshape(h, w, 3).permute(2, 0, 1)
        return self.transform(tensor)

    def set_params(self):
        ImageReader = javabridge.JClassWrapper("loci.formats.ImageReader")
        reader = ImageReader()
        reader.setId(self.wsi)
        self.width = reader.getSizeX()
        self.height = reader.getSizeY()
        reader.close()

try:
    tile_size = 256*40
    wsi = '../../data/FUSCC2/vlm/batch1/NeoPlatform/2022-34841.svs'
    dataset = DatasetWithBioformats(wsi, tile_size)
    start_time = time.time()
    print('start')
    for i in range(1):
        dataset.__getitem__(i)
    print(f"Time elapsed: {time.time()-start_time:.2f} s")
    print('Done')
finally:
    javabridge.kill_vm()

CDPDisk avatar Nov 28 '24 16:11 CDPDisk

By the way, I'm getting the following error when I close ImageReader, but I have no idea:

Exception ignored in: <function DatasetWithBioformats.__del__ at 0x2aaca72745e0>
Traceback (most recent call last):
  File "test_multiprocess_java2.py", line 31, in __del__
  File "/data/aim_nuist/aim_chendp/.conda/envs/envtest/lib/python3.8/site-packages/bioformats/formatreader.py", line 712, in close
  File "/data/aim_nuist/aim_chendp/.conda/envs/envtest/lib/python3.8/site-packages/javabridge/jutil.py", line 961, in method
  File "/data/aim_nuist/aim_chendp/.conda/envs/envtest/lib/python3.8/site-packages/javabridge/jutil.py", line 888, in call
  File "/data/aim_nuist/aim_chendp/.conda/envs/envtest/lib/python3.8/site-packages/javabridge/jutil.py", line 842, in make_call
AttributeError: 'NoneType' object has no attribute 'get_object_class'

CDPDisk avatar Nov 28 '24 16:11 CDPDisk