Eventual error when converting svs to tiff

[Tomaster134]

Windows 10 Python 3.12.2 libvips 8.16 [all] 64gb RAM

Using John's method detailed here, I was successfully converting a 469MB .svs file into a .tiff format. I was fiddling with settings and continuously deleting and running the same snippet of code to see if I could get it to where I wanted it, when suddenly it started converting to grayscale, and then after a few more tries it started tossing out these errors. I haven't been able to determine what the cause is. Am I missing something obvious here?

Code

import os
vipsbin = r'E:\Home\Tomaster\vips-dev-8.16\bin'
add_dll_dir = getattr(os, 'add_dll_directory', None)
if callable(add_dll_dir):
    add_dll_dir(vipsbin)
else:
    os.environ['PATH'] = os.pathsep.join((vipsbin, os.environ['PATH']))


import pyvips

im = pyvips.Image.new_from_file("test/test.svs", attach_associated=True, rgb=True)

image_height = im.height

# split to separate image planes and stack vertically ready for OME 
im = pyvips.Image.arrayjoin(im.bandsplit(), across=1)

# set minimal OME metadata
# before we can modify an image (set metadata in this case), we must take a 
# private copy
im = im.copy()
im.set_type(pyvips.GValue.gint_type, "page-height", image_height)
im.set_type(pyvips.GValue.gstr_type, "image-description",
f"""<?xml version="1.0" encoding="UTF-8"?>
<OME xmlns="http://www.openmicroscopy.org/Schemas/OME/2016-06"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2016-06 http://www.openmicroscopy.org/Schemas/OME/2016-06/ome.xsd">
    <Image ID="Image:0">
        <!-- Minimum required fields about image dimensions -->
        <Pixels DimensionOrder="XYCZT"
                ID="Pixels:0"
                SizeC="3"
                SizeT="1"
                SizeX="{im.width}"
                SizeY="{image_height}"
                SizeZ="1"
                Type="uint8">
        </Pixels>
    </Image>
</OME>""")

im.write_to_file("test/tada.tiff", compression="jpeg", tile=True,
            tile_width=512, tile_height=512,
            pyramid=True, subifd=True, bigtiff=True)

Error

Traceback (most recent call last): File "E:\Home\Tomaster\Documents\tiff_co\svs_to_tiff.py", line 44, in im.write_to_file("test/tada.tiff", compression="jpeg", tile=True, File "E:\Home\Tomaster\Documents\tiff_co\venv\Lib\site-packages\pyvips\vimage.py", line 781, in write_to_file return pyvips.Operation.call(name, self, filename, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\Home\Tomaster\Documents\tiff_co\venv\Lib\site-packages\pyvips\voperation.py", line 301, in call raise Error(f'unable to call {operation_name}') pyvips.error.Error: unable to call VipsForeignSaveTiffFile openslide2vips: reading region: Couldn't open test/test.svs: Too many open files test/test.svs: Cannot read TIFF header openslide2vips: reading region: Couldn't open test/test.svs: Too many open files openslide2vips: reading region: Couldn't open test/test.svs: Too many open files ...repeats ad nauseam with several hundred more errors

[jcupitt]

Hi @Tomaster134,

openslide2vips: reading region: Couldn't open test/test.svs: Too many open files

Windows has a limit of 1024 open files, perhaps you've somehow hit that? Restarting your python session should reset it.

[Tomaster134]

I've tried killing the terminal window, deleting and rebuilding the venv, and resetting my whole computer, all to no avail. Is there something else that I'm missing that would be worth trying?

[jcupitt]

Oh dear :( Maybe the file is corrupt? Have you tried a known-good SVS? For example:

https://openslide.cs.cmu.edu/download/openslide-testdata/Aperio/CMU-1.svs

[jcupitt]

I'd remove attach_associated=True, I don't think you need it.

[Tomaster134]

Hmm. I just ran it on the CMU-1.svs, and it didn't throw any errors, but it converted to grayscale?

Also removed the attach_associated since you are correct, it seems unnecessary.

[jcupitt]

What are you using to view the TIFF file? It'll need to be something like QuPath or vipsdisp that has a fancy OME-TIFF loader.

[Tomaster134]

Oops! I was being lazy and going off the thumbnail, it comes out perfectly in QuPath. I guess my .svs was just corrupted in some way? I'm curious as to why it didn't start throwing error messages until a few hours in, but oh well. It also doesn't seem to throw the error messages if I'm not using the im = im.copy() to edit metadata, but maybe that's just due to the corruption?

[jcupitt]

I could have a look at the file if you have somewhere to upload it.

[Tomaster134]

I can definitely do that, but I'd have to remove identifying information from the slide label. Would it still be useful if it's been changed in that way?

[jcupitt]

Sure, but you may not be able to anonymise it if the file has been truncated :(