nvImageCodec
nvImageCodec copied to clipboard
NVIDIA
Incorrect decoding of 16-bit TIFF images
Version
0.2.0.7
Describe the bug.
Hi, I'm encountering an issue when decoding 16-bit TIFF images using the nvImageCodec library. The decoded image appears to be duplicated and the range of pixel values is different from the expected range.
Expected result: The decoded image should match the original image, both in terms of pixel values and visual appearance.
Actual result: The decoded image appears to be duplicated and the range of pixel values is different from the expected range.
Expected Output:
Actual Output:
Minimum reproducible example
import cupy as cp
import numpy as np
from nvidia import nvimgcodec
import matplotlib.pyplot as plt
from tifffile import imread, imwrite
def sh_info(img, header=""):
print(header)
print(f"\tShape: {img.shape}")
print(f"\tMin value: {img.min()}")
print(f"\tMax value: {img.max()}")
print(f"\tData type: {img.dtype}")
# Creating a random 16-bit image
arr = np.random.random((4096, 4096))
arr *= 60000
arr[100:400, 100:400] = 61000
arr = arr.astype(np.uint16)
sh_info(arr, "Original Image")
# Writing the image to a tif file and reading it back
imwrite('test.tif', arr, dtype=np.uint16)
img0 = imread('test.tif')
sh_info(img0, "tifffile Decoded Image")
# Decode the image using nvimgcodec
params = nvimgcodec.DecodeParams(color_spec=nvimgcodec.ColorSpec.UNCHANGED, allow_any_depth=True)
dec = nvimgcodec.Decoder()
img_decoded = dec.read('test.tif', params)
# Convert the decoded image to a CuPy array
cp_img = cp.asarray(img_decoded).get()
sh_info(cp_img, "nvImageCodec Decoded Image")
# Create a figure with 3 subplots
plt.figure(figsize=(15, 5))
plt.subplot(1, 3, 1)
plt.title('Original Image')
plt.imshow(arr)
plt.subplot(1, 3, 2)
plt.title("tifffile Decoded Image")
plt.imshow(img0)
plt.subplot(1, 3, 3)
plt.title('nvImageCodec Decoded Image')
plt.imshow(cp_img)
plt.tight_layout()
plt.show()
Environment details
<details><summary>Click here to see environment details</summary><pre>
**git***
commit b1b3f28d3bd52804370bc246d145164cf412f8df (HEAD -> data-loading-enhancement, origin/data-loading-enhancement)
Author: Crackvignoule <[email protected]>
Date: Tue Apr 23 14:52:58 2024 +0200
update tests
**git submodules***
***OS Information***
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.6 LTS"
NAME="Ubuntu"
VERSION="20.04.6 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.6 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Linux aar164 5.4.0-169-generic #187-Ubuntu SMP Thu Nov 23 14:52:28 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
***GPU Information***
Wed Apr 24 09:44:28 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.39.01 Driver Version: 510.39.01 CUDA Version: 11.6 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA A100 80G... On | 00000000:CA:00.0 Off | 0 |
| N/A 29C P0 64W / 300W | 5982MiB / 81920MiB | 0% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| 0 N/A N/A 12726 C .../envs/env_cuda/bin/python 539MiB |
| 0 N/A N/A 13127 C ...a_v11/ollama_llama_server 5441MiB |
+-----------------------------------------------------------------------------+
***CPU***
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 46 bits physical, 57 bits virtual
CPU(s): 128
On-line CPU(s) list: 0-127
Thread(s) per core: 2
Core(s) per socket: 32
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 106
Model name: Intel(R) Xeon(R) Platinum 8362 CPU @ 2.80GHz
Stepping: 6
CPU MHz: 800.101
BogoMIPS: 5600.00
Virtualization: VT-x
L1d cache: 3 MiB
L1i cache: 2 MiB
L2 cache: 80 MiB
L3 cache: 96 MiB
NUMA node0 CPU(s): 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62,64,66,68,70,72,74,76,78,80,82,84,86,88,90,92,94,96,98,100,102,104,106,108,110,112,114,116,118,120,122,124,126
NUMA node1 CPU(s): 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,57,59,61,63,65,67,69,71,73,75,77,79,81,83,85,87,89,91,93,95,97,99,101,103,105,107,109,111,113,115,117,119,121,123,125,127
Vulnerability Gather data sampling: Mitigation; Microcode
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Mitigation; Clear CPU buffers; SMT vulnerable
Vulnerability Retbleed: Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 invpcid_single ssbd mba ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb intel_pt avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local wbnoinvd dtherm ida arat pln pts avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg tme avx512_vpopcntdq rdpid md_clear pconfig flush_l1d arch_capabilities
***CMake***
/bin/cmake
cmake version 3.16.3
CMake suite maintained and supported by Kitware (kitware.com/cmake).
***g++***
/bin/g++
g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
***nvcc***
/usr/local/cuda-11/bin/nvcc
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Fri_Dec_17_18:16:03_PST_2021
Cuda compilation tools, release 11.6, V11.6.55
Build cuda_11.6.r11.6/compiler.30794723_0
***Python***
/nobackup/kp276129/envs/env_cuda/bin/python
Python 3.10.13
***Environment Variables***
PATH : /nobackup/kp276129/envs/env_cuda/bin:/nobackup/kp276129/data/miniconda3/condabin:/home/kp276129/.local/bin:/usr/local/cuda-11/bin:/bin:/sbin:/usr/X11R6/bin:/usr/sbin:/usr/bin:/etc:/home/cmd:/usr/ccs/bin:/usr/openwin/bin:/usr/dt/bin:/opt/sfw/bin:/home/prog/SUNWspro/bin:/home/prog/s1studio/ee/bin:/home/systeme/SUNWspro/bin:/home/systeme/SUNWste/bin:/usr/ucb:/usr/local/bin:.
LD_LIBRARY_PATH : /nobackup/Python/envs//lib:/usr/local/cuda/lib64:/usr/lib:/usr/openwin/lib:/home/prog/SUNWspro/lib:/home/prog/s1studio/ee/lib:/home/systeme/SUNWspro/lib:/home/systeme/SUNWspro/lib/v9:/opt/sfw/lib:/usr/ucblib:/home/prog/librairies/nag:/home/systeme/hpnp.6.15/lib:/usr/dt/lib:/opt/SUNWits/Graphics-sw/xil/lib/pipelines:/usr/local/lib:/usr/local/mysql/lib/mysql
NUMBAPRO_NVVM :
NUMBAPRO_LIBDEVICE :
CONDA_PREFIX : /nobackup/kp276129/envs/env_cuda
PYTHON_PATH :
***conda packages***
/nobackup/kp276129/data/miniconda3/condabin/conda
# packages in environment at /nobackup/kp276129/envs/env_cuda:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
appdirs 1.4.4 pypi_0 pypi
asttokens 2.4.1 pyhd8ed1ab_0 conda-forge
astunparse 1.6.3 pypi_0 pypi
blessed 1.20.0 pypi_0 pypi
build 1.0.3 pypi_0 pypi
bzip2 1.0.8 h7b6447c_0
ca-certificates 2023.11.17 hbcca054_0 conda-forge
cachecontrol 0.13.1 pypi_0 pypi
certifi 2023.11.17 pypi_0 pypi
cffi 1.16.0 pypi_0 pypi
charset-normalizer 3.3.2 pypi_0 pypi
cleo 2.1.0 pypi_0 pypi
click 8.1.7 pypi_0 pypi
click-default-group 1.2.4 pypi_0 pypi
cloudpickle 3.0.0 pypi_0 pypi
cloup 2.1.2 pypi_0 pypi
comm 0.1.4 pyhd8ed1ab_0 conda-forge
contourpy 1.2.0 pypi_0 pypi
crashtest 0.4.1 pypi_0 pypi
cryptography 41.0.7 pypi_0 pypi
cupy-cuda11x 12.3.0 pypi_0 pypi
cycler 0.12.1 pypi_0 pypi
debugpy 1.6.7 py310h6a678d5_0
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
distlib 0.3.8 pypi_0 pypi
dm-tree 0.1.8 pypi_0 pypi
dulwich 0.21.7 pypi_0 pypi
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
exceptiongroup 1.0.4 py310h06a4308_0
executing 2.0.1 pyhd8ed1ab_0 conda-forge
fastjsonschema 2.19.0 pypi_0 pypi
fastrlock 0.8.2 pypi_0 pypi
filelock 3.13.1 pypi_0 pypi
fonttools 4.45.1 pypi_0 pypi
fsspec 2024.3.1 pypi_0 pypi
gast 0.5.4 pypi_0 pypi
gitdb 4.0.11 pypi_0 pypi
glcontext 2.5.0 pypi_0 pypi
gpustat 1.1.1 pypi_0 pypi
idna 3.6 pypi_0 pypi
imageio 2.33.0 pypi_0 pypi
importlib-metadata 7.0.0 pypi_0 pypi
iniconfig 1.1.1 pyhd3eb1b0_0
installer 0.7.0 pypi_0 pypi
ipykernel 6.26.0 pyhf8b6a83_0 conda-forge
ipympl 0.9.3 pypi_0 pypi
ipython 8.18.1 pyh707e725_3 conda-forge
ipython-genutils 0.2.0 pypi_0 pypi
ipywidgets 8.1.1 pypi_0 pypi
isosurfaces 0.1.0 pypi_0 pypi
jaraco-classes 3.3.0 pypi_0 pypi
jedi 0.19.1 pyhd8ed1ab_0 conda-forge
jeepney 0.8.0 pypi_0 pypi
jupyter_client 7.3.4 pyhd8ed1ab_0 conda-forge
jupyter_core 5.5.0 py310hff52083_0 conda-forge
jupyterlab-widgets 3.0.9 pypi_0 pypi
keyring 24.3.0 pypi_0 pypi
kiwisolver 1.4.5 pypi_0 pypi
lazy-loader 0.3 pypi_0 pypi
ld_impl_linux-64 2.38 h1181459_1
libffi 3.4.4 h6a678d5_0
libgcc-ng 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libsodium 1.0.18 h36c2ea0_1 conda-forge
libstdcxx-ng 11.2.0 h1234567_1
libuuid 1.41.5 h5eee18b_0
line-profiler 4.1.2 pypi_0 pypi
llvmlite 0.42.0 pypi_0 pypi
locket 1.0.0 pypi_0 pypi
mako 1.2.4 pypi_0 pypi
manim 0.18.0 pypi_0 pypi
manimpango 0.5.0 pypi_0 pypi
mapbox-earcut 1.0.1 pypi_0 pypi
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 2.1.3 pypi_0 pypi
matplotlib 3.8.2 pypi_0 pypi
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
mdurl 0.1.2 pypi_0 pypi
memory-profiler 0.61.0 pypi_0 pypi
moderngl 5.9.0 pypi_0 pypi
moderngl-window 2.4.4 pypi_0 pypi
more-itertools 10.1.0 pypi_0 pypi
msgpack 1.0.7 pypi_0 pypi
multipledispatch 1.0.0 pypi_0 pypi
ncurses 6.4 h6a678d5_0
nest-asyncio 1.5.8 pyhd8ed1ab_0 conda-forge
networkx 3.2.1 pypi_0 pypi
numpy 1.26.0 pypi_0 pypi
nvidia-ml-py 12.535.133 pypi_0 pypi
nvidia-nvimgcodec-cu11 0.2.0.7 pypi_0 pypi
openssl 3.0.12 h7f8727e_0
packaging 23.1 py310h06a4308_0
parso 0.8.3 pyhd8ed1ab_0 conda-forge
partd 1.4.1 pypi_0 pypi
pexpect 4.8.0 pyh1a96a4e_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 9.5.0 pypi_0 pypi
pip 23.2.1 py310h06a4308_0
pkginfo 1.9.6 pypi_0 pypi
platformdirs 3.11.0 pypi_0 pypi
pluggy 1.0.0 py310h06a4308_1
poetry 1.7.1 pypi_0 pypi
poetry-core 1.8.1 pypi_0 pypi
poetry-plugin-export 1.6.0 pypi_0 pypi
prompt-toolkit 3.0.41 pyha770c72_0 conda-forge
psutil 5.9.0 py310h5eee18b_0
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pycairo 1.25.1 pypi_0 pypi
pycparser 2.21 pypi_0 pypi
pycuda 2022.2.2 pypi_0 pypi
pydub 0.25.1 pypi_0 pypi
pyglet 2.0.10 pypi_0 pypi
pygments 2.17.2 pyhd8ed1ab_0 conda-forge
pyparsing 3.1.1 pypi_0 pypi
pyproject-hooks 1.0.0 pypi_0 pypi
pyrr 0.10.3 pypi_0 pypi
pytest 7.4.0 py310h06a4308_0
python 3.10.13 h955ad1f_0
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python_abi 3.10 2_cp310 conda-forge
pytools 2023.1.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
pyzmq 25.1.0 py310h6a678d5_0
rapidfuzz 3.5.2 pypi_0 pypi
readline 8.2 h5eee18b_0
requests 2.31.0 pypi_0 pypi
requests-toolbelt 1.0.0 pypi_0 pypi
rich 13.7.0 pypi_0 pypi
scikit-image 0.22.0 pypi_0 pypi
scipy 1.11.4 pypi_0 pypi
screeninfo 0.8.1 pypi_0 pypi
secretstorage 3.3.3 pypi_0 pypi
setuptools 68.0.0 py310h06a4308_0
shellingham 1.5.4 pypi_0 pypi
six 1.16.0 pyh6c4a22f_0 conda-forge
skia-pathops 0.7.4 pypi_0 pypi
smmap 5.0.1 pypi_0 pypi
sqlite 3.41.2 h5eee18b_0
srt 3.5.3 pypi_0 pypi
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
svgelements 1.9.6 pypi_0 pypi
tifffile 2023.9.26 pypi_0 pypi
tk 8.6.12 h1ccaba5_0
tomli 2.0.1 py310h06a4308_0
tomlkit 0.12.3 pypi_0 pypi
toolz 0.12.1 pypi_0 pypi
tornado 6.1 py310h5764c6d_3 conda-forge
tqdm 4.66.1 pypi_0 pypi
traitlets 5.14.0 pyhd8ed1ab_0 conda-forge
trove-classifiers 2023.11.29 pypi_0 pypi
typing_extensions 4.8.0 pyha770c72_0 conda-forge
tzdata 2023c h04d1e81_0
urllib3 2.1.0 pypi_0 pypi
virtualenv 20.25.0 pypi_0 pypi
watchdog 3.0.0 pypi_0 pypi
wcwidth 0.2.12 pyhd8ed1ab_0 conda-forge
wheel 0.41.2 py310h06a4308_0
widgetsnbextension 4.0.9 pypi_0 pypi
xz 5.4.2 h5eee18b_0
zeromq 4.3.4 h2531618_0
zipp 3.17.0 pypi_0 pypi
zlib 1.2.13 h5eee18b_0
</pre></details>
Relevant log output
No response
Other/Misc.
The allow_any_depth=True parameter in DecodeParams seems to be causing the issue. However, setting it to False is not a viable solution for me, as I need to preserve the original data type of the image. When allow_any_depth is set to False, the only remaining issue is the incorrect range of pixel values.
Check for duplicates
- [X] I have searched the open bugs/issues and have found no duplicates for this bug report
~@Crackvignoule Thanks for reporting this. Could you please share the TIFF image so that I can reproduce on my end? Thank you~
Nevermind, I see in the snippet it is generated.
@Crackvignoule Thank you for reporting this. We can confirm it is indeed a bug and it will be fixed in the next nvImageCodec release.
@Crackvignoule Thank you for reporting this. We can confirm it is indeed a bug and it will be fixed in the next nvImageCodec release.
Hi, it's me again, do you have any clue when the next release should arrive ? Thank you
@Crackvignoule We are targeting July (next month) for this release. Thank you for your patience.