Runtime error using Numpy readers: Unknown Numpy type c8

[thomas-bouvier]

Is this a new feature, an improvement, or a change to existing functionality?

Improvement

How would you describe the priority of this feature request

Must have (e.g. DALI adoption is impossible due to lack in functionality).

Please provide a clear description of problem this feature solves

Hello,

Currently, the numpy reader operator does not support complex-floating point types like c8 or c16. This would be a nice addition to load such data from npy files.

When loading such a npy file, the following error gets raised Error in thread 0: [/opt/dali/dali/operators/reader/loader/numpy_loader_gpu.cc:42] [/opt/dali/dali/util/numpy.cc:40] Unknown Numpy type string: c8:

[1,0]<stderr>:  File "/root/distributed-continual-learning/data/load.py", line 202, in __init__
[1,0]<stderr>:    test = taskset_pipeline.run()
[1,0]<stderr>:  File "/my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/pipeline.py", line 1113, in run
[1,0]<stderr>:    return self.outputs()
[1,0]<stderr>:  File "/my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/pipeline.py", line 956, in outputs
[1,0]<stderr>:    return self._outputs()
[1,0]<stderr>:  File "/my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/pipeline.py", line 1040, in _outputs
[1,0]<stderr>:    return self._pipe.Outputs()
[1,0]<stderr>:RuntimeError: Critical error in pipeline:
[1,0]<stderr>:Error when executing GPU operator readers__Numpy, instance name: "__Numpy_1", encountered:
[1,0]<stderr>:Error in thread 0: [/opt/dali/dali/operators/reader/loader/numpy_loader_gpu.cc:42] [/opt/dali/dali/util/numpy.cc:40] Unknown Numpy type string: c8
[1,0]<stderr>:Stacktrace (10 entries):
[1,0]<stderr>:[frame 0]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali.so(+0xd300b) [0x7f510477900b]
[1,0]<stderr>:[frame 1]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali.so(+0x961d8) [0x7f510473c1d8]
[1,0]<stderr>:[frame 2]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali.so(dali::numpy::ParseHeaderContents(dali::numpy::HeaderData&, std::string const&)+0x10b) [0x7f51048b8a0b]
[1,0]<stderr>:[frame 3]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali.so(+0x2132bd) [0x7f51048b92bd]
[1,0]<stderr>:[frame 4]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali.so(dali::numpy::ParseHeader(dali::numpy::HeaderData&, dali::InputStream*)+0x290) [0x7f51048ba0a0]     
[1,0]<stderr>:[frame 5]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali_operators.so(+0x3c7691f) [0x7f509160791f]
[1,0]<stderr>:[frame 6]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali.so(dali::ThreadPool::ThreadMain(int, int, bool, std::string const&)+0x1e6) [0x7f510485fe86]
[1,0]<stderr>:[frame 7]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali.so(+0x769dd0) [0x7f5104e0fdd0]
[1,0]<stderr>:[frame 8]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x7ea7) [0x7f51bae8fea7]
[1,0]<stderr>:[frame 9]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f51bac60a2f]
[1,0]<stderr>:. File: /my-spack/datasets/Ptycho/train/204/patched_psi.npy
[1,0]<stderr>:Stacktrace (6 entries):
[1,0]<stderr>:[frame 0]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali_operators.so(+0x686e8e) [0x7f508e017e8e]
[1,0]<stderr>:[frame 1]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali_operators.so(+0x530252) [0x7f508dec1252]
[1,0]<stderr>:[frame 2]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali.so(dali::ThreadPool::ThreadMain(int, int, bool, std::string const&)+0x1e6) [0x7f510485fe86]
[1,0]<stderr>:[frame 3]: /my-spack/view/._chifflot-v100/44wryknjqgrv67asl5pa7vbcfrxorbcb/lib/python3.10/site-packages/nvidia/dali/libdali.so(+0x769dd0) [0x7f5104e0fdd0]
[1,0]<stderr>:[frame 4]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x7ea7) [0x7f51bae8fea7]
[1,0]<stderr>:[frame 5]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f51bac60a2f]
[1,0]<stderr>:
[1,0]<stderr>:Current pipeline object is no longer valid.

Error is triggered here.

Feature Description

I would like to load c8 complex types from a npy file.

A following request for bool here.

Describe your ideal solution

Ideally, I would like to be able to load this file using a numpy reader.

[I will upload a npy file very soon]

Describe any alternatives you have considered

No response

Additional context

No response

Check for duplicates

• [X] I have searched the open bugs/issues and have found no duplicates for this bug report

[JanuszL]

Hi @thomas-bouvier,

While technically feasible (still challenging) I'm not sure if we see a good use case for it. It would be very helpful if you could describe what is the workflow you want to use this feature for. From the DALI point of view, we would need to create an internal representation (either one tensor that stores this type or two for real and img part) of it and think which operator should support it.

[thomas-bouvier]

Hello @JanuszL, thank you for the feedback (and sorry for the delay).

I understand that this would be an advanced feature, probably not useful to many. Still, let me explain my use case for complex-floating point types.

I am working on xray imaging using diffraction patterns as input data. These diffraction patterns are acquired by a synchrotron light source. A DNN model is used to reconstruct 2 images for every single diffraction pattern : a structure image (amplitude) and a phase image. This is where working with floating point types is needed: the ground-truth data is a collection of numpy arrays containing complex types, from which one can calculate the ground-truth structure and phase above.

As of now, this is how I calculate the ground-truth structure and phase images from the complex numpy arrays rspace_data (raw ground-truth data):

task_ampli_data = []
task_phase_data = []
for i, _ in enumerate(tqdm(file_paths, desc=f"Loading {len(file_paths)} perspectives")):
    # Complex data
    rspace_data = np.load(rspace_paths[i])
    # Calculating the phase and amplitude from the real-space data
    ampli_data = np.abs(rspace_data)
    phase_data = np.angle(rspace_data)

    # Concatenating scan position(s) for this task
    ...
    task_ampli_data.extend(ampli_data[idx][shard_offset : shard_offset + shard_size])
    task_phase_data.extend(phase_data[idx][shard_offset : shard_offset + shard_size])

task_ampli_data = np.array(task_ampli_data, dtype=np.float32)
task_phase_data = np.array(task_phase_data, dtype=np.float32)
taskset = (task_diff_data, task_ampli_data, task_phase_data)

The shape of an individual rspace_data npy file is 1000x1x256x256, giving ampli_data and phase_data of same shapes.

Ideally, I would be able to write the following:

@pipeline_def(batch_size=1, num_threads=1, device_id=device_id)
def input_pipeline():
    file_paths = taskset.get_raw_samples()[0]
    rspace_paths = [f"{p}/patched_psi.npy" for p in file_paths]

    # This npy file contains complex numbers, unfortunately not
    # supported by DALI
    rspace_data = fn.readers.numpy(
        device="gpu",
        files=rspace_paths,
        shard_id=shard_id,
        num_shards=num_shards,
    )

Later on, we could leverage abs and angle operators to calculate ampli_data and phase_data in the pipeline directly.

This is just an idea, your feedback is appreciated :)

[JanuszL]

Hi @thomas-bouvier,

Thank you for providing more background regarding your use case. Based on how we understand the ask, this is to make the numpy reader capable of loading files with complex number types, adding a complex type representation to DALI, and reviewing the available operations to see if they should support complex types. We will evaluate the request and see how it fits our roadmap, in the meantime can you try Python operator on the GPU which can wrap the loading part and the conversion from the complex numbers to the real one, then you can process the data further using the existing DALI operators.

[thomas-bouvier]

Thank you for the feedback. Here is an archive containing 5 diffraction patterns patterns.npy.tar.gz.