NVIDIA
How can one use DALI's C API, eg. interface with caffe
For now, image operations such as resizing, cropping (like, using opencv) before feeding to caffe model are in host CPU. Can we use DALI to load, decode, resize images in GPU and then directly feed GPU data to caffe? It seems that DALI's examples are all interfacing with Python based frameworks like TF and that DALI does not contain a high level interface for C/C++. c_api.h is rather simple, with which one cannot do specified operations on images and it only includes methods on pipeline.
Hi, It is a very good question. The idea behind DALI it to operate in the way you described - to leave the whole processing pipeline to DALI and get data in GPU memory at the end. Our initial goal is to support most popular frameworks and we provide plugins for MXNet, TensorFlow, and PyTorch. Current API is python based, one you see for C is meant to be used only for TensorFlow integration so it is nor easy to use nor feature completed. We have complete C/C++ API in our roadmap but no sooner than after version 1.0 is released. So please be patient and stay tuned for new releases.
After consideration let us keep that open as a place when we can gather all question, requirements and use cases for C/C++ API. This will allow us to decide what is worth implementing and what is not worth an effort. Internally tracked DALI-186.
Also, with the upcoming Tensorflow 1.12 (now 1.12.0-rc1):
TensorFlow binaries are built with XLA support linked in by default.
For TensorFlow we already use some simple C API which allows you to build serialized python and run it.
I think that as Dali pipelines could already roughly be built in c++ we could start to have just some very minimal glue code from consuming the pipeline sink and different Tensors c++ (pytorch, tensorflow) without too much integration. This could boost c++ production inference as a first use case without requiring to much CPU/GPU memory move at least for decoding, scaling, normalization and inference (that is a quite standard pipeline).
A minimal c++ inference coverage could be also useful for your own inference engine (TensorRT) use case that has already both c++ and python interfaces: https://github.com/NVIDIA/DALI/issues/17
Inference use case is partially related to https://github.com/NVIDIA/DALI/issues/135 but for c++
For who here is interested in pytorch c++ traced/scripted model and consuming gpu blob input it can check https://github.com/pytorch/pytorch/issues/13898
This is now the top first sorted issue of the repository for positive reactions.
Hi @bhack, Fully understand your concern. #330, #338, #346 - are parts of bigger rework we are doing now. We still remember about you ask and this is important for us as well. We just don't wan to make any C++ API unless we are more or less sure it will last for some time. If you cannot wait longer please use exiting one - you can refer to https://github.com/NVIDIA/DALI/blob/master/dali/pipeline/pipeline_test.cc to see how that works.
@JanuszL I understand and I totally agree with you that you want API stability first. But I think that a good compromise in the meantime is to just to maintain a very very minimal example in the repository. What i suggest is: if you can create a minimal example on how to connect one reader class (I suppose image sequence or Videoreader are the most interesting ones), convert the output to c++ pytorch Aten tensor and Tensorflow c++ tensor and then compile this sample as standalone (linking, flags etc..). It could be enough and quite fast to have for you until Dali is ready to really support a full C++ API.
We probably need a C/C++ API. This allows everyone to plug DALI in, instead of only used by Python.
We probably need a C/C++ API. This allows everyone to plug DALI in, instead of only used by Python.
+1 !
Hi @JanuszL, what is the status of supporting a C++ API? is it still in the roadmap? Thanks in advance
Hi @AhmadZakaria,
We plan to get to it eventually. Could you tell me what is your use case and how you would like to use the C++ API?
I'm currently doing DL inference in C++ using Tensorrt as well as Onnxruntime, and I would like to also do the preprocessing on the gpu. Preprocessing includes scaling the image, rearranging the channels, multiplying different channels with different coefficients, etc...
DALI has a C API that allows you to provide input, run the serialized pipeline, and copy back the outputs. This API is used in the DALI TRITON backend. In many cases, it is more convenient to define the pipeline once in python, serialize it and then store it in the server. In your case, this could be a way to go.