c++ onnxruntime linux gpu 1.14.1 can't infer

[neophack]

when infer decoder onnx model, some times got error in session.Run, some times got 0x0 size mask output. but python version onnxruntime-gpu 1.14.1 is work fine. error as following:

2023-04-26 10:54:56.695981402 [E:onnxruntime:, sequential_executor.cc:494 ExecuteKernel] Non-zero status code returned while running Resize node. Name:'/Resize_1' Status Message: upsamplebase.h:334 ScalesValidation Scale value should be greater than 0.
Non-zero status code returned while running Resize node. Name:'/Resize_1' Status Message: upsamplebase.h:334 ScalesValidation Scale value should be greater than 0.

or 0x0 size mask:

masks out shape: 1	4	0	0	
iou_predictions out shape: 1	4	
low_res_masks out shape: 1	4	256	256	

onnx models:

wget https://huggingface.co/visheratin/segment-anything-vit-b/resolve/main/encoder-quant.onnx
wget https://huggingface.co/visheratin/segment-anything-vit-b/resolve/main/decoder-quant.onnx

infer code:

    std::vector<std::shared_ptr<std::vector<float>>>
    Model::execute(std::vector<std::vector<float>> input_datas) {
        auto g_values = ((GlobalValues *) globel_values_);
        std::vector<Ort::Value> input_tensors;
        std::vector<const char *> input_names_cstr;
        for (int i = 0; i < g_values->input_names.size(); i++) {
            input_names_cstr.emplace_back(g_values->input_names[i].c_str());
//            size_t input_tensor_size = utils::vectorProduct(g_values->input_tensor_shape[i]);
            std::vector<float> input_tensor_values(input_datas[i].data(), input_datas[i].data() + input_datas[i].size());
            auto new_shape = g_values->input_tensor_shape[i];
            int negind=-1;
            size_t mulsize=1;
            for (int j=0;j<g_values->input_tensor_shape[i].size();j++){
                if(g_values->input_tensor_shape[i][j]==-1){
                    negind=j;
                }else{
                    mulsize*=g_values->input_tensor_shape[i][j];
                }
            }
            if(negind>0){
                new_shape[negind]=input_datas[i].size()/mulsize;
            }

            Ort::MemoryInfo memory_info = Ort::MemoryInfo::CreateCpu(
                    OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault);
            input_tensors.emplace_back(Ort::Value::CreateTensor<float>(
                    memory_info, input_tensor_values.data(),  input_datas[i].size(),
                    new_shape.data(), g_values->input_tensor_shape[i].size()
            ));
        }

        std::vector<const char *> output_names_cstr;
        for (int i = 0; i < g_values->output_names.size(); i++) {
            output_names_cstr.emplace_back(g_values->output_names[i].c_str());
        }
//        uint64_t startPostTime2 = Perception::utils::getMonotonicTimeMs();
        std::vector<Ort::Value> output_tensors = g_values->session.Run(Ort::RunOptions{nullptr},
                                                                       input_names_cstr.data(),
                                                                       input_tensors.data(),
                                                                       input_tensors.size(),
                                                                       output_names_cstr.data(),
                                                                       output_names_cstr.size());
//        std::cerr << "infer inside Process elapsed: " << Perception::utils::getMonotonicTimeMs() - startPostTime2
//                  << "(ms)" << std::endl;

        std::vector<std::shared_ptr<std::vector<float>>> out_tensors;
        g_values->output_tensor_shape.clear();
        for (int i = 0; i < g_values->output_names.size(); i++) {
            std::vector<int64_t> output_shape = output_tensors[i].GetTensorTypeAndShapeInfo().GetShape();
            size_t count = output_tensors[i].GetTensorTypeAndShapeInfo().GetElementCount();
            auto *rawOutput = output_tensors[i].GetTensorData<float>();
            std::shared_ptr<std::vector<float>> output = std::make_shared<std::vector<float>>(rawOutput,
                                                                                              rawOutput + count);
            out_tensors.emplace_back(output);
            g_values->output_tensor_shape.emplace_back(output_shape);
            std::cout << "out shape: ";
            for (auto shape: output_shape)
                std::cout << shape << "\t";
            std::cout << std::endl;
        }
        return out_tensors;
    }

[tempdeltavalue]

some values inside of your tensors maybe lower than zero

I'm also getting strange mask shape in my case is 1x1x1x1

@neophack Did you manage to run the decoder in C++?

[OroChippw]

when infer decoder onnx model, some times got error in session.Run, some times got 0x0 size mask output. but python version onnxruntime-gpu 1.14.1 is work fine. error as following:

2023-04-26 10:54:56.695981402 [E:onnxruntime:, sequential_executor.cc:494 ExecuteKernel] Non-zero status code returned while running Resize node. Name:'/Resize_1' Status Message: upsamplebase.h:334 ScalesValidation Scale value should be greater than 0.
Non-zero status code returned while running Resize node. Name:'/Resize_1' Status Message: upsamplebase.h:334 ScalesValidation Scale value should be greater than 0.

or 0x0 size mask:

masks out shape: 1	4	0	0	
iou_predictions out shape: 1	4	
low_res_masks out shape: 1	4	256	256	

onnx models:

wget https://huggingface.co/visheratin/segment-anything-vit-b/resolve/main/encoder-quant.onnx
wget https://huggingface.co/visheratin/segment-anything-vit-b/resolve/main/decoder-quant.onnx

infer code:

    std::vector<std::shared_ptr<std::vector<float>>>
    Model::execute(std::vector<std::vector<float>> input_datas) {
        auto g_values = ((GlobalValues *) globel_values_);
        std::vector<Ort::Value> input_tensors;
        std::vector<const char *> input_names_cstr;
        for (int i = 0; i < g_values->input_names.size(); i++) {
            input_names_cstr.emplace_back(g_values->input_names[i].c_str());
//            size_t input_tensor_size = utils::vectorProduct(g_values->input_tensor_shape[i]);
            std::vector<float> input_tensor_values(input_datas[i].data(), input_datas[i].data() + input_datas[i].size());
            auto new_shape = g_values->input_tensor_shape[i];
            int negind=-1;
            size_t mulsize=1;
            for (int j=0;j<g_values->input_tensor_shape[i].size();j++){
                if(g_values->input_tensor_shape[i][j]==-1){
                    negind=j;
                }else{
                    mulsize*=g_values->input_tensor_shape[i][j];
                }
            }
            if(negind>0){
                new_shape[negind]=input_datas[i].size()/mulsize;
            }

            Ort::MemoryInfo memory_info = Ort::MemoryInfo::CreateCpu(
                    OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault);
            input_tensors.emplace_back(Ort::Value::CreateTensor<float>(
                    memory_info, input_tensor_values.data(),  input_datas[i].size(),
                    new_shape.data(), g_values->input_tensor_shape[i].size()
            ));
        }

        std::vector<const char *> output_names_cstr;
        for (int i = 0; i < g_values->output_names.size(); i++) {
            output_names_cstr.emplace_back(g_values->output_names[i].c_str());
        }
//        uint64_t startPostTime2 = Perception::utils::getMonotonicTimeMs();
        std::vector<Ort::Value> output_tensors = g_values->session.Run(Ort::RunOptions{nullptr},
                                                                       input_names_cstr.data(),
                                                                       input_tensors.data(),
                                                                       input_tensors.size(),
                                                                       output_names_cstr.data(),
                                                                       output_names_cstr.size());
//        std::cerr << "infer inside Process elapsed: " << Perception::utils::getMonotonicTimeMs() - startPostTime2
//                  << "(ms)" << std::endl;

        std::vector<std::shared_ptr<std::vector<float>>> out_tensors;
        g_values->output_tensor_shape.clear();
        for (int i = 0; i < g_values->output_names.size(); i++) {
            std::vector<int64_t> output_shape = output_tensors[i].GetTensorTypeAndShapeInfo().GetShape();
            size_t count = output_tensors[i].GetTensorTypeAndShapeInfo().GetElementCount();
            auto *rawOutput = output_tensors[i].GetTensorData<float>();
            std::shared_ptr<std::vector<float>> output = std::make_shared<std::vector<float>>(rawOutput,
                                                                                              rawOutput + count);
            out_tensors.emplace_back(output);
            g_values->output_tensor_shape.emplace_back(output_shape);
            std::cout << "out shape: ";
            for (auto shape: output_shape)
                std::cout << shape << "\t";
            std::cout << std::endl;
        }
        return out_tensors;
    }

I meet the same problem , when i run the decoder in C++ ，my masks output shape is [1,4,0,0] , have you solve it? and when i get the iou_predictions shape[0] it will error , bu i can get the shape[1]😥