Huge GPU memory consumption for RetinaFace(resnet50)

[barzan-hayati]

Env

• GPU: 1080Ti
• OS: Ubunut 18.04
• Cuda version: 11.4
• TensorRT version: 8.2.0.6

About this repo

• which branch/tag/commit are you using? tensorrtx retinaface
• which model? RetinaFace(resnet50)

Your problem

• what is your command?

I want to utilize this code to do inference on a retinaface model:

void check_retinaface_TRT_file_correctness(bool& check_file_once_retina)
{
    if (file_retina.good() && check_file_once_retina)
    {
        cudaSetDevice(DEVICE_retina);
        file_retina.seekg(0, file_retina.end);
        size_retina = file_retina.tellg();
        file_retina.seekg(0, file_retina.beg);
        trtModelStream_retina = new char[size_retina];
        assert(trtModelStream_retina);
        file_retina.read(trtModelStream_retina, size_retina);
        file_retina.close();

        runtime_retina = createInferRuntime(gLogger_retina);
        // assert(runtime != nullptr);
        engine_retina = runtime_retina->deserializeCudaEngine(trtModelStream_retina, size_retina);
        //  assert(engine != nullptr);
        context_retina = engine_retina->createExecutionContext();
        //  assert(context != nullptr);
        std::cout<<"\033[1;36m Cheking retinaface Tensorrt file was successful!  \033[0m"<<std::endl;
    }
    if(check_file_once_retina)
        delete[] trtModelStream_retina;

    check_file_once_retina=false;

}

It works properly but what wonder me is : consuming about 1.2GB of GPU Memory. Why this model with approximate 5MB weight takes such a huge Memory?

• what output do you expect? I expect to low GPU consumtion not more.