ServerCore only support device_types::kMain type for resource tracker

[sw2921]

Currently I am trying to use server core for the model management, but from the code, ServerCore only create ResourceTracker with main memory (kMain) support. Do we consider adding the GPU(kGpu) device into resource tracker of ServerCore? Or do we have some alternatives to support main ram and gpu ram together in resource tracker?

[netfs]

i think we should track GPU resource usage under kGPU to help differentiate from main. monitoring tools can aggregate (sum/rates etc.) as desired.

[huijiao1120]

Has the gpu resource estimator been supported yet? I see a gpu device type is available here: const char* const kGpu = "gpu"; but when I do set_device(tensorflow::serving::device_types::kGpu) I get this error: tensorflow_serving/resources/ resource _util.cc:116] Invalid argument: Invalid resource allocation: Invalid device gpu

[singhniraj08]

@sw2921,

GPU resource tracker for model server is tracked in kGpu as shown here. Please let us know if this issue can be closed.

@huijiao1120, Please make sure you have installed Tensorflow Serving for GPU and make sure the pre-requisites are met.

Thank you!

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.