serving icon indicating copy to clipboard operation
serving copied to clipboard

Why is one core docker container better than four cores docker container ?

Open luoyger opened this issue 3 years ago • 0 comments

I had started an one-core docker container and a four-cores docker container, and test by 100 batch data and the same QPS. The model is recommend domain model.

resource pod num p99(ms) p95(ms) p80(ms) avg(ms) total cpu num cpu improve
4c2g 87 13.06 9.63 8.11 5.03 348  
1c2g 219 9.95 9.55 8.04 5.03 219 37.1%

I want to know why one-core tfserving deploying is better than four-cores ? more cores should be better ?

At the same, I configured the batching config file, changed all params value below, it did not make any difference, I am not sure I had configured it by the right way. Can someone tell me how to adjust these params to make sense ?

max_batch_size { value: 1000 }
batch_timeout_micros { value: 0 }
max_enqueued_batches { value: 1 }
num_batch_threads { value: 1 }

luoyger avatar May 26 '22 12:05 luoyger