serving
serving copied to clipboard
Why is one core docker container better than four cores docker container ?
I had started an one-core docker container and a four-cores docker container, and test by 100 batch data and the same QPS. The model is recommend domain model.
| resource | pod num | p99(ms) | p95(ms) | p80(ms) | avg(ms) | total cpu num | cpu improve |
|---|---|---|---|---|---|---|---|
| 4c2g | 87 | 13.06 | 9.63 | 8.11 | 5.03 | 348 | |
| 1c2g | 219 | 9.95 | 9.55 | 8.04 | 5.03 | 219 | 37.1% |
I want to know why one-core tfserving deploying is better than four-cores ? more cores should be better ?
At the same, I configured the batching config file, changed all params value below, it did not make any difference, I am not sure I had configured it by the right way. Can someone tell me how to adjust these params to make sense ?
max_batch_size { value: 1000 }
batch_timeout_micros { value: 0 }
max_enqueued_batches { value: 1 }
num_batch_threads { value: 1 }