[Paper-Cut:ACI Deployment] Very slow deploying a single model container into ACI

[CESARDELATORRE]

A container into ACI should only need a matter of seconds. If it needs so much time it is probably because it is re-creating Docker images or because ACI should have the Docker images cached so it won't need to pull the image on every deployment.

It'd be important to have those Docker images cached per model so it won't need to pull the Docker image into ACI for every deployment and deployment to ACI would be a lot faster.

Example of timing of deployment to ACI:

Model deployment to ACI: --- 200.5308485031128 seconds needed to deploy to ACI ---

--> That is around 3.4 minutes for deploying a single Docker container..

[danielsc]

if you are using the 'new' deployment path, then deploying to AKS (after the first time) takes only a few seconds. ACI is distinctly slower (not sure why) -- @jpe316, do you know?

[CESARDELATORRE]

I think AKS is able to cache the Docker Image from the ACR Registry after doing the first pull of the image. However with ACI it might need to always pull the Docker image. But this improvement should be followed up with the ACI team

[CESARDELATORRE]

Filed relared perf papercut: https://msdata.visualstudio.com/Vienna/_workitems/edit/587847