bugbug
bugbug copied to clipboard
mozilla
Update python image in task cluster
@Amotul-raheem I would advise convert the images one by one. Also, you do not need to convert all of them, only where possible. Installing the missed tools should not significantly impact the build time compared to using the full Python image.
Note: It could be easier for you to experiment locally (create a Docker file that reproduces the same steps).
@Amotul-raheem I would advise covert the images one by one. Also, you do not need to convert all of them, only where possible. Installing the missed tools should not significantly impact the build time compared to using the full Python image.
Note: It could be easier for you to experiment locally (create a Docker file that reproduces the same steps).
Yes, I did this exact same thing. I created a sample flask server running on docker and I sh into the running container using docker exec it [container_id] sh to test some of the commands to see what dependencies I need to install.
I think everything is working fine now, but bugbug tests status is exception and I'm not sure of what to do. Maybe rerun the task again by making a dummy commit?
@suhaibmujahid all tasks passed successfully.However, codecov dropped as expected. Please, how do I go about resolving this?
@Amotul-raheem could you evaluate if using slim + apt update + apt install git is more or less efficient than just using the full image?
@Amotul-raheem could you evaluate if using slim + apt update + apt install git is more or less efficient than just using the full image?
could you please explain what you mean by efficient here?
could you please explain what you mean by efficient here?
I mean time to build (how fast)
@suhaibmujahid I did the analysis and my findings are in the table below.
| image | build time | size |
|---|---|---|
| python full image | 1m 28.463s | 915MB |
| python-slim + apt update + apt install git | 19.566s | 225MB |
I think python-slim is better as it has less build time and smaller image size
Thank you, @Amotul-raheem!
The time saving seems to be highly correlated with the image size. This indicates that the time saving is a reflection of the lower network traffic. I guess these results are based on a local test. I'm not sure if it will be the same on CI. If Taskcluster has any kind of caching or an on-prem image registry, traffic will have a smaller footprint on the overall result.
Thank you, @Amotul-raheem!
The time saving seems to be highly correlated with the image size. This indicates that the time saving is a reflection of the lower network traffic. I guess these results are based on a local test. I'm not sure if it will be the same on CI. If Taskcluster has any kind of caching or an on-prem image registry, traffic will have a smaller footprint on the overall result.
Interesting, thinking about it now it makes sense. Should we close the PR and the issue then?
Agreed, timings on Taskcluster might be different. We can compare runs between master and the PR to see what the effect is.
It made sense to add python slim package to only two tasks; the bugbug lint testand bugbug tag version check.
After triggering the task cluster using python slim and without python slim, the bugbug lint test executed with the same time,(3 minutes).
With this, i don't think using the python slim image is going to have any significant effect on the run time of these processes.
@suhaibmujahid @marco-c , After these analysis, should the PR and issue be closed?
Thank you for valuable contrbution here, @Amotul-raheem! I lean toward using the full Python images. A smaller image size could still be slightly benfitule in case we are pushing the built images to the registry. @marco-c wdyt?
I agree, let's keep using the full Python image. Thank you anyway @Amotul-raheem, this was very useful investigation!