deepforge icon indicating copy to clipboard operation
deepforge copied to clipboard

Default installed packages in workers

Open brollb opened this issue 4 years ago • 1 comments

Although operations can define their own custom dependencies, it might be nice to include more default packages in the operations. This is especially motivating since conda is somewhat slow so installation of these common data science packages may be a bit annoying.

@umesh-timalsina and I have been talking about this somewhat recently and @dustinjoe also brought it up recently here so it is probably about time it got an official issue :)

If we decide to go this route, some candidates for being included in the base environment:

  • scikit learn
  • numpy
  • pandas

brollb avatar Jul 13 '20 21:07 brollb

This would be nice. One of the issues I faced while working on #1747 is that the more packages there are in your base environments, the more it takes for conda to export it. It will not be an issue for local and gme computes, but can cause lag problems in for example -> ephemeral computes like sciserver-compute, where base conda environments do not exist

umesh-timalsina avatar Jul 13 '20 21:07 umesh-timalsina