How much of the software is available on cvfms? Why are you polluting my laptop

[alexandertuna]

As a final note: the latter set of dependencies are non-standard in the ATLAS workflow but are very common in the data science world and are easy to install with standard installation tools like pip. As such Matt will factorize things: first he'll produce data files on lxplus, then he'll run the training on his laptop or some other system with minimal dependencies on "HEP" tools.

^ I reject this. Didn't you ask the sft people to add these?

[alexandertuna]

Actually, I wonder if this already exists. lsetup sft (and the subsequent lcgenv's) has a lot of stuff.

lsetup sft -h | grep keras
 releases/LCG_86/keras/1.1.0
 releases/LCG_87/keras/1.1.0
 releases/LCG_88/keras/1.1.0
 releases/LCG_88Py3/keras/1.1.0
 releases/LCG_89/keras/1.1.0
 releases/LCG_89python3/keras/1.1.0
 releases/LCG_90/keras/1.1.0
 releases/LCG_90a/keras/1.1.0
 releases/LCG_90python3/keras/1.1.0
 releases/LCG_91/keras/1.1.0
 releases/LCG_91python3/keras/1.1.0
 releases/LCG_92/keras/2.1.1
 releases/LCG_92python3/keras/2.1.1

[alexandertuna]

Lemme attempt to work through this entirely on cvmfs.

[dguest]

Cool, let me know how it goes. I was thinking about updating things to work with lxplus, but even then I think my recommended approach would be via images in singularity. Actually I'm supposed to review some instructions to use that (see here).

LCG is very slow with releases and I don't realistically expect them to keep every useful ML package up to date. Once I get this working with containers it won't matter if it's lxplus or your laptop, since you'll be able to run either one via an image.