galaxyproject
Jupyter notebook (JupyLab tool) resources limit?
I’m doing an analysis using salmon alevin command and after running the command the process stops at around 25% of analysed cells (that’s around 6500 cells - I know that from the output log). It happens when I’m doing it in jupyter notebook (JupyLab tool) but the notebook cell has [*] next to it suggesting that it’s still running (but no new log output is displayed). But when I run the same command with the same input files locally (VSC) the analysis goes to completion. I had to massively down sample the dataset for that command to be executed successfully on .eu server. Interestingly, the same dataset still fails on .org server. I have no idea what causes that - is there a resource limit on interactive jupyter notebook? Well, I used monocle3 and did huge dimensionality reduction in jupyter notebook which seemed to be much more computationally intensive and took up to 15mins - but it worked, in contrast to alevin now...
This really depends on the server you are using. Can you please ask those questions at help.galaxyproject.org. In general ITs are limited. We offer salmon alevin as tool that gets more resources maybe you can use that? See you at help.galaxyproject.org.
It doesn't make sense that the resources are that low, that works on Commandline on an average or indeed poor laptop. She's not putting in a giant sample there, it should be easily doable. Something else is amiss
I can check the notebook if you like and can share it. We need to restrict the ITs a bit, this ensures that more people can run ITs in parallel.
This would be awesome - @wee-snufkin can you send him your notebook?
The resources allocated for Jupyter on usegalaxy.org are very very limited (1.5 CPUs, 1.5 GB of memory, and a 12 hour max runtime). I can increase that with some idea of how much might be needed. Like @bgruening mentioned though, this has to be kept intentionally low in order to prevent abuse. I think with some work I can grant increased resources to individual users, however.
Above it's the tutorial with the notebook but for your convenience you can download it directly from here