Distributed-CellProfiler
Distributed-CellProfiler copied to clipboard
DistributedScience
Run encapsulated docker containers with CellProfiler in the Amazon Web Services infrastructure.
Until we have DCP in a workflow system (e.g. AUSPICES) might be nice to have some extra logging to S3 since Cloudwatch logs are automatically cleared out after some number...
If your dockers run out of memory jobs fail silently. It's annoying. Our per-instance logs do regularly print instance metrics that include memory in use and memory available metrics. However,...
``` Request in process. Wait until your machines are available in the cluster. SpotFleetRequestId sfr-e02926f4-4567-4670-b92a-036aefc0cdc2 Updating service Service updated. . . . Your spot fleet request is causing an error...
Have not explored if this is possible. Use case its when there are permission issues that prevent mounting of the bucket.
To better explain `# PLUGINS` in `config.py` Add: - minimum DCP version - more on what each flag does and why you would set each one to True or False...
AWS has changed their role creation thing AGAIN, so setup is now harder than it was. They do now offer "use cases" though, one of which is EC2 - Fleet,...
``` /home/ubuntu/.pyenv/versions/3.8.10/lib/python3.8/site-packages/botocore/client.py:625: FutureWarning: The sqs client is currently using a deprecated endpoint: queue.amazonaws.com. In the next minor version this will be moved to sqs.us-east-1.amazonaws.com. See https://github.com/boto/botocore/issues/2705 for more details ```
Add optional functionality to enable CloudWatch Agent on the EC2 instances Downside is that agent incurs additional costs. But if we keep it as optional and suggest default of False...
Metadata
Owner
Metadata
Run encapsulated docker containers with CellProfiler in the Amazon Web Services infrastructure.