DistributedScience
Error using `cellpainting-gallery`
I haven't yet tracked down where this problem is coming from as it's new to me and I've had read/write from the cellpainting-gallery working. Receive this error with both cellprofiler/distributed-cellprofiler:2.2.0_4.2.8 and erinweisbart/distributed-cellprofiler:2.2.0rc2_4.2.4.
File "cp-worker.py", line 114, in runCellProfiler
rootlist=os.listdir(DATA_ROOT)
ConnectionAbortedError: [Errno 103] Software caused connection abort: '/home/ubuntu/bucket'
Was using my cpg-access profile which assumes a role with read/write permission on the bucket. (And if I bypass this error with #187 I can confirm that read/write are working).
If I remove line 114 then it tells me it can't find the pipeline file which may not actually be a helpful diagnostic because that could be happening because of improper mount or because of whatever mystery bug 114 was put in place to prevent (the documentation of which seems to have been lost to the sands of time)
Lost to the sands of time maybe, but not my ability to look up random trivial things that don't matter lol
https://github.com/DistributedScience/Distributed-CellProfiler/commit/aab088948eea0da4d46d763b84658a250adb58ff Will also pass you a link in Slack to the associated Slack discussion from 2016.
(but, yes, can't find the pipeline file is exactly the mystery issue from 2016, glad S3FS has never fixed it in 8 years lol. We'd considered removing that line several times in the last several years because we were like "surely this isn't needed anymore" - nope, guess it still is)
Have started troubleshooting.
Breadcrumbs: Created new docker with checkout of s3fs v1.91. It works for CPG with User credentials but not role credentials.
Tried explicitly setting iam_role="auto"
Tried setting based on name of role that is passed to instance iam_role="add-jump-cp-role"
Tried adding flag -o imdsv1only