Lon Blauvelt comments

Results 63 comments of


                                            Lon Blauvelt

Make leader wait for expected updates to be visible in the job store, or fail the job

> @DailyDreaming Yes, it should be. > > Do you want me to always request review from you specifically when something is ready? Usually what I do is mark it...

Replace Mesos-based automatically-deployed clusters and Toil-integrated scaling with Kubernetes-based automatically-deployed clusters running some kind of stock autoscaler

@adamnovak I agree we're getting behind with mesos stuck on ubuntu 16 and coreOS getting dropped. I took a look at Charmed Kubernetes. The interface looks nice. It had a...

Implement wall clock time resource requests in Toil and expose to CWL and WDL jobs

@ielis Unfortunately, that's a scheduler specific param and there isn't a second way of passing it through at the moment. We welcome contributions though if this is something you'd like...

Improve debugging experience

> Maybe we can drop --debugWorker in favor of toil debug-job only? The --debugWorker flag allows debugging in pycharm and with pdb. It's something I often use, particularly with toil-cwl-runner...

Add random exponential backoff to with_retries: Toil goes rogue if with_retries fails, pipeline should exit

The delay and retry makes sense. I'm not sure if adding a failure here would affect other pipelines (@joelarmstrong ?).

Update to Python 3.12

I thought that this was passing, but may have merged it prematurely. It's been reverted from `master` and should be reopened as a PR.

Thread-safe file import on S3

@w-gao @adamnovak Reference is here: https://boto3.amazonaws.com/v1/documentation/api/1.14.31/guide/session.html#multithreading-or-multiprocessing-with-sessions

Refactor Toil to be snake_case while retaining compatibility with Cactus

Wouldn't mind seeing this either, but fairly low priority for us. Will likely just see it slowly added to the codebase over time.

Add `seff` to Toil Slurm?

Toil could maybe have an alert system when resource usage is too high maybe? Making sure that the spurious to useful alerts wouldn't be too high, though this may be...

Update default runner to python 3.7.

@kislyuk This was actually started as a direct result of realizing `python3.6-dev` is no longer a part of the default packages upon upgrading my personal laptop to Ubuntu 19.04. Will...