spitz-dan-l

(I'm on redun 0.8.6, so apologies if this is fixed in a more recent version.) After running a task that took 27 hours to complete, redun errors with `psycopg2.OperationalError: SSL...

Currently script tasks cannot use code packaging. This would be useful for a variety of purposes, including running non-python code snippets that may import user-authored source code. To do it,...

Task-specific code packaging

1

Hey there redun team! This one is a bit of a corner case. I would like to disable code packaging for a particular task in a pipeline, but keep it...

Cleaning up Batch jobs on unexpected termination

1

If a keyboard interrupt halts an in-progress redun execution, any in-flight AWS Batch jobs will keep on running after redun has exited. Ideally those jobs would be cancelled before redun...

Hey there Redun team! This is a scaling issue that we've run into on my team. When running a pipeline in which multiple tasks return large python objects, the redun...

Greetings @mattrasmus and team! This is a preliminary PR with an executor for running tasks within local conda environments. I'll be curious to hear your feedback on it! ## Overview...

For script tasks, `tempdir=True` with remote executor produces an error

2

Hi there! The following task: ```python @task() def main(): return script('echo HELLO', tempdir=True, executor='batch') ``` Produces this output: ``` Traceback (most recent call last): File "***/bin/redun", line 11, in client.execute()...

This is changes how redun scripts create and manage the optional temp dir. Previously the temp dir was created and later removed by the "host" machine. This led to errors...

Code packaging files in parent directories

2

I have a codebase that is structured as follows: ``` /my_project ├── __init__.py ├── library │ ├── __init__.py │ └── library_1.py └── workflows ├── __init__.py ├── workflow_1 │ ├── .redun...