Pete Walsh

Support "baskets": long term scheduler backends

2

The current system (and essentially how Python Celery does it) for handling tasks with a far-off ETA is to have workers consume them as usual but use the `delay_for` async...

Support workflow primitives

1

https://docs.celeryproject.org/en/latest/getting-started/next-steps.html#the-primitives

Add SmBoP to demo

3

From @OhadRubin's work https://github.com/OhadRubin/SmBop, see the discussion [here](https://github.com/allenai/allennlp-gallery/issues/40) and the Google Colab demo [here](https://colab.research.google.com/drive/1KGlETGn9wngUPQrkFfa7ySecU-t_I3Y2#scrollTo=X1v6F3TlOMKH). This is a SQL semantic parsing demo where the user picks out of a dropdown menu...

See this discussion: https://github.com/allenai/allennlp-demo/pull/423#discussion_r423869374

Allowing indexing into step results in config

1

For example, ref steps can now look like `{type: "ref", ref: some_step, key: 0}` Closes #334

Or, as @dirkgr suggested, we could just not stream the logs. But I think we should merge this either way so that we have this fix *for the record*.

### 🚀 The feature, motivation and pitch We're already using `rich`, and our `tqdm` story is kind of a mess at the moment. cc @dirkgr We did this in `cached_path`...

### 🚀 The feature, motivation and pitch PyTorch's native FSDP (adapted from FairScale) is [now in beta](https://pytorch.org/blog/pytorch-1.12-released/#beta-fully-sharded-data-parallel-fsdp-api) as of v1.12. We should support it in our `TorchTrainStep`. ### Alternatives Don't...