David Eliahu

### Description [AWS's docs](https://docs.aws.amazon.com/autoscaling/ec2/userguide/capacity-rebalance.html), [blog 1](https://aws.amazon.com/blogs/compute/proactively-manage-spot-instance-lifecycle-using-the-new-capacity-rebalancing-feature-for-ec2-auto-scaling/), [blog 2](https://aws.amazon.com/about-aws/whats-new/2020/11/amazon-ec2-auto-scaling-ec2-fleet-and-spot-fleet-announce-capacity-rebalancing-for-ec2-spot-instances/) ### Additional context This is supported by eksctl Related to https://github.com/cortexlabs/cortex/issues/2185 (since it is recommended to use the capacity-optimized allocation strategy with...

### Description [Info](https://aws.amazon.com/about-aws/whats-new/2019/08/new-capacity-optimized-allocation-strategy-for-provisioning-amazon-ec2-spot-instances/) (with additional links at the bottom) ### Additional context This is supported by eksctl

### Description Consider using [NLB IP mode](https://kubernetes-sigs.github.io/aws-load-balancer-controller/guide/service/nlb_ip_mode/) with `externalTrafficPolicy: Local`. This would reduce an extra network hop. Related tickets: * https://github.com/kubernetes/ingress-nginx/issues/6828 * https://github.com/kubernetes/cloud-provider-aws/issues/87

#### Implementation notes * It will require installing the [AMD GPU device plugin](https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/#deploying-amd-gpu-device-plugin). * The user will need specify that they are requesting AMD gpus (e.g. `gpu_amd` in the `compute`...

### Description Allow (or support out-of-the-box) the ability to retain metrics data from Prometheus for long periods of time. It would probably be best to make the retention period configurable....

### Implementation notes This could be achieved by adding a field to the TaskAPI (e.g. `cron`), which would result in a task being submitted on the specified schedule. Another alternative...

### Notes We currently use `gorilla/handlers` for CORS, which does not support `*` in their `handlers.AllowedHeaders()` function. Therefore, if the user sets a custom header in their request, the async-gateway...

#### Description If configured, when a request is taken off of the queue for processing, if the time spent on the queue is greater than the configured timeout, the request...