Alberto Alonso

slow / memory loading?

17

Trying to test the progress bar as per the zone10 sample in the homepage with a 1.7GB file. Unfortunately I never see the progress bar, but I see the logs...

For a large number of jobs, scheduling can be pretty slow. This is specifically true in AWS batch, where it can take several seconds per queued job.

We should allow running with remote configurations.

In hydra/cloud/aws_platform.py there are hard coded values, but they should be variables defined by the user. ``` self.batch = boto3.client( service_name='batch', region_name='us-east-1', endpoint_url='https://batch.us-east-1.amazonaws.com' ) ```