apache
Updated logic to allow AWS Batch Hook `get_job_description` retries to be more effective
Update logic in Amazon/aws providers for batch_client hook's parse_job_description function. This function is ONLY called within the get_job_description function that allows for retries; however, there is only handling of botocore.exceptions.ClientError error in this function.
This means, that if AWS simply fails to provide the job description - if there is a delay in AWS to register job information, this function will immediate fail instead of trying. This may happen under certain conditions, especially when using the batch operators for submitting jobs which immediately use returned job_id to query for the job information - but again, if AWS fails to register the job description in time - this will cause an unnecessary failure in Airflow side of things.
The proposed change allows parse_job_description to return None instead of raising if zero jobs are returned by AWS. Similarly, if more than one matching job is returned, the function will provide the description of the first job.
