Affinity group creation sometimes fails due to race condition

[mrog]

ISSUE TYPE

• Bug Report

COMPONENT NAME

API

CLOUDSTACK VERSION

4.14.1.0

CONFIGURATION

N/A

OS / ENVIRONMENT

CentOS 7, but this is likely N/A.

SUMMARY

When using the API to create affinity groups, it usually works. On rare instances, a race condition causes a NullPointerException.

STEPS TO REPRODUCE

Use the API to create an affinity group.  This may take many tries to reproduce the issue.

EXPECTED RESULTS

The affinity group should be created.

ACTUAL RESULTS

There is sometimes a NullPointerException.

See the attached log. It shows that an async job is created and runs. The output from the async job includes the message Update db status for job-131043, which indicates that the transaction has started which will set the removed date for the async job. Then when ApiServer.getBaseAsyncCreateResponse() looks for the job on line 662, it gets a null value because the record's removed value isn't null. This causes the NPE to happen on line 663.

NullPointerException.log