[Bug] active PR CI is not updated if there's merge event

[Jeffwan]

Search before asking

• [X] I searched the issues and found no similar issues.

KubeRay Component

Others

What happened + What you expected to happen

PR merged and post submit test failed.

Actually, the CI when I merged it is OK.

I think the problem is

Reproduction script

n/a

Anything else

No response

Are you willing to submit a PR?

• [X] Yes I am willing to submit a PR!

[Jeffwan]

This is due to previous PR https://github.com/ray-project/kuberay/pull/416

[Jeffwan]

We should fix the upstream code for short term. /cc @MissiontoMars