Constant messages on closed PRs that "release failed to be created"

[dyladan]

You can see an example of the issue here: https://github.com/open-telemetry/opentelemetry-js-contrib/pull/671

We believe we have properly configured release-please and it seems to work ok, but it constantly spams us with messages on old merged PRs that it has failed to create a release. Is there something we've done wrong in our configuration?