alertmanager icon indicating copy to clipboard operation
alertmanager copied to clipboard

Jira Integration: Allow configuring issue update via parameter

Open holger-waschke opened this issue 2 months ago • 15 comments

This change allows disabling the update of issue descriptions.

It´s a non breaking change the default is still to update everything. Only if you set the parameter disable_update_description explicit to true the description won´t get updated anymore.

We need this feature as there a lot of jira automations which updates the description itself, jiralert would revert the changes back to the original templated description state once the repeat_interval kicks in and the alert is still active, so it´s handy to have the possibility to disable it.

I added a new unit test for this function to test.

Exempel config

receivers:
- name: jira_alarm
  jira_configs:
  - project: MON
    issue_type: Bug
    resolve_transition: "Geschlossen"
    reopen_transition: "Erfasst"
    reopen_duration: "1h"
    disable_update_description: true

holger-waschke avatar Oct 21 '25 14:10 holger-waschke

this is also important due to upcoming rate-limiting https://developer.atlassian.com/cloud/jira/platform/rate-limiting/

cjedro avatar Oct 27 '25 19:10 cjedro

this is also important due to upcoming rate-limiting https://developer.atlassian.com/cloud/jira/platform/rate-limiting/

tbh this sounds to me like it will be mandatory to have some kind of back off mechanism to avoid running into these enforced rate limits?

holger-waschke avatar Oct 28 '25 15:10 holger-waschke

@holger-waschke i think this is already implemented https://github.com/prometheus/alertmanager/blob/f7ff687529f86467e11fe1336e56756c0f275bdb/notify/jira/jira.go#L63

cjedro avatar Oct 28 '25 15:10 cjedro

I'm with you, but this handles every alarm separate. So in the worst-case when many alarms are open and firing, the rate-limit could result in no new alarms opened. This could happen because of the push-mechanism that re-sends old alarms for updates.

I think generally an optional circuit-breaker could solve this. Spontaneously I would think about two queues:

  1. more important (e.g. new alarms)
  2. less important (e.g. update/close alarms)

Maybe we should discuss this in an issue.

fluktuid avatar Oct 29 '25 09:10 fluktuid

I like this idea, but I'm wondering if we should have a more generic implementation. I'm less familiar with the Jira integration than I should be, honestly. Are there any other fields which we might want to avoid updating on repeat notifications? I'm wondering if we should allow fields to have sub-keys that indicate if we should update them or not.

I also think the discussion about rate-limiting is important enough to move to a dedicated issue. It's really a problem for any receiver, so it's something we should ideally solve generically.

Spaceman1701 avatar Nov 03 '25 21:11 Spaceman1701

I like this idea, but I'm wondering if we should have a more generic implementation. I'm less familiar with the Jira integration than I should be, honestly. Are there any other fields which we might want to avoid updating on repeat notifications? I'm wondering if we should allow fields to have sub-keys that indicate if we should update them or not.

I also think the discussion about rate-limiting is important enough to move to a dedicated issue. It's really a problem for any receiver, so it's something we should ideally solve generically.

we´re using this Jira API Endpoint to update existing issues https://developer.atlassian.com/cloud/jira/platform/rest/v2/api-group-issues/#api-rest-api-2-issue-issueidorkey-put

In the request body we send the fields we would like to update. which are the same as creating a new issue so basically everything. from my experience what should be configurable (whats most relevant to users) is the summary and description. customfields, priority and other stuff is more static in most cases. (for example customfield hostname or environment will not change)

holger-waschke avatar Nov 04 '25 12:11 holger-waschke

from my experience what should be configurable (whats most relevant to users) is the summary and description. customfields, priority and other stuff is more static in most cases. (for example customfield hostname or environment will not change)

That makes sense to me... The thing I'm worried about is that we'll end up having multiple disable_update_* keys floating around (which seems a little ugly to me).

What do you think about introducing a new JiraField type in the config that we can either deserialize from string (to maintain backwards compatibility) or from an object with keys that control updating? It's a bit more complexity, but I think it's a bit cleaner in the long run.

This would result in config like:

receivers:
- name: jira_alarm
  jira_configs:
  - project: MON
    issue_type: Bug
    resolve_transition: "Geschlossen"
    reopen_transition: "Erfasst"
    reopen_duration: "1h"
    description:
      disable_update: true

We could then reuse the type for summary or any other field that might end up needing this behavior.

cc @siavashs and @grobinson-grafana since they might have a different opinion.

Spaceman1701 avatar Nov 04 '25 16:11 Spaceman1701

@Spaceman1701 I implemented your suggestion. The unmarshalling remains backward-compatible, so nothing breaks — both formats are now supported:

# Old/legacy format (just a string)
summary: '{{ template "jira.default.summary" . }}'
description: '{{ template "jira.default.description" . }}'
# New format (object with fields)
summary:
  template: '{{ template "jira.default.summary" . }}'
  disable_update: true
description:
  template: '{{ template "jira.default.description" . }}'
  disable_update: true

The jira_test required a fair amount of refactoring to accommodate this, but it feels justified.

holger-waschke avatar Nov 14 '25 20:11 holger-waschke

Built in more improvements

  • improved log output when updating existing issues debug output will now print out which labels are being skipped, e.g.

level=DEBUG source=jira.go:119 msg="updating existing issue without summary and description" integration=jira group_key="{}/{test="true"}:{alertname="E2E Failed Application", environment="Abnahme"}" issue_key=ASPMIG-447810

  • improved JiraFieldConfig unmarshaling to respect the template field default. Setting summary and description is fully optional. you can have it like this or leave it full out.
- name: jira_asp_alarm
  jira_configs:
  - project: ASPMIG
    issue_type: Alarm
    summary:
      template: '{{ template "jira.default.summary" . }}'
    description:
      disable_update: true

new logic for the atlassian v3 api endpoint. (which uses atlassian document format). I had to rewrite this because of the string pointer anyway so It´s optimized for only check for valid JSON. The actual encoding takes place in the doAPIRequestFullPath function anyway.

	descriptionCopy := issueDescriptionString
	if isAPIv3Path(n.conf.APIURL.Path) {
		if !json.Valid([]byte(descriptionCopy)) {
			return issue{}, fmt.Errorf("description template: invalid JSON for API v3")
		}
	}
	requestBody.Fields.Description = &descriptionCopy

However, v3 endpoint with atlassian document format is broken atm, but this is another issue I´ll reply to https://github.com/prometheus/alertmanager/issues/4585. This needs quite some refactoring, too.

holger-waschke avatar Nov 17 '25 14:11 holger-waschke