Notifiarr Integration

[hvmzx]

Is your feature request related to a problem? Please describe.

I use Notifiarr which manages sending notifications to Discord, I would like to know if it is possible integrating Notifiarr in the notification services.

(This would be used by many people as I saw that being requested by many on Discord)

Notifiarr Wiki : https://notifiarr.wiki/

Edit : After making some tests to see what was received to a custom WATCHTOWER_NOTIFICATION_URL, no JSON is received somehow and that would be what is needed to connect notifiarr to watchtower

Describe the solution you'd like

Integration of Notifiarr into Watchtower

Describe alternatives you've considered

I have used Pullio which handles Notifiarr for now but would gladly try Watchtower if it's supported also

Additional context

No response

[github-actions[bot]]

Hi there! 👋🏼 As you're new to this repo, we'd like to suggest that you read our code of conduct as well as our contribution guidelines. Thanks a bunch for opening your first issue! 🙏

[austinwbest]

Although the apikey being sent via a header would be ideal (i dont think there is a way to send custom headers with the service), I can make it work with a generic webhook if it could be figured out how that actually works. I would assume you just put in a webhook url such as generic+url or generic://url and watchtower would send a proper json payload to shoutrr which would then pass that along to the generic url but that doesn't seem to be the case. There is no json or post data received.

They tried adding &template=json which didnt help, tried using a different content type, didnt do anything different.

[piksel]

The payload for generic is just the message, there are no special templates built in. I can probably take a stab at implementing support for Notifiarr, but I didn't see anything in the wiki about how to integrate with it...

[austinwbest]

The payload for generic is just the message, there are no special templates built in. I can probably take a stab at implementing support for Notifiarr, but I didn't see anything in the wiki about how to integrate with it...

To send just some test payloads with no apikey required, basically for me to review the payloads:

https://dev.notifiarr.app/api/v1/notification/test?event=Watchtower

To send real payloads to users (once i have some example payloads and can implement it fully):

Header apikey (preferred): https://notifiarr.com/api/v1/notification/watchtower Header X-API-Key: <notifiarr-api-key-here>

URL apikey (less preferred): https://notifiarr.com/api/v1/notification/watchtower/<notifiarr-api-key-here>

You can test with 2bdb19df-43f2-4cd2-8e20-bd752e77c3d6 as the notifiarr api key to get responses

All of them preferable to be json payloads with the json header which i think is the default anyways Some examples could be:

{
    event: "update_started",
    container: "some-container-name",
    oldVersion: "0.0.1",
    newVersion: "0.0.2",
    ... whatever ...
}

{
    event: "update_failed",
    container: "some-container-name",
    oldVersion: "0.0.1",
    newVersion: "0.0.2",
    ... whatever ...
}

{
    event: "update_completed",
    container: "some-container-name",
    oldVersion: "0.0.1",
    newVersion: "0.0.2",
    ... whatever ...
}

What is in the payload is not set in stone. You can make it whatever keys, objects, types, etc and i will accomodate for what is there. The more info the better as thay can be made available to users!

The test endpoint will always return a 200 and just consumes the payloads for review The real endpoint can return a 400, 401 or 200 with a reason for the error if it is a non 200

Below is a couple example error responses (details.response is likely all that matters here)

{
    "code": 400,
    "result": "error",
    "details": {
        "environment": "NIGHTLY",
        "code": 400,
        "response": "no channel picked",
        "help": "https://discord.gg/AURf8Yz",
        "apikey": "2bdb......d6",
        "version": "v1",
        "route": "notification",
        "path": "watchtower",
        "parameters": []
    }
}

{
    "code": 401,
    "result": "error",
    "details": {
        "environment": "NIGHTLY",
        "code": 401,
        "response": "Missing integration specific key access",
        "help": "https://discord.gg/AURf8Yz",
        "apikey": "2bdb......d6",
        "version": "v1",
        "route": "notification",
        "path": "watchtower",
        "parameters": []
    }
}

[piksel]

@austinwbest, sorry about the confusion around template=json, it has indeed been added, but you need to use the generic://dev.notifiarr.app/api/v1/notification/test?event=Watchtower&template=json (and not generic+https://, since that just sends all query parameters through to the target).

The payload is just {"title": "...", "body": "..."} though, which isn't really helpful. There is planned support for a generic json template for watchtower though...

[austinwbest]

@austinwbest, sorry about the confusion around template=json, it has indeed been added, but you need to use the generic://dev.notifiarr.app/api/v1/notification/test?event=Watchtower&template=json (and not generic+https://, since that just sends all query parameters through to the target).

The payload is just {"title": "...", "body": "..."} though, which isn't really helpful. There is planned support for a generic json template for watchtower though...

I am fairly certain we tried this and never got a proper JSON payload. I do recall seeing a content-length header of like 2k chars which made me think there should have been one but there was no JSON captured from it (it wasnt in a POST variable and wasnt in a php://input with a json header or something, not really sure what)

I even retested with some other integrations to make sure i didn't break the test endpoint and it captured their JSON payloads fine so i really dont think it is in how things are being captured unfortunately. This endpoint has worked to get nearly 40 integrations stood up and running so far.

Just to be sure we have a couple users adding the URL you provided to theirs and we can try it again for sure.

[austinwbest]

Ok, so we where able to get a payload to come in but after working with Pullio for so long i am curious about the decision for this content. It does not seem very useful.

- /jellyfin (jellyfin/jellyfin:latest): 35440e4297cb updated to 9c03191ab57d

Why would a hash be desired over a version? Maybe add both (yes it is messy with this everything in one message key scenario)?

- /jellyfin (jellyfin/jellyfin:latest): v0.0.1 (35440e4297cb) updated to v0.0.2 (9c03191ab57d)

What is the purpose of the hash in the title? Do people really memorize hashes that constantly change? Im not a docker user so just looking for clarification is all

Watchtower updates on 858fbb1b407c

[piksel]

Why would a hash be desired over a version?

How would that even work? An image tag has no concept of a version. There is nothing that identifies what the version of some software running inside the image is.

What is the purpose of the hash in the title? Do people really memorize hashes that constantly change?

That is the hostname of the container, which is set to the container hash if not specified and host networking isn't used (the default for docker run, are you using compose mayhaps?)

[austinwbest]

Why would a hash be desired over a version?

How would that even work? An image tag has no concept of a version. There is nothing that identifies what the version of some software running inside the image is.

What is the purpose of the hash in the title? Do people really memorize hashes that constantly change?

That is the hostname of the container, which is set to the container hash if not specified and host networking isn't used (the default for docker run, are you using compose mayhaps?)

I have zero clue about docker and things like that. I just know Pullio does is all i was going by lol

This is what ive been able to come up with for Watchtower thus far. Green would be all success, yellow is a mix of success and fail, red would be all fails. I dont have any example failures so i am not even sure what one looks like but i just guess there is some kinda reason added to it.

[piksel]

Just as a heads-up, this is how the report looks using the prototype JSON template:

Message sent after a "session":

{
  "entries": [
    {
      "data": {"tagged_data_key": "tagged_data_value"},
      "level": "info",
      "message": "Log message outputted during run",
      "time": "2023-01-23T10:10:18.575980739+01:00"
    },
  ],
  "host": "Ketsueki",
  "report": {
    "failed": [],
    "fresh": [
      {
        "currentImageId": "sha256:d0d39a358e4860804efc78cd5320845e947522ddbd7438b63ddeea9a66d1ea02",
        "error": null,
        "id": "0d4e9b7de7053d524b664fcba136eefc5f3ce92d7ca26162ae092fe626ebcbff",
        "imageName": "koenkk/zigbee2mqtt:latest",
        "latestImageId": "sha256:d0d39a358e4860804efc78cd5320845e947522ddbd7438b63ddeea9a66d1ea02",
        "name": "/wt-test-1",
        "state": "Fresh"
      },
      {
        "currentImageId": "sha256:e982339a20a53052bd5f2b2e8438b3c95c91013f653ee781a67934cd1f9f9631",
        "error": null,
        "id": "e4b0a3a886d80a74c1185e7e24f7f3af0bb2f5075c8a4dbd24cd0a27075385aa",
        "imageName": "mysql:5.7",
        "latestImageId": "sha256:e982339a20a53052bd5f2b2e8438b3c95c91013f653ee781a67934cd1f9f9631",
        "name": "/events.mysql",
        "state": "Fresh"
      }
    ],
    "scanned": [
      {
        "currentImageId": "sha256:d0d39a358e4860804efc78cd5320845e947522ddbd7438b63ddeea9a66d1ea02",
        "error": null,
        "id": "0d4e9b7de7053d524b664fcba136eefc5f3ce92d7ca26162ae092fe626ebcbff",
        "imageName": "koenkk/zigbee2mqtt:latest",
        "latestImageId": "sha256:d0d39a358e4860804efc78cd5320845e947522ddbd7438b63ddeea9a66d1ea02",
        "name": "/wt-test-1",
        "state": "Fresh"
      },
      {
        "currentImageId": "sha256:e982339a20a53052bd5f2b2e8438b3c95c91013f653ee781a67934cd1f9f9631",
        "error": null,
        "id": "e4b0a3a886d80a74c1185e7e24f7f3af0bb2f5075c8a4dbd24cd0a27075385aa",
        "imageName": "mysql:5.7",
        "latestImageId": "sha256:e982339a20a53052bd5f2b2e8438b3c95c91013f653ee781a67934cd1f9f9631",
        "name": "/events.mysql",
        "state": "Fresh"
      }
    ],
    "skipped": [],
    "stale": [],
    "updated": []
  },
  "title": "Watchtower updates on Ketsueki"
}

Message sent for any logs outside of a session (mainly on startup/exit):

{
  "entries": [
    {
      "data": {},
      "level": "info",
      "message": "Watchtower v0.0.0-unknown",
      "time": "2023-01-23T10:10:18.575980739+01:00"
    },
    {
      "data": {},
      "level": "info",
      "message": "Using notifications: logger",
      "time": "2023-01-23T10:10:18.576043196+01:00"
    },
    {
      "data": {},
      "level": "info",
      "message": "Checking all containers (except explicitly disabled with label)",
      "time": "2023-01-23T10:10:18.576046563+01:00"
    },
    {
      "data": {},
      "level": "info",
      "message": "Running a one time update.",
      "time": "2023-01-23T10:10:18.57605043+01:00"
    }
  ],
  "host": "Ketsueki",
  "report": null,
  "title": "Watchtower updates on Ketsueki"
}

So, basically, the report is only populated when a update session is done, and entries are all the log entries during that session. If something is logged outside sessions, that will be sent using the same structure, but report will be null.

I dont have any example failures so i am not even sure what one looks like but i just guess there is some kinda reason added to it.

Yeah, the report.failed.[].error includes it, and in the default "human-readable" template it's added after the container info: https://github.com/containrrr/watchtower/blob/main/pkg/notifications/common_templates.go#L21

[austinwbest]

Sounds good. Ive got it working with generic and using the apikey in the URL (although i would prefer a header for that) but can make changes whenever pretty quickly now with it all in place.

[austinwbest]

Just as a heads-up, this is how the report looks using the prototype JSON template:

Did something change for the webhooks?

The content used to start with 33 Scanned, 5 Updated, 2 Failed..... but getting reports of notifications not working anymore so when i looked at log files it appears that line is not even there anymore and the line now starts with Found new .... and does not contain the counters.

I confirmed the ?template=json was in the URL and it was generic so im not sure.

[piksel]

That sounds like --notification-report is missing. Then it only sends the log messages instead of the summary ("report").

[piksel]

@austinwbest the structured JSON format is now available in watchtower if you wanted to add an integration.

Instead of the ?template=json suffix in the config URL, you set the watchtower template to just json.v1:

WATCHTOWER_NOTIFICATION_TEMPLATE: json.v1

[austinwbest]

@austinwbest the structured JSON format is now available in watchtower if you wanted to add an integration.

Instead of the ?template=json suffix in the config URL, you set the watchtower template to just json.v1:

WATCHTOWER_NOTIFICATION_TEMPLATE: json.v1

Ok, thanks for the update, I'll update the instructions. Currently i am trying to work with another user who simply cant get notifications to work at all when using monitor only (even with the template being the one that says it will notify for every run). I think there is a discussion going on for it as well.