edr monitor --teams-webhook giving error

[JyotsnaMalode]

Describe the bug When I execute below command: edr monitor --teams-webhook giving error

Getting below error in Power Automate workflow Oops, we failed to format the alert ! -_-' Please share this with the Elementary team via https://elementary-data.com/community or a https://github.com/elementary-data/elementary/issues/new|GitHub issue."

To Reproduce Steps to reproduce the behavior:

1. Run dbt tests
2. Execute command "edr monitor --teams-webhook "
3. Getting error as below, even though i have ?api-version=2016-06-01& in webhook url

Also getting below error in PowerAutomate

Expected behavior I was expecting to see alerts in Teams channel

Screenshots If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• Elementary CLI (edr) version: [e.g. 0.5.3], can be found by running pip show elementary-data Version: 0.15.1
• Elementary dbt package version: [e.g. 0.4.1], can be found in packages.yml file version: 0.14.1
• dbt version you're using [e.g. 1.8.1] 1.7.10
• Data warehouse [e.g. snowflake] Redshift
• Infrastructure details (e.g. operating system, prod / dev / staging, deployment infra, CI system, etc)

Additional context Add any other context about the problem here.

Would you be willing to contribute a fix for this issue? Let us know if you're willing to contribute so that we'll be able to provide guidance.

[dapollak]

Hey @JyotsnaMalode, How do you pass the webhook url to the CLI ?

[JyotsnaMalode]

hi @dapollak , I am using below command edr monitor --teams-webhook "url"

I am using Teams workflow and not Teams webhook as it will soon be not supported

When I check the raw output for "When a Teams webhook request is received" in Wrokflow, I get below data "title": "Oops, we failed to format the alert ! --' Please share this with the Elementary team via https://elementary-data.com/community or a https://github.com/elementary-data/elementary/issues/new|GitHub issue.",_

[mriveraFacephi]

Hi, same error here. pymsteams appears not to have a solution yet.

A workaround could be sent alerts in plain text and allowing ok status code between 200 <= x < 300. Not the best solution but at least it allows to continue receive alerts.

[JyotsnaMalode]

Thanks @mriveraFacephi , will try your suggestion.

[Jonathan-MW]

I also tried to setup MS Teams alerts with the new Power Automate workflow method, because the connector already does not work anymore for us. Getting this error: "ERROR — Unable to send alert via Teams: 'Response' object has no attribute 'body'"

[JyotsnaMalode]

Update - workaround suggested by @mriveraFacephi worked for me to receive the text message.

[FredRodrigues]

Hey @Jonathan-MW , I am having the same issue. Did you try the solution provided by @mriveraFacephi ?

I tried but does not seem to work :/

[FredRodrigues]

In short, we will need for an update from pymsteams in order to solve this issue or is there any other option we can use to solve this problem?

[FrankTub]

@FredRodrigues I think we could also suggest to the elementary team to start using https://github.com/AeroFlorian/MsTeamsAdaptiveCard/tree/master instead of pymsteams to switch to the new MS Teams way of doing things.

[FrankTub]

@ellakz I just looked into this issue and verified that moving the existing webhooks we have to the new webhooks as defined by MS Teams, but then I also get an error:

ERROR — Unable to send alert fallback via Teams: 'Response' object has no attribute 'body'

When looking at the documentation of MS Teams we will have to migrate to a new solution before 31 januari 2025:

URLs for all webhook-based connectors must be updated by January 31, 2025 for the connectors to continue posting messages into Teams

What do you think about my above suggestion to start using another package to send messages to MS Teams?

[MikaKerman]

Hi @JyotsnaMalode, Thanks for raising the issue!

As the deprecation deadline for Office 365 Connectors approaches, we're working on adding support for sending alerts to Teams via workflows. Workflows only support the new Adaptive Cards format (see the note here), whereas elementary currently uses the older message card format supported by Office 365 Connectors.

Currently, when we can't format the message, we send a fallback message with JSON data to ensure you're notified about the alert. However, this fallback message is also not in the Adaptive Cards format, so it fails in workflows, which is why you see the error in Power Automate. You might also see another error related to the original alert.

Until we roll out support for the Adaptive Cards format, I suggest using the Incoming Webhook application as a temporary solution, if possible. You can pass the URL generated by the Incoming Webhook to edr monitor, and alerts will route correctly.

Regarding the ERROR log, @FrankTub, this was a minor bug in logging the actual error when sending the alert. I've deployed a quick fix for this today, so it should now print a more accurate message.

[FrankTub]

@MikaKerman , can I test this by pinning the version of elementary to use as the master branch?

[FrankTub]

@MikaKerman , I tried to test it out for you but this results in worse behavior in my opinion, see explanation below.

I pinned Elementary to the last commit on master in my requirements.txt by adding it to:

elementary-data @ git+https://github.com/elementary-data/elementary@master

Next I ran something like:

edr monitor --profiles-dir $DBT_PROFILES_DIR --teams-webhook "$MS_TEAMS_DATA_QUALITY_WEBHOOK_URL" --report-url "$DATA_QUALITY_REPORT_URL"

This does not result in an error, but no messages are sent to the Teams webhook:

    ________                          __                  
   / ____/ /__  ____ ___  ___  ____  / /_____ ________  __
  / __/ / / _ \/ __ `__ \/ _ \/ __ \/ __/ __ `/ ___/ / / /
 / /___/ /  __/ / / / / /  __/ / / / /_/ /_/ / /  / /_/ / 
/_____/_/\___/_/ /_/ /_/\___/_/ /_/\__/\__,_/_/   \__, /  
                                                 /____/   

Any feedback and suggestions are welcomed! join our community here - https://bit.ly/slack-elementary

2024-11-25 11:32:52 — INFO — Running with edr=0.16.1
2024-11-25 11:32:54 — INFO — edr (0.16.1) and Elementary's dbt package (0.16.2) are compatible.
2024-11-25 11:32:55 — INFO — Elementary's database and schema: '"staging.elementary"'
2024-11-25 11:32:55 — INFO — Running internal dbt run to populate alerts
2024-11-25 11:32:55 — INFO — Running dbt command run -m elementary_cli.alerts.alerts_v2 --project-dir /Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /Users/frank.tubbing/.dbt --vars {"days_back": 1}
2024-11-25 11:32:58 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_pending_alerts", "macro_args": {"days_back": 1, "type": null}} --project-dir /Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /Users/frank.tubbing/.dbt
2024-11-25 11:33:00 — INFO — Running dbt command --log-format json run-operation elementary.log_macro_results --args {"macro_name": "elementary_cli.get_last_alert_sent_times", "macro_args": {"days_back": 1, "type": null}} --project-dir /Users/frank.tubbing/source/repos/data-warehouse-etl-dbt/venv/lib/python3.11/site-packages/elementary/monitor/dbt_project --profiles-dir /Users/frank.tubbing/.dbt
2024-11-25 11:33:01 — INFO — Update skipped alerts
Sending alerts |████████████████████████████████████████| 20/20 [100%] in 0.0s (872.94/s)
2024-11-25 11:33:01 — INFO — Update sent alerts

[FrankTub]

@MikaKerman , I tried out following some time ago and that works for this new webhook in Teams:

import requests
from adaptive_cards.card import AdaptiveCard
from adaptive_cards.elements import TextBlock, Image
import adaptive_cards.card_types as types

# Replace with your Teams webhook URL
webhook_url = "some-url"

text_block: TextBlock = TextBlock(
    text="It's your third card",
    color=types.Colors.ACCENT,
    size=types.FontSize.EXTRA_LARGE,
    horizontal_alignment=types.HorizontalAlignment.CENTER,
)

image: Image = Image(url="https://adaptivecards.io/content/bf-logo.png")

version: str = "1.5"
card = AdaptiveCard.new() \
        .version(version) \
        .add_items([text_block, image]) \
        .create()

payload = {
    "type": "message",
    "attachments": [
        {
            "contentType": "application/vnd.microsoft.card.adaptive",
            "content": card.to_dict()
        }
    ]
}

response = requests.post(webhook_url, json=payload)

if response.status_code == 200:
    print("Message sent successfully!")
else:
    print(f"Failed to send message. Status code: {response.status_code}, response: {response.text}")

Resulting output in Teams:

Perhaps this can point you in the right direction?

[FredRodrigues]

@FrankTub dont forget the suppression-interval is by default 24hours

edr monitor --profiles-dir $DBT_PROFILES_DIR --teams-webhook "$MS_TEAMS_DATA_QUALITY_WEBHOOK_URL" --report-url "$DATA_QUALITY_REPORT_URL"

I would add --supression-interval 0 and should send messages to teams.

We are currently facing a similar issue, we get the message "ERROR — Unable to send alert via Teams: 'Response' object has no attribute 'body'" but somehow it still sends the message, we are using workflows on teams to parse the message and post the alert on the channel because for our use case we only need the title and description.

[FrankTub]

@FredRodrigues , pretty sure that suppression-interval is irrelevant in my test case for the master branch since I just generated some sample errors that I could verify with edr report. Also you can see that there are 20/20 test results sent according to elementary, however none are received in the Teams channel.

[MikaKerman]

Hi @FrankTub
Thank you for the suggestion!
I'm currently adding support for the adaptive cards format.
We'll update you on how to migrate to the new method.

[MikaKerman]

Hi everyone! We just released version 0.16.2, which utilizes the Adaptive Cards format to send alerts to Microsoft Teams. This update enables users to use both Power Automate workflows and webhook connectors to generate webhook URLs to send alerts. Here's an example of an alert sent by the new package.

⚠ Important Elementary CLI cannot verify message delivery in Power Automate workflows, since the webhook trigger consistently returns a 202 status code. In this case, the CLI will assume success. As shown in the attached screenshot, the first step, "When a Teams webhook request is received," passes and returns a 202 response, despite the failure of the second step. If you choose to use workflows, you will need to monitor your workflow runs in Power Automate to check for any delivery issues.