incubator-devlake icon indicating copy to clipboard operation
incubator-devlake copied to clipboard

Published 20 hours ago • verified publisher icon apache

[Feature][Azure devops] Collect Boards from Azure DevOps to supply incidents data for DORA

Open klemen-df opened this issue 1 year ago • 23 comments

Search before asking

  • [X] I had searched in the issues and found no similar feature requirement.

Use case

We use Azure repos/boards/pipelines.

The documentation says:

DORA - Median Time to Restore Service
What is this metric?
The time to restore service after service incidents, rollbacks, or any type of production failure happened.

and

DORA - Change Failure Rate
What is this metric?
The percentage of changes that were made to a code that then resulted in incidents, rollbacks, or any type of production failure.

Our 'outage' of services can be seen in our board by viewing tickets with type 'Bug'. There's no other way to report an 'INCIDENT' in Azure devops board. When something is wrong you open a ticket with type='Bug'. You can also set priority (like Critical) and add some labels (like production).

I wish to see DORA metrics in Grafana ; 'Median time to restore service' and 'Change failure rate'.

Description

This should be possible by creating a different 'scrapper' for Azure Devops.

Related issues

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

klemen-df avatar Aug 31 '23 08:08 klemen-df

@keon94 I think we need to collect AzDo tickes/issues data for those metrics to work. Would you like to work on it? Thanks.

klesh avatar Sep 01 '23 02:09 klesh

@KlemenDanfoss Hi, the Azure Devops plugin is written in Python, I would like to ask if you are interested in contributing such a feature? @keon94 could help if you have any questions.

klesh avatar Sep 01 '23 02:09 klesh

@KlemenDanfoss Hi, the Azure Devops plugin is written in Python, I would like to ask if you are interested in contributing such a feature? @keon94 could help if you have any questions.

Sorry, probably can't help with a PR

klemen-df avatar Sep 01 '23 06:09 klemen-df

Alright, let's collect votes first. cc. @keon94

klesh avatar Sep 01 '23 07:09 klesh

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Nov 01 '23 00:11 github-actions[bot]

Anything new here?

klemen-df avatar Nov 01 '23 08:11 klemen-df

@KlemenDanfoss No progress yet. You can try to share it with the Slack community and see if anyone is interested in it. Azure DevOps is implemented with Python btw.

Startrekzky avatar Nov 07 '23 03:11 Startrekzky

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Jan 07 '24 00:01 github-actions[bot]

Any thing new here? Is this really a feature that doesn't make sense?

klemen-df avatar Jan 09 '24 07:01 klemen-df

Hi @KlemenDanfoss , as a feature it makes sense. However, we don't have the bandwidth to do this recently.

Startrekzky avatar Jan 09 '24 11:01 Startrekzky

We would need this feature as well. Especially the Change Failure Rate can't be only calculated based on Health Checks or Automated Post Deployment Tests. Sometimes there are bugs which are not covered by this automated tests and a Bug is then created.

  • Is it possible to manually mark a deployment as a failure?
  • Probably the same code which exists for gitlab or other providers can be used, just the fetching of the items needs to be adapted based on Azure Devops REST Apis?

edolic avatar Feb 13 '24 09:02 edolic

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

github-actions[bot] avatar Apr 29 '24 00:04 github-actions[bot]

This issue has been closed because it has been inactive for a long time. You can reopen it if you encounter the similar problem in the future.

github-actions[bot] avatar May 06 '24 00:05 github-actions[bot]

Any chance we can reopen this? If given directions, I can submit a PR for this.

slaterx avatar May 30 '24 22:05 slaterx

@slaterx This issue is reopened. There are two implementations about AzureDevOps in DevLake so far, but they both don't collect incidents. I think it is similar to jira. step1: issues in Azure DevOps' board should be collected step2: make a rule about how to regard issues as incidents. (I think @Startrekzky can help.) step3: Update dora and AzureDevOps dashboard if neccessary.

cc @klesh

d4x1 avatar May 31 '24 09:05 d4x1

Hi @d4x1, I've reviewed the repo and found out that we have two plugins - one in python and another one in go.

Based on the comment here, the Go plugin skips disabled git repos, but in Azure DevOps, it is a valid use case to have disabled repos hosting work items. I would like some guidance to understand if I can remove this skip in the Go repo, otherwise, I'll attempt the implementation in the python plugin.

The reason for my question is because we have migrated from Azure DevOps to GitHub, however the issues are still collected in the disabled Azure DevOps repos. Keen to hear your thoughts.

cc @klesh

slaterx avatar Jun 06 '24 23:06 slaterx

@slaterx Your use case sounds reasonable to me. @mr-ks I think we should allow adding disabled repos in case like this.

klesh avatar Jun 07 '24 09:06 klesh

Once #7809 is merged, just tweak the query in grafana to obtain all bug type work items from Azure DevOps.

slaterx avatar Aug 01 '24 02:08 slaterx

@mr-ks Would you like to take a look at the implementation? Thanks in advance.

klesh avatar Aug 01 '24 03:08 klesh