incubator-devlake
incubator-devlake copied to clipboard
apache
[Feature][Azure Devops] Investigate Azure Devops plugin
Search before asking
- [X] I had searched in the issues and found no similar feature requirement.
Description
According to issue https://github.com/apache/incubator-devlake/issues/2537 , we need to investigate Azure Devops and write a table schema in this issue.
Use case
We need to investigate Azure Devops first, and try to implement collector and extractor. Then we can compare several CI/CD plugins' tool layers and go back to work on https://github.com/apache/incubator-devlake/issues/2537
Related issues
No response
Are you willing to submit a PR?
- [ ] Yes I am willing to submit a PR!
Code of Conduct
- [X] I agree to follow this project's Code of Conduct
This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.
I would like my team to adopt Apache Dev Lake but we are using Azure Devops (yeah, I know, sighs), has this been de-scoped?
Hi @mxwlf, we have a very basic plugin for Azure DevOps that only collects Repo and Build data right now. It's missing a lot of entities and not ready for end users yet. Do you think you could describe your use cases (e.g. what metrics/dashboards you're looking to set up) so that we can look into what needs to be done for the plugin? We would love to find a way to support your team.
Hi! @hezyin I'm the same situation as @mxwlf. Our use cases would be having reporting from all of our ADO repositories on each of the four the dora metrics and having the reports be visible on a dashboard. Would you have some rough time estimate on when the plugin would be ready for end user use? Thanks!
Hi @Sarakanerva, to track DORA metrics, DevLake would need three key entities: code changes (usually in the form of pull requests), deployments (from CI/CD systems), and incidents (from issue trackers or dedicated incident management systems). Do you think you could describe what DevOps tools you’re using internally for these entities? With this information, I can evaluate what needs to be done on the DevLake side to help you implement DORA metrics.
And if it makes communication easier, feel free to join our Slack channel and ping me there: https://join.slack.com/t/devlake-io/shared_invite/zt-17b6vuvps-x98pqseoUagM7EAmKC82xQ
Hi @hezyin sure I can take a look at that plugin and as well as @Sarakanerva I'm looking to implement DORA metrics in our project. I will take a look and even see if I can contribute with something. On my side we are using Azure DevOps as code repo and Jira for issue management, so we are halfway to be feasible to implement DevLake. Will join your Slack channel as well but I may be more active until next year. Thanks!
Appreciate it @mxwlf! Do you also track incidents entity with Jira? With multiple users requesting Azure DevOps plugin, we plan to add this plugin to our v0.16 release (around Feb 2023). It would be very helpful if you could pilot this plugin once its beta version is ready.
Let me know when you join Slack, I'll create a channel for Azure DevOps plugin users.
Have a wonderful holiday season!
Hi @hezyin, we are also very interested in this integration. like @mxwlf We use Azure DevOps (Git) as code repository, Azure Pipelines for CI/CD and jira for issue management.
We'll look into developing this plugin using the new Python SDK being developed here: https://github.com/apache/incubator-devlake/issues/3701
@KasperSJensen At the moment yes. Will update this thread if there are changes to timelines.
Hi @KasperSJensen @mxwlf . I'm currently trying to write the logic that is needed for DORA metric collection, and would appreciate your input as we are not very familiar with Azure Devops. Azure treats Pipeline builds and Release deployments as two separate entities, both UI-wise and on the API side. My question is, for the sake of DORA, should we be targeting only release deployments in our data collections, or can pipeline builds also potentially trigger deployments (perhaps not by Azure standards per se)? cc @hezyin
... My question is, for the sake of DORA, should we be targeting only release deployments in our data collections, or can pipeline builds also potentially trigger deployments (perhaps not by Azure standards per se)? cc @hezyin
I can answer that one, as @KasperSJensen and I are working on the same.
Azure Devops has two ways of deploying. The "classic"/legacy way where release is its own entity. And the newer multi-stage yaml-pipelines. Here's a SO thread discussing the two.
Ideally both should be supported by DevLake, but yaml-pipelines should have priority as that is the way going forward.
Actually, Github Actions is forked from Azure Pipelines (yaml-pipelines) so perhaps the implementation can be based on that.
@AThomsen Okay, that's good to know. The yaml-pipelines scheme is what we are working on supporting for now. Another question: Do you run pipelines for "non-Azure repos"? e.g. Github, Bitbucket, etc. cc @KasperSJensen
Thank you all for helping to developer this feature. It means alot for us and we are looking so much forward to the release so we can test it
Likewise, would be very interested in seeing this land as we use Azure Devops heavily.
This issue has been finished by #4450 and #4504 , which is released in v0.17 beta versions.
Can i use apache devlake for taking DORA Metrics from release pipelines ? Specifically production. ?
Can i use apache devlake for taking DORA Metrics from release pipelines ? Specifically production. ?
@Startrekzky Please have a look at this.
Hi @DimDob , DevLake allows you to configure the production deployments via regex in the scope config of Azure. Please check this doc for details https://devlake.apache.org/docs/Configuration/AzureDevOps#step-13---add-scope-config-optional .
Is there a specific reason for using Python for the Azure DevOps plugin? Also, would you consider a rewrite in Go? I'm eager to contribute.
When we first wrote it we wanted to leverage the Python framework we had developed and have a "proof of concept" that a plugin could be done in Python. That said, there are hardly any advantages to having it in Python, and it all it's doing is complicating the tech stack and logic. Not to mention, it's limited in DB migrations (with ugly workarounds), and the framework lacks support of certain important features we have in Go such as incremental collections. We could implement such things with quite a bit of effort but then the question is exactly why? We'll just end up with duplicated code across two different languages, a lot of extra maintenance; the gain here is both little and unclear to me.
So, IMHO, I personally would support a rewrite in Go. I think if we want to insist on supporting Python it should be reserved for simple plugins, and Azure Devops is NOT one of them.
@keon94 Thanks for the input. I personally agree with you. But I would like to add that one of the reasons to write it in Python is that we believed Python is one of the most popular languages, supporting writing plugin in Python could help Apache DevLake grows.
@mr-ks I have a couple of questions after talking to some of our PPMC:
- Why would you want to contribute a Golang version of it?
- Do you need the AzDo in your work?
- Is it too slow or lack of features to fit your need?
- Are you willing to maintain it in the near feature?
@klesh Thanks for considering my input.
We are using Azure DevOps at work. Currently, the plugin only supports Repos and Pipelines and I am looking to also integrate Boards (Sprints and Work Items). For this, I‘d rather use Go than Python. Yes, I‘d like to maintain it in the near feature.
Thanks for the quick reply and the offering. LGTM. @Startrekzky @hezyin @keon94 What do you think?