How to prevent an API from being promoted across environments

[fabienmazieres]

Please describe the feature.

Our developers have some API they are using in Dev that should not progress to other environments and I am wondering what will be the best method to prevent (or at least make unusable) the deployment of specific API.

I was thinking to have the API being deployed but the configuration file for prod is sending the traffic to a dummy URL (i.e. https://apimblock.mycompany.com/):

configuration.prod.yaml:

apis:
  - name: api-for-dev-only
    properties:
      serviceUrl: "https://apimblock.mycompany.com"

but that feels hacky and I wish there could be a flag to specifically prevent an API to be deployed, such as: configuration.prod.yaml:

apis:
  - name: api-for-dev-only
    do-not-deploy: true

Any other way this can be achieved without a new flag in the config?

[github-actions[bot]]

  Thank you for opening this issue! Please be patient while we will look into it and get back to you as this is an open source project. In the meantime make sure you take a look at the [closed issues](https://github.com/Azure/apiops/issues?q=is%3Aissue+is%3Aclosed) in case your question has already been answered. Don't forget to provide any additional information if needed (e.g. scrubbed logs, detailed feature requests,etc.).
  Whenever it's feasible, please don't hesitate to send a Pull Request (PR) our way. We'd greatly appreciate it, and we'll gladly assess and incorporate your changes.

[waelkdouh]

Please take a look at this .

Please make sure you read the docs and the wiki.

[fabienmazieres]

Thank you @waelkdouh for the quick response. We are using a mono repo and we have a configuration for each of our environments however I am not really sure how the document answer my question. One part relevant to the extractor may be of use for me:

A apiNames section which includes a bogus name (e.g. ignore) in configuration (apiNames:[ignore]) means no apis will be extracted.

So does that means that if I make a bogus API entry details it will prevent the API to be deployed?

[waelkdouh]

First of all you would have to use a repository for each team. Then within each repository you would specify which apis you want to extract using the configuration.extractor.yaml. This way when you get to the publishing part the publisher will publish only what you have extracted. By using multiple repositories you can allow each team to have their own configuration files for each environment. Remember our tools embraces git so any tricks that git allows (e.g. gitignore) can be used here as well.

[fabienmazieres]

@waelkdouh ,

We do not want to use multiple repo, the goal of using APIOps was to move to a centralized repo for all APIM config to enhance security and make it easier to maintain our APIMs.

Could it be something easily implemented as a new feature or is it quite complex due to dependencies (i.e. if the API is not deployed but is part of a product configuration, then the product will fail to deploy)?

[waelkdouh]

Thank you @waelkdouh for the quick response. We are using a mono repo and we have a configuration for each of our environments however I am not really sure how the document answer my question. One part relevant to the extractor may be of use for me:

A apiNames section which includes a bogus name (e.g. ignore) in configuration (apiNames:[ignore]) means no apis will be extracted.

So does that means that if I make a bogus API entry details it will prevent the API to be deployed?

If you don't enter anything in the apis section in the extractor then it will extract everything. If you specify an api/s then it will only extract the specified ones.

[fabienmazieres]

@waelkdouh,

We are not intending to work with the extractor at this point: Teams update the spec files directly into the repo and this will push the config to dev and other environment. Could the publisher be configured to behave the same way as the extracror, if the API is not defined it will be skipped?

[waelkdouh]

The publisher utilizes git to capture the differences between commits. You can always use the git ignore file to help you with what you're trying to achieve here.

[anish2808]

@waelkdouh ,

We do not want to use multiple repo, the goal of using APIOps was to move to a centralized repo for all APIM config to enhance security and make it easier to maintain our APIMs.

Could it be something easily implemented as a new feature or is it quite complex due to dependencies (i.e. if the API is not deployed but is part of a product configuration, then the product will fail to deploy)?

Hi @fabienmazieres - I'm in the process of implementing a mono-repo approach for ApiOps as well. Multi-team with multiple environments (Dev,QA,Prod). I would like to understand:

1. The branching strategy you're employing
2. If you're using Releases in Azure Pipelines for publishing or just using the Publisher pipeline with environments to handle this

Any inputs would be highly appreciated.

[fabienmazieres]

Hi, Dev create can create a branch at anytime and this will trigger an automated deployment to our Dev APIM instance.The automated deployment is based on a github workflow that trigger on any branch other than main and publish the last commit data only.This allows devs to quickly test their change in Dev environment without requiring PR approved and merged, they have read acces on the deb instance to see the effect of their deployment if they need. Once they are happy with the API they can submit a PR that will be squash merged after approval and, a workflow deploys automatically to Staging. This is a workflow triggered on push to main branch only and once again we have it set to publish the last commit only.Staging actually contains 2 different APIM instances so we have a workflow matrix to run the flow against both but one after the other. We have not into Prod yet but the idea would be to have the workflow deploying to Staging wait for an approval in Github (this allows for change management process to happen) before the change is released in our prod instances. At this point we never really used the extractor, we expect devs to come up with the proper API spec from the software build process and then they can create their policies by working the code in their branch and then test it once the workflow deployed it. We had some APIM already existing out of the APIops process so some most dev have a knowledge of the policies or can find examples with the existing instances for what they need to achieve. One of the big challenges we have faced so far has been the reliability of this process: if a team creates 5 API in their first commit but 1 has a syntax error, then fixing the syntax will not trigger a push of API that were not created in the first automated run. Some manual workflow to push all artifact in the branch is needed to reset the work and allow things to work again but this may override some update to an existing API done in another branch. In regards to my original question, limiting API to some instances only, I think we came with a good workaround which would be to make API unusable via Policy unless they are tagged for publication. This means some mock/test API may make its way to the prod instance but the policy will prevent its use. I am still open to suggestion if there any other ways of working that can improve our processes. The tool has good potential but needs a few improvements to make it perfect, so thanks for your hard work and the continuing support!

Sent from Yahoo Mail on Android

On Wed, 31 Jan 2024 at 18:00, @.***> wrote:

@waelkdouh ,

We do not want to use multiple repo, the goal of using APIOps was to move to a centralized repo for all APIM config to enhance security and make it easier to maintain our APIMs.

Could it be something easily implemented as a new feature or is it quite complex due to dependencies (i.e. if the API is not deployed but is part of a product configuration, then the product will fail to deploy)?

Hi @fabienmazieres - I'm in the process of implementing a mono-repo approach for ApiOps as well. Multi-team with multiple environments (Dev,QA,Prod). I would like to understand:

• The branching strategy you're employing
• If you're using Releases in Azure Pipelines for publishing or just using the Publisher pipeline with environments to handle this

Any inputs would be highly appreciated.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[zcarroll4]

@fabienmazieres if you're still looking for a solution and I understand your question correctly. All you would have to do when you run the publisher command is to include a publisher config .yaml file for each region. And as you can see the differences below. These would be three separate files (publisher.dev.yaml, publisher.qa.yaml & publisher.prod.yaml)

[ics-development]

Guys, does this work for revisions, so I only want to publish the latest revisions for example. The reason being, I cant get the publisher to work when I have multiple revisions:

info: Publisher[0] Processing modified files in commit ID... info: Publisher[0] Putting API ipoint-shifts... info: Publisher[0] Putting API ipoint-shifts;rev=1... crit: Publisher[0] System.Net.Http.HttpRequestException: HTTP request to URI https://management.azure.com/subscriptions/***/resourceGroups/rg-apim-staging/providers/Microsoft.ApiManagement/service/apim-acacium-staging/apis/ipoint-shifts?api-version=2022-04-01-preview&asyncId=663162474634610610534ed4&asyncCode=201 failed with status code 409. Content is '{"error":{"code":"IdentifierAlreadyInUse","message":"Resource already exists.","details":null}}'.

[zcarroll4]

I had no issues when running the publisher a few days ago with multiple revisions, I however at the time was not using a config yaml file but just publishing everything located in my artifacts folder. If you;re using a config file, what does that look like?

[fabienmazieres]

The trick with the config file does not seem to work.

I do have a config file for the different environment (configuration.d1.yaml, configuration.s1.yaml and configuration.p1.yaml) and as an example, an API called cl-as has all the required files into the apis folder, but is not declared into any of the configuration files. This API was deployed in all APIM environments. We are mainly doing publishing with last commit in d1 and s1, p1 is published via all artifacts when we need to release to prod. s1 and d1 are sometimes published with full artifact when things have gone wrong and need catchup.

[zcarroll4]

You may want to look at the parameter PUBLISH_CONFIGURATION_ARTIFACTS. I know it used to exist in previous versions. But from my understanding, if that was set to true - then it would only deploy artifacts that are defined in the config file. Whereas if that value was false, it would deploy everything within the artifacts folder. @waelkdouh or @guythetechie may have more insight to what is the best approach now with that variable being deprecated.