[BUG] Removing groups from products fail

[ehsan-uberleads]

Release version

4.0.2

Describe the bug

Publisher fails during removal of a group from a product that previously had the group.

Expected behavior

Missing groups detected on the APIM instance which are not present in the code are detected and flagged for removal and should be removed from the product policy.

Actual behavior

Publisher fails to run the delete group process:

      Application is shutting down...
fail: Microsoft.Extensions.Hosting.Internal.Host[9]
      BackgroundService failed
      System.InvalidOperationException: HTTP request to URI https://management.azure.com/subscriptions/b1adcc35-7256-4488-93ff-0bd35a0600ca/resourceGroups/rg-product-prod-eastus-001/providers/Microsoft.ApiManagement/service/apim-product-prod-eastus/products/unlimited/groups/guests?api-version=2021-12-01-preview failed with status code 500. Content is '{"error":{"code":"InternalServerError","message":"Request processing failed due to internal error.","details":null}}'.
         at common.HttpPipelineExtensions.Validate(Response response, Uri requestUri)
         at common.HttpPipelineExtensions.DeleteResource(HttpPipeline pipeline, Uri uri, CancellationToken cancellationToken)
         at publisher.Program.<>c__DisplayClass10_0.<<GetDeleteRestResource>b__0>d.MoveNext()
      --- End of stack trace from previous location ---
         at publisher.ProductGroup.Delete(GroupName groupName, ProductUri productUri, DeleteRestResource deleteRestResource, CancellationToken cancellationToken)
         at publisher.ProductGroup.<>c__DisplayClass7_0.<<Put>b__0>d.MoveNext()
      --- End of stack trace from previous location ---
         at System.Threading.Tasks.Parallel.<>c__50`1.<<ForEachAsync>b__50_0>d.MoveNext()
      --- End of stack trace from previous location ---
         at common.IEnumerableExtensions.ForEachParallel[T](IEnumerable`1 enumerable, Func`2 action, CancellationToken cancellationToken)
         at publisher.ProductGroup.Put(ProductName productName, IReadOnlyCollection`1 groupNames, ServiceUri serviceUri, ListRestResources listRestResources, PutRestResource putRestResource, DeleteRestResource deleteRestResource, ILogger logger, CancellationToken cancellationToken)
         at publisher.ProductGroup.<>c__DisplayClass1_0.<<Put>b__0>d.MoveNext()
      --- End of stack trace from previous location ---
         at System.Threading.Tasks.Parallel.<>c__50`1.<<ForEachAsync>b__50_0>d.MoveNext()
      --- End of stack trace from previous location ---
         at common.IEnumerableExtensions.ForEachParallel[T](IEnumerable`1 enumerable, Func`2 action, CancellationToken cancellationToken)
         at publisher.ProductGroup.Put(IReadOnlyCollection`1 files, JsonObject configurationJson, ServiceDirectory serviceDirectory, ServiceUri serviceUri, ListRestResources listRestResources, PutRestResource putRestResource, DeleteRestResource deleteRestResource, ILogger logger, CancellationToken cancellationToken)
         at publisher.ProductGroup.ProcessArtifactsToPut(IReadOnlyCollection`1 files, JsonObject configurationJson, ServiceDirectory serviceDirectory, ServiceUri serviceUri, ListRestResources listRestResources, PutRestResource putRestResource, DeleteRestResource deleteRestResource, ILogger logger, CancellationToken cancellationToken)
         at publisher.Service.ProcessArtifactsToPut(IReadOnlyCollection`1 files, JsonObject configurationJson, ServiceDirectory serviceDirectory, ServiceUri serviceUri, ListRestResources listRestResources, PutRestResource putRestResource, DeleteRestResource deleteRestResource, ILogger logger, CancellationToken cancellationToken)
         at publisher.Publisher.RunWithoutCommitId(CancellationToken cancellationToken)
         at publisher.Publisher.Run(CancellationToken cancellationToken)
         at publisher.Publisher.ExecuteAsync(CancellationToken cancellationToken)
         at Microsoft.Extensions.Hosting.Internal.Host.TryExecuteBackgroundServiceAsync(BackgroundService backgroundService)

Reproduction Steps

1. Use default provisioned APIM products.
2. for Unlimited product inside groups.json file use the following:

[
  {
    "name": "administrators"
  },
  {
    "name": "developers"
  }
]

3. Run the publisher tool

[dannythomas13]

I love this project and it seems great for additions/modifications but it doesn't feel robust for deletions just yet.

1. I tried removing Echo API from my lower env and as it was the only api within Starter product it attempted to remove that which failed.
2. After the above failed I reverted and then simply tried removing two operations from Echo API and that failed also.

[waelkdouh]

@dannythomas13 happy to hear you are loving this product. As an open source project it thrives through feedback like yours. And rest assured we are always taking input similar to yours.

@guythetechie any thoughts on this?

[dannythomas13]

@dannythomas13 happy to hear you are loving this product. As an open source project it thrives through feedback like yours. And rest assured we are always taking input similar to yours.

@guythetechie any thoughts on this?

Hi both, do we have any update on how to go about deleting? Should this be done manually through all environments for now?

[waelkdouh]

@dannythomas13 please try v6 which should address this issue.