azure-rest-api-specs icon indicating copy to clipboard operation
azure-rest-api-specs copied to clipboard

add extension metadata V2 API and example

Open jiaqi-beep opened this issue 1 year ago • 37 comments

ARM (Control Plane) API Specification Update Pull Request

[!TIP] Overwhelmed by all this guidance? See the Getting help section at the bottom of this PR description.

PR review workflow diagram

Please understand this diagram before proceeding. It explains how to get your PR approved & merged.

spec_pr_review_workflow_diagram

Purpose of this PR

What's the purpose of this PR? Check the specific option that applies. This is mandatory!

  • [ ] New resource provider.
  • [x] New API version for an existing resource provider. (If API spec is not defined in TypeSpec, the PR should have been created in adherence to OpenAPI specs PR creation guidance).
  • [ ] Update existing version for a new feature. (This is applicable only when you are revising a private preview API version.)
  • [ ] Update existing version to fix OpenAPI spec quality issues in S360.
  • [ ] Convert existing OpenAPI spec to TypeSpec spec (do not combine this with implementing changes for a new API version).
  • [ ] Other, please clarify:
    • edit this with your clarification

Due diligence checklist

To merge this PR, you must go through the following checklist and confirm you understood and followed the instructions by checking all the boxes:

  • [x] I confirm this PR is modifying Azure Resource Manager (ARM) related specifications, and not data plane related specifications.
  • [x] I have reviewed following Resource Provider guidelines, including ARM resource provider contract and REST guidelines (estimated time: 4 hours).
    I understand this is required before I can proceed to the diagram Step 2, "ARM API changes review", for this PR.

Additional information

Viewing API changes

For convenient view of the API changes made by this PR, refer to the URLs provided in the table in the Generated ApiView comment added to this PR. You can use ApiView to show API versions diff.

Suppressing failures

If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the suppressions guide to get approval.

Getting help

  • First, please carefully read through this PR description, from top to bottom. Please fill out the Purpose of this PR and Due diligence checklist.
  • If you don't have permissions to remove or add labels to the PR, request write access per aka.ms/azsdk/access#request-access-to-rest-api-or-sdk-repositories
  • To understand what you must do next to merge this PR, see the Next Steps to Merge comment. It will appear within few minutes of submitting this PR and will continue to be up-to-date with current PR state.
  • For guidance on fixing this PR CI check failures, see the hyperlinks provided in given failure and https://aka.ms/ci-fix.
  • For help with ARM review (PR workflow diagram Step 2), see https://aka.ms/azsdk/pr-arm-review.
  • If the PR CI checks appear to be stuck in queued state, please add a comment with contents /azp run. This should result in a new comment denoting a PR validation pipeline has started and the checks should be updated after few minutes.
  • If the help provided by the previous points is not enough, post to https://aka.ms/azsdk/support/specreview-channel and link to this PR.

jiaqi-beep avatar Sep 19 '24 23:09 jiaqi-beep

Next Steps to Merge

Next steps that must be taken to merge this PR:
  • ❌ This PR is labelled with ArcReview. For this PR to be merged, it must pass an ARC review and be labelled ArcSignedOff.
    Email the ARC board to request review per this Contact section.
  • ❌ The required check named Automated merging requirements met has failed. This is the final check that must pass. Refer to the check in the PR's 'Checks' tab for details on how to fix it and consult the aka.ms/ci-fix guide. In addition, refer to step 4 in the PR workflow diagram

Generated ApiView

Language Package Name ApiView Link
Go sdk/resourcemanager/hybridcompute/armhybridcompute https://apiview.dev/Assemblies/Review/cc99eb4c91204d199d8e40aed71bc06e?revisionId=5b65495205c44f9f81cef09a0e5222a7
JavaScript @azure/arm-hybridcompute https://apiview.dev/Assemblies/Review/bd79793b6b53409d8de21847c605dc64?revisionId=e3397cae43fa4e38950388ed4d5dd81f
.Net Azure.ResourceManager.HybridCompute There is no API change compared with the previous version
Java azure-resourcemanager-hybridcompute https://apiview.dev/Assemblies/Review/b79886348ebe497b9ddc99d286cb6e99?revisionId=9cd67fa7e0cd4612aea17ac5e494bf3d
Swagger Microsoft.HybridCompute https://apiview.dev/Assemblies/Review/31dae74112a04fb7b3a4a58e2c398536?revisionId=b7de4f0577c14fedba4258ea28a66218

Update the Purpose of this PR as this is a version update from what i see in file

razvanbadea-msft avatar Sep 20 '24 10:09 razvanbadea-msft

/azp run

razvanbadea-msft avatar Sep 20 '24 10:09 razvanbadea-msft

Azure Pipelines successfully started running 3 pipeline(s).

azure-pipelines[bot] avatar Sep 20 '24 10:09 azure-pipelines[bot]

"/subscriptions/{subscriptionId}/providers/Microsoft.HybridCompute/locations/{location}/publishers/{publisher}/extensionTypes/{extensionType}/versions/{version}": {

should the newly added models v2 match with this sub level operations?


Refers to: specification/hybridcompute/resource-manager/Microsoft.HybridCompute/preview/2024-09-10-preview/HybridCompute.json:1514 in 9f88be5. [](commit_id = 9f88be51576097c7728ad71434cfa98b4de8b3e5, deletion_comment = False)

razvanbadea-msft avatar Sep 20 '24 10:09 razvanbadea-msft

check also the Swagger ModelValidation failures

razvanbadea-msft avatar Sep 20 '24 11:09 razvanbadea-msft

"/subscriptions/{subscriptionId}/providers/Microsoft.HybridCompute/locations/{location}/publishers/{publisher}/extensionTypes/{extensionType}/versions/{version}": {

should the newly added models v2 match with this sub level operations?

Refers to: specification/hybridcompute/resource-manager/Microsoft.HybridCompute/preview/2024-09-10-preview/HybridCompute.json:1514 in 9f88be5. [](commit_id = 9f88be5, deletion_comment = False)

We are trying to introduce extension metadata V2 APIs that do not require subscriptionID parameters. Is this allowed? Would this be considered as breaking change? The existing metadata API would not change.

jiaqi-beep avatar Sep 20 '24 18:09 jiaqi-beep

About the LintDiff errors:

  • Why is ExtensionValueV2 considered tracked resource?
  • Seems to me all the resources that are collections only have value in their properties
  • Not sure why operation 'ExtensionMetadata_List' tries to match the response definition of 'ExtensionMetadataV2_Get'
  • If I add a property to a definition in the future, would that be a breaking change?

jiaqi-beep avatar Sep 20 '24 23:09 jiaqi-beep

  • ExtensionValueV2 is being considered Tracked because it defines a top level "location" property. If u intend this to be a proxy resource, pl remove the location property.
  • They must have value and the nextLink properties as indicated in the help file for the violation. You need to add the nextLink property.
  • Please add a suppression for the GetCollectionResponseSchema rule. https://eng.ms/docs/products/azure-developer-experience/design/specs-pr-guides/pr-suppressions
  • You must add it in a new API version to be considered as not a breaking change..

In reply to: 2364760239

rkmanda avatar Oct 04 '24 00:10 rkmanda

@rkmanda Requesting suppression for GetCollectionResponseSchema error. Justification: This appears to be a false positive case.

jiaqi-beep avatar Oct 07 '24 22:10 jiaqi-beep

@rkmanda The changes I am committing to are for 2024-09-10-preview, which is a new API version that has not been merged into main, see https://github.com/Azure/azure-rest-api-specs/pull/30546.

My teammate made a similar PR against the release branch where the breaking change check is able to pass https://github.com/Azure/azure-rest-api-specs/pull/30592

jiaqi-beep avatar Oct 07 '24 22:10 jiaqi-beep

"/subscriptions/{subscriptionId}/providers/Microsoft.HybridCompute/locations/{location}/publishers/{publisher}/extensionTypes/{extensionType}/versions/{version}": {

please respond to this query as well


In reply to: 2363448681


Refers to: specification/hybridcompute/resource-manager/Microsoft.HybridCompute/preview/2024-09-10-preview/HybridCompute.json:1514 in 9f88be5. [](commit_id = 9f88be51576097c7728ad71434cfa98b4de8b3e5, deletion_comment = False)

raosuhas avatar Oct 08 '24 23:10 raosuhas

@rkmanda Requesting suppression for GetCollectionResponseSchema error. Justification: This appears to be a false positive case.

you need to add the suppression yourself. Please check the automated comment above on adding suppressions

raosuhas avatar Oct 08 '24 23:10 raosuhas

"/subscriptions/{subscriptionId}/providers/Microsoft.HybridCompute/locations/{location}/publishers/{publisher}/extensionTypes/{extensionType}/versions/{version}": {

should the newly added models v2 match with this sub level operations?

Refers to: specification/hybridcompute/resource-manager/Microsoft.HybridCompute/preview/2024-09-10-preview/HybridCompute.json:1514 in 9f88be5. [](commit_id = 9f88be5, deletion_comment = False)

No it shouldn't. We mean to remove the subscription param in v2 API.

jiaqi-beep avatar Oct 08 '24 23:10 jiaqi-beep

@rkmanda Requesting suppression for GetCollectionResponseSchema error. Justification: This appears to be a false positive case.

you need to add the suppression yourself. Please check the automated comment above on adding suppressions

@raosuhas I am confused how to add suppression. The doc just says mentions this in the comment. What am I missing? image

jiaqi-beep avatar Oct 09 '24 00:10 jiaqi-beep

"/subscriptions/{subscriptionId}/providers/Microsoft.HybridCompute/locations/{location}/publishers/{publisher}/extensionTypes/{extensionType}/versions/{version}": {

should the newly added models v2 match with this sub level operations? Refers to: specification/hybridcompute/resource-manager/Microsoft.HybridCompute/preview/2024-09-10-preview/HybridCompute.json:1514 in 9f88be5. [](commit_id = 9f88be5, deletion_comment = False)

We are trying to introduce extension metadata V2 APIs that do not require subscriptionID parameters. Is this allowed? Would this be considered as breaking change? The existing metadata API would not change.

I believe this is the core question with respect to the LintDiff error GetCollectionResponseSchema. Should this scenario, where two paths in a spec only vary by a /subscriptions/{subscriptionId} prefix, but return different data types, even be allowed?

If not, then the rule is technically correct to raise an error here, though the error message is misleading due to the similar paths.

If so, then the rule will need to be updated to allow it. See https://github.com/Azure/azure-openapi-validator/issues/750 for a likely root cause.

mikeharder avatar Oct 17 '24 03:10 mikeharder

"/subscriptions/{subscriptionId}/providers/Microsoft.HybridCompute/locations/{location}/publishers/{publisher}/extensionTypes/{extensionType}/versions/{version}": {

should the newly added models v2 match with this sub level operations? Refers to: specification/hybridcompute/resource-manager/Microsoft.HybridCompute/preview/2024-09-10-preview/HybridCompute.json:1514 in 9f88be5. [](commit_id = 9f88be5, deletion_comment = False)

We are trying to introduce extension metadata V2 APIs that do not require subscriptionID parameters. Is this allowed? Would this be considered as breaking change? The existing metadata API would not change.

I believe this is the core question with respect to the LintDiff error GetCollectionResponseSchema. Should this scenario, where two paths in a spec only vary by a /subscriptions/{subscriptionId} prefix, but return different data types, even be allowed?

If not, then the rule is technically correct to raise an error here, though the error message is misleading due to the similar paths.

If so, then the rule will need to be updated to allow it. See Azure/azure-openapi-validator#750 for a likely root cause.

@mikeharder How do we get an answer for this?

jiaqi-beep avatar Oct 17 '24 17:10 jiaqi-beep

I believe this is the core question with respect to the LintDiff error GetCollectionResponseSchema. Should this scenario, where two paths in a spec only vary by a /subscriptions/{subscriptionId} prefix, but return different data types, even be allowed? If not, then the rule is technically correct to raise an error here, though the error message is misleading due to the similar paths. If so, then the rule will need to be updated to allow it. See Azure/azure-openapi-validator#750 for a likely root cause.

@mikeharder How do we get an answer for this?

The ARM Reviewer who reviews your PR (and the suppression) should be able to find an answer.

mikeharder avatar Oct 17 '24 17:10 mikeharder

I believe this is the core question with respect to the LintDiff error GetCollectionResponseSchema. Should this scenario, where two paths in a spec only vary by a /subscriptions/{subscriptionId} prefix, but return different data types, even be allowed? If not, then the rule is technically correct to raise an error here, though the error message is misleading due to the similar paths. If so, then the rule will need to be updated to allow it. See Azure/azure-openapi-validator#750 for a likely root cause.

@mikeharder How do we get an answer for this?

The ARM Reviewer who reviews your PR (and the suppression) should be able to find an answer.

See above comment. You need to unify the two payloads and add a filter parameter to support the subset object scenario.

mentat9 avatar Oct 17 '24 21:10 mentat9

Please review https://github.com/Azure/azure-rest-api-specs/pull/31063 that handles breaking change cross version failure

jiaqi-beep avatar Oct 17 '24 22:10 jiaqi-beep

I believe this is the core question with respect to the LintDiff error GetCollectionResponseSchema. Should this scenario, where two paths in a spec only vary by a /subscriptions/{subscriptionId} prefix, but return different data types, even be allowed? If not, then the rule is technically correct to raise an error here, though the error message is misleading due to the similar paths. If so, then the rule will need to be updated to allow it. See Azure/azure-openapi-validator#750 for a likely root cause.

@mikeharder How do we get an answer for this?

The ARM Reviewer who reviews your PR (and the suppression) should be able to find an answer.

See above comment. You need to unify the two payloads and add a filter parameter to support the subset object scenario.

Hi @mentat9, the question we had was if it's a valid scenario to introduce new API paths that only differ from existing paths by missing subscription ID param, and different response data types. Please confirm this is valid or not.

If we still want to implement subset object scenario, I think select parameter is more appropriate than filter.

Also the most recent commit has already unified the payloads. Suppression review is still needed because of the bug in GetCollectionResponseSchema check. See https://github.com/Azure/azure-rest-api-specs/pull/30656#discussion_r1802151641

jiaqi-beep avatar Oct 17 '24 22:10 jiaqi-beep

I believe this is the core question with respect to the LintDiff error GetCollectionResponseSchema. Should this scenario, where two paths in a spec only vary by a /subscriptions/{subscriptionId} prefix, but return different data types, even be allowed? If not, then the rule is technically correct to raise an error here, though the error message is misleading due to the similar paths. If so, then the rule will need to be updated to allow it. See Azure/azure-openapi-validator#750 for a likely root cause.

@mikeharder How do we get an answer for this?

The ARM Reviewer who reviews your PR (and the suppression) should be able to find an answer.

See above comment. You need to unify the two payloads and add a filter parameter to support the subset object scenario.

Hi @mentat9, the question we had was if it's a valid scenario to introduce new API paths that only differ from existing paths by missing subscription ID param, and different response data types. Please confirm this is valid or not.

If we still want to implement subset object scenario, I think select parameter is more appropriate than filter.

Also the most recent commit has already unified the payloads. Suppression review is still needed because of the bug in GetCollectionResponseSchema check. See #30656 (comment)

Sorry, I meant to say query parameter instead of filter parameter. Agree that select is the right choice of query parameter.

Returning different payloads at different scopes for what is nominally the same resource type seems questionable. Discussing this with the ARM reviewer community to get more input.

mentat9 avatar Oct 18 '24 00:10 mentat9

I believe this is the core question with respect to the LintDiff error GetCollectionResponseSchema. Should this scenario, where two paths in a spec only vary by a /subscriptions/{subscriptionId} prefix, but return different data types, even be allowed? If not, then the rule is technically correct to raise an error here, though the error message is misleading due to the similar paths. If so, then the rule will need to be updated to allow it. See Azure/azure-openapi-validator#750 for a likely root cause.

@mikeharder How do we get an answer for this?

The ARM Reviewer who reviews your PR (and the suppression) should be able to find an answer.

See above comment. You need to unify the two payloads and add a filter parameter to support the subset object scenario.

Hi @mentat9, the question we had was if it's a valid scenario to introduce new API paths that only differ from existing paths by missing subscription ID param, and different response data types. Please confirm this is valid or not. If we still want to implement subset object scenario, I think select parameter is more appropriate than filter. Also the most recent commit has already unified the payloads. Suppression review is still needed because of the bug in GetCollectionResponseSchema check. See #30656 (comment)

Sorry, I meant to say query parameter instead of filter parameter. Agree that select is the right choice of query parameter.

Returning different payloads at different scopes for what is nominally the same resource type seems questionable. Discussing this with the ARM reviewer community to get more input.

Quoting the response on this question "Since the schema is different, the name of the Type must also be different within a namespace. The ARM Manifest would have a single entry for the type and doesnt document the scopes separately, so it makes even more sense to not allow the pattern where the definition varies at each scope. "

Could you take this feedback and make changes accordingly ?

ramoka178 avatar Oct 21 '24 18:10 ramoka178

I believe this is the core question with respect to the LintDiff error GetCollectionResponseSchema. Should this scenario, where two paths in a spec only vary by a /subscriptions/{subscriptionId} prefix, but return different data types, even be allowed? If not, then the rule is technically correct to raise an error here, though the error message is misleading due to the similar paths. If so, then the rule will need to be updated to allow it. See Azure/azure-openapi-validator#750 for a likely root cause.

@mikeharder How do we get an answer for this?

The ARM Reviewer who reviews your PR (and the suppression) should be able to find an answer.

See above comment. You need to unify the two payloads and add a filter parameter to support the subset object scenario.

Hi @mentat9, the question we had was if it's a valid scenario to introduce new API paths that only differ from existing paths by missing subscription ID param, and different response data types. Please confirm this is valid or not. If we still want to implement subset object scenario, I think select parameter is more appropriate than filter. Also the most recent commit has already unified the payloads. Suppression review is still needed because of the bug in GetCollectionResponseSchema check. See #30656 (comment)

Sorry, I meant to say query parameter instead of filter parameter. Agree that select is the right choice of query parameter. Returning different payloads at different scopes for what is nominally the same resource type seems questionable. Discussing this with the ARM reviewer community to get more input.

Quoting the response on this question "Since the schema is different, the name of the Type must also be different within a namespace. The ARM Manifest would have a single entry for the type and doesnt document the scopes separately, so it makes even more sense to not allow the pattern where the definition varies at each scope. "

Could you take this feedback and make changes accordingly ?

Hi @ramoka178, to verify, are you saying that we need to modify the metadata V2 API paths in this PR so that they have different scopes from the existing API? If so, are there current practices that do this and how would you recommend we modify the paths?

jiaqi-beep avatar Oct 22 '24 19:10 jiaqi-beep

I believe this is the core question with respect to the LintDiff error GetCollectionResponseSchema. Should this scenario, where two paths in a spec only vary by a /subscriptions/{subscriptionId} prefix, but return different data types, even be allowed? If not, then the rule is technically correct to raise an error here, though the error message is misleading due to the similar paths. If so, then the rule will need to be updated to allow it. See Azure/azure-openapi-validator#750 for a likely root cause.

@mikeharder How do we get an answer for this?

The ARM Reviewer who reviews your PR (and the suppression) should be able to find an answer.

See above comment. You need to unify the two payloads and add a filter parameter to support the subset object scenario.

Hi @mentat9, the question we had was if it's a valid scenario to introduce new API paths that only differ from existing paths by missing subscription ID param, and different response data types. Please confirm this is valid or not. If we still want to implement subset object scenario, I think select parameter is more appropriate than filter. Also the most recent commit has already unified the payloads. Suppression review is still needed because of the bug in GetCollectionResponseSchema check. See #30656 (comment)

Sorry, I meant to say query parameter instead of filter parameter. Agree that select is the right choice of query parameter. Returning different payloads at different scopes for what is nominally the same resource type seems questionable. Discussing this with the ARM reviewer community to get more input.

Quoting the response on this question "Since the schema is different, the name of the Type must also be different within a namespace. The ARM Manifest would have a single entry for the type and doesnt document the scopes separately, so it makes even more sense to not allow the pattern where the definition varies at each scope. " Could you take this feedback and make changes accordingly ?

Hi @ramoka178, to verify, are you saying that we need to modify the metadata V2 API paths in this PR so that they have different scopes from the existing API? If so, are there current practices that do this and how would you recommend we modify the paths?

You may have to change the API path "/providers/Microsoft.HybridCompute/locations/{location}/publishers/{publisher}/extensionTypes/{extensionType}/versions" that is common for both to be different for each case. for example: "/providers/Microsoft.HybridCompute/locations/{location}/publishers/{publisher}/aggregatedExtensionTypes/{extensionType}/versions" for the paths where ExtensionValueListResultV2 kind of result is to be published.

ramoka178 avatar Oct 22 '24 20:10 ramoka178

Hi @ramoka178 , it would actually make more sense to keep the same API path starting at provider level, than introducing a new scope, since our new API functions as v2 of current API -- they perform same logical functions.

  • The new API has different response schema because we want to add additional properties to enrich the response.
  • We don't want to introduce new version on existing API, because we don't want the API tied to subscription id, since this API should not be subscription scoped.

So using the same provider/location/publisher/extensionType/version scope is logically correct, and it would be very confusing for users to have a separate API path for the same logical function.

jiaqi-beep avatar Oct 23 '24 18:10 jiaqi-beep

Also, please provide a suppression review.

jiaqi-beep avatar Oct 23 '24 18:10 jiaqi-beep

Hi @ramoka178 , it would actually make more sense to keep the same API path starting at provider level, than introducing a new scope, since our new API functions as v2 of current API -- they perform same logical functions.

  • The new API has different response schema because we want to add additional properties to enrich the response.
  • We don't want to introduce new version on existing API, because we don't want the API tied to subscription id, since this API should not be subscription scoped.

So using the same provider/location/publisher/extensionType/version scope is logically correct, and it would be very confusing for users to have a separate API path for the same logical function.

@jiaqi-beep As discussed in the office hours , the API path is "provider/location/publisher/extensionType/version" and any requests using this API path should be consistent in how it responds to.

If this API is not subscription scoped and yet gives a V2 kind of response for /subscription/{subscriptionId}/provider/location/publisher/extensionType/version" now, it is giving a sense that this API response is actually scope based.

If this needs a modelling review, please book a slot for API modelling office hours.

ramoka178 avatar Oct 23 '24 18:10 ramoka178

If this API is not subscription scoped and yet gives a V2 kind of response for /subscription/{subscriptionId}/provider/location/publisher/extensionType/version" now, it is giving a sense that this API response is actually scope based.

Hi @ramoka178 - Irrespective of what the current API does (at Subscription scope level), the new API is designed to serve Provider artifacts. The API response properties will not contain any references to Subscription. Based on Customer and Partner feedback, we came up with the path at Provider level.

Please elaborate on this statement:

"If this API is not subscription scoped and yet gives a V2 kind of response for /subscription/{subscriptionId}/provider/location/publisher/extensionType/version" now, it is giving a sense that this API response is actually scope based."

raghushantha avatar Oct 24 '24 17:10 raghushantha