azure-rest-api-specs icon indicating copy to clipboard operation
azure-rest-api-specs copied to clipboard

[Maps] Async Batch APIs for Forward/Reverse Geocode and Routing

Open danielthank opened this issue 1 year ago • 31 comments

Data Plane API - Pull Request

Add async batch APIs for forward geocode, reverse geocode, and routing.

API Info: The Basics

Most of the information about your service should be captured in the issue that serves as your API Spec engagement record.

  • Link to API Spec engagement record issue: https://github.com/Azure/azure-rest-api-specs/issues/27158

Is this review for (select one):

  • [ ] a private preview
  • [x] a public preview
  • [ ] GA release

Change Scope

This section will help us focus on the specific parts of your API that are new or have been modified.
Please share a link to the design document for the new APIs, a link to the previous API Spec document (if applicable), and the root paths that have been updated.

  • Design Document:
    • Service Design: https://microsoft.sharepoint.com/:w:/t/Azure-Maps/Eb8_fl1CuqpMkFYqGIW5cqUB5W-SY1MPbEwqOmS14mToVw?e=b3Bg37
    • API spec: https://microsoft.sharepoint.com/:w:/t/Azure-Maps/ERzkm1S1_ehPmrpXSKwtcogBruwDXaBQnKq99N9UeVa0-g?e=bwoPYY
  • Previous API Spec Doc: NA
  • Updated paths:

Viewing API changes

For convenient view of the API changes made by this PR, refer to the URLs provided in the table in the Generated ApiView comment added to this PR. You can use ApiView to show API versions diff.

Suppressing failures

If one or multiple validation error/warning suppression(s) is detected in your PR, please follow the Swagger-Suppression-Process to get approval.

❔Got questions? Need additional info?? We are here to help!

Contact us!

The Azure API Review Board is dedicated to helping you create amazing APIs. You can read about our mission and learn more about our process on our wiki.

Click here for links to tools, specs, guidelines & other good stuff

Tooling

Guidelines & Specifications

Helpful Links

Checks stuck in `queued` state? If the PR CI checks appear to be stuck in `queued` state, please add a comment with contents `/azp run`. This should result in a new comment denoting a `PR validation pipeline` has started and the checks should be updated after few minutes.

danielthank avatar Dec 26 '23 03:12 danielthank

Next Steps to Merge

Important checks have failed. As of today they are not blocking this PR, but in near future they will.
Addressing the following failures is highly recommended:
  • ⚠️ The check named TypeSpec Requirement (data-plane) has failed. TypeSpec usage is required for all new (greenfield) services. This is currently enforced as a warning for data-plane specs, but will be made a blocking error in the near future. For information on converting from OpenAPI specs to TypeSpec specs or on data-plane (DP) policies, refer to aka.ms/azsdk/typespec. If you have general questions on resource provider (RP) policies, refer to aka.ms/rphelp.
If you still want to proceed merging this PR without addressing the above failures, refer to step 4 in the PR workflow diagram (even if your PR is for data plane, not ARM).

Swagger Validation Report

️️✔️BreakingChange succeeded [Detail] [Expand]
There are no breaking changes.
️⚠️Breaking Change(Cross-Version): 9 Warnings warning [Detail]
Compared specs (v0.10.8) new version base version
route.json 2024-04-01-preview(18e0f51) 2023-10-01-preview(main)
search.json 2024-04-01-preview(18e0f51) 2023-06-01(main)
search.json 2024-04-01-preview(18e0f51) 2022-12-01-preview(main)

The following breaking changes are detected by comparison with the latest stable version:

Rule Message
1006 - RemovedDefinition The new version is missing a definition that was found in the old version. Was 'default' removed or renamed?
New: Search/preview/2024-04-01-preview/search.json#L52:3
Old: Search/stable/2023-06-01/search.json#L52:3


The following breaking changes are detected by comparison with the latest preview version:

Rule Message
:warning: 1006 - RemovedDefinition The new version is missing a definition that was found in the old version. Was 'default' removed or renamed?
New: Search/preview/2024-04-01-preview/search.json#L52:3
Old: Search/preview/2022-12-01-preview/search.json#L52:3
:warning: 1007 - RemovedClientParameter The new version is missing a client parameter that was found in the old version. Was 'Location' removed or renamed?
New: Search/preview/2024-04-01-preview/search.json#L53:3
Old: Search/preview/2022-12-01-preview/search.json#L53:3
:warning: 1027 - DefaultValueChanged The new version has a different default value than the previous one.
New: Search/preview/2024-04-01-preview/search.json#L888:9
Old: Search/preview/2022-12-01-preview/search.json#L767:9
:warning: 1033 - RemovedProperty The new version is missing a property found in the old version. Was 'countryRegionSet' renamed or removed?
New: Search/preview/2024-04-01-preview/search.json#L853:7
Old: Search/preview/2022-12-01-preview/search.json#L729:7
:warning: 1033 - RemovedProperty The new version is missing a property found in the old version. Was 'location' renamed or removed?
New: Search/preview/2024-04-01-preview/search.json#L853:7
Old: Search/preview/2022-12-01-preview/search.json#L729:7
:warning: 1033 - RemovedProperty The new version is missing a property found in the old version. Was 'strictMatch' renamed or removed?
New: Search/preview/2024-04-01-preview/search.json#L853:7
Old: Search/preview/2022-12-01-preview/search.json#L729:7
:warning: 1046 - RemovedOptionalParameter The optional parameter 'countryRegionSet' was removed in the new version.
Old: Search/preview/2022-12-01-preview/search.json#L333:11
:warning: 1046 - RemovedOptionalParameter The optional parameter 'location' was removed in the new version.
Old: Search/preview/2022-12-01-preview/search.json#L309:9
:warning: 1046 - RemovedOptionalParameter The optional parameter 'strictMatch' was removed in the new version.
Old: Search/preview/2022-12-01-preview/search.json#L389:11
️️✔️CredScan succeeded [Detail] [Expand]
There is no credential detected.
️⚠️LintDiff: 34 Warnings warning [Detail]
Compared specs (v2.2.1) new version base version
package-preview-2024-04 package-preview-2024-04(18e0f51) default(main)
package-preview-2024-04 package-preview-2024-04(18e0f51) default(main)
package-2024-04-01-preview package-2024-04-01-preview(18e0f51) default(main)
package-stable-2023-06-01 package-stable-2023-06-01(18e0f51) package-stable-2023-06-01(main)
package-preview-2024-04 package-preview-2024-04(18e0f51) default(main)

[must fix]The following errors/warnings are introduced by current PR:

Only 30 items are listed, please refer to log for more details.

Rule Message Related RPC [For API reviewers]
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: AsyncBatchManagement/preview/2024-04-01-preview/asyncBatchManagement.json#L120
:warning: ErrorResponse The error property in the error response schema should be required.
Location: AsyncBatchManagement/preview/2024-04-01-preview/asyncBatchManagement.json#L120
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: AsyncBatchManagement/preview/2024-04-01-preview/asyncBatchManagement.json#L160
:warning: ErrorResponse The error property in the error response schema should be required.
Location: AsyncBatchManagement/preview/2024-04-01-preview/asyncBatchManagement.json#L160
:warning: AvoidNestedProperties Consider using x-ms-client-flatten to provide a better end user experience
Location: AsyncBatchManagement/preview/2024-04-01-preview/asyncBatchManagement.json#L302
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Route/preview/2024-04-01-preview/route.json#L244
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Search/preview/2024-04-01-preview/search.json#L515
:warning: ErrorResponse The error property in the error response schema should be required.
Location: Search/preview/2024-04-01-preview/search.json#L515
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Search/preview/2024-04-01-preview/search.json#L764
:warning: ErrorResponse The error property in the error response schema should be required.
Location: Search/preview/2024-04-01-preview/search.json#L764
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: AsyncBatchManagement/preview/2024-04-01-preview/asyncBatchManagement.json#L120
:warning: ErrorResponse The error property in the error response schema should be required.
Location: AsyncBatchManagement/preview/2024-04-01-preview/asyncBatchManagement.json#L120
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: AsyncBatchManagement/preview/2024-04-01-preview/asyncBatchManagement.json#L160
:warning: ErrorResponse The error property in the error response schema should be required.
Location: AsyncBatchManagement/preview/2024-04-01-preview/asyncBatchManagement.json#L160
:warning: AvoidNestedProperties Consider using x-ms-client-flatten to provide a better end user experience
Location: AsyncBatchManagement/preview/2024-04-01-preview/asyncBatchManagement.json#L302
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Route/preview/2024-04-01-preview/route.json#L121
:warning: PaginationResponse Operation might be pageable. Consider adding the x-ms-pageable extension.
Location: Route/preview/2024-04-01-preview/route.json#L136
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Route/preview/2024-04-01-preview/route.json#L184
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Route/preview/2024-04-01-preview/route.json#L244
:warning: EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
Location: Route/preview/2024-04-01-preview/route.json#L354
:warning: PropertyType Property should have a defined type.
Location: Route/preview/2024-04-01-preview/route.json#L848
:warning: PropertyType Property should have a defined type.
Location: Route/preview/2024-04-01-preview/route.json#L853
:warning: PropertyType Property should have a defined type.
Location: Route/preview/2024-04-01-preview/route.json#L996
:warning: PropertyType Property should have a defined type.
Location: Route/preview/2024-04-01-preview/route.json#L1001
:warning: EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
Location: Route/preview/2024-04-01-preview/route.json#L1791
:warning: AvoidNestedProperties Consider using x-ms-client-flatten to provide a better end user experience
Location: Route/preview/2024-04-01-preview/route.json#L2048
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Search/preview/2024-04-01-preview/search.json#L515
:warning: ErrorResponse The error property in the error response schema should be required.
Location: Search/preview/2024-04-01-preview/search.json#L515
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Search/preview/2024-04-01-preview/search.json#L515
:warning: ErrorResponse The error property in the error response schema should be required.
Location: Search/preview/2024-04-01-preview/search.json#L515


The following errors/warnings exist before current PR submission:

Only 30 items are listed, please refer to log for more details.

Rule Message
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Route/preview/2024-04-01-preview/route.json#L121
:warning: PaginationResponse Operation might be pageable. Consider adding the x-ms-pageable extension.
Location: Route/preview/2024-04-01-preview/route.json#L136
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Route/preview/2024-04-01-preview/route.json#L184
:warning: EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
Location: Route/preview/2024-04-01-preview/route.json#L354
:warning: PropertyType Property should have a defined type.
Location: Route/preview/2024-04-01-preview/route.json#L848
:warning: PropertyType Property should have a defined type.
Location: Route/preview/2024-04-01-preview/route.json#L853
:warning: PropertyType Property should have a defined type.
Location: Route/preview/2024-04-01-preview/route.json#L996
:warning: PropertyType Property should have a defined type.
Location: Route/preview/2024-04-01-preview/route.json#L1001
:warning: EnumInsteadOfBoolean Booleans properties are not descriptive in all cases and can make them to use, evaluate whether is makes sense to keep the property as boolean or turn it into an enum.
Location: Route/preview/2024-04-01-preview/route.json#L1791
:warning: AvoidNestedProperties Consider using x-ms-client-flatten to provide a better end user experience
Location: Route/preview/2024-04-01-preview/route.json#L2048
:warning: PaginationResponse Operation might be pageable. Consider adding the x-ms-pageable extension.
Location: Search/preview/2024-04-01-preview/search.json#L289
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Search/preview/2024-04-01-preview/search.json#L403
:warning: ErrorResponse The error property in the error response schema should be required.
Location: Search/preview/2024-04-01-preview/search.json#L403
:warning: PaginationResponse Operation might be pageable. Consider adding the x-ms-pageable extension.
Location: Search/preview/2024-04-01-preview/search.json#L418
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Search/preview/2024-04-01-preview/search.json#L455
:warning: ErrorResponse The error property in the error response schema should be required.
Location: Search/preview/2024-04-01-preview/search.json#L455
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Search/preview/2024-04-01-preview/search.json#L585
:warning: ErrorResponse The error property in the error response schema should be required.
Location: Search/preview/2024-04-01-preview/search.json#L585
:warning: PaginationResponse Operation might be pageable. Consider adding the x-ms-pageable extension.
Location: Search/preview/2024-04-01-preview/search.json#L600
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Search/preview/2024-04-01-preview/search.json#L652
:warning: ErrorResponse The error property in the error response schema should be required.
Location: Search/preview/2024-04-01-preview/search.json#L652
:warning: PaginationResponse Operation might be pageable. Consider adding the x-ms-pageable extension.
Location: Search/preview/2024-04-01-preview/search.json#L667
:warning: ErrorResponse Error schema should define code and message properties as required.
Location: Search/preview/2024-04-01-preview/search.json#L704
:warning: ErrorResponse The error property in the error response schema should be required.
Location: Search/preview/2024-04-01-preview/search.json#L704
:warning: SchemaDescriptionOrTitle Schema should have a description or title.
Location: Search/preview/2024-04-01-preview/search.json#L780
:warning: SchemaDescriptionOrTitle Schema should have a description or title.
Location: Search/preview/2024-04-01-preview/search.json#L1049
:warning: AvoidNestedProperties Consider using x-ms-client-flatten to provide a better end user experience
Location: Search/preview/2024-04-01-preview/search.json#L1074
:warning: SchemaDescriptionOrTitle Schema should have a description or title.
Location: Search/preview/2024-04-01-preview/search.json#L1102
:warning: SchemaDescriptionOrTitle Schema should have a description or title.
Location: Search/preview/2024-04-01-preview/search.json#L1108
:warning: AvoidNestedProperties Consider using x-ms-client-flatten to provide a better end user experience
Location: Search/preview/2024-04-01-preview/search.json#L1131
️⚠️Avocado: 1 Warnings warning [Detail]
Rule Message
:warning: MULTIPLE_API_VERSION The default tag contains multiple API versions swaggers.
readme: specification/maps/data-plane/readme.md
tag: specification/maps/data-plane/readme.md#tag-package-stable-2023-06-01
️️✔️SwaggerAPIView succeeded [Detail] [Expand]
️️✔️TypeSpecAPIView succeeded [Detail] [Expand]
️️✔️ModelValidation succeeded [Detail] [Expand]
Validation passes for ModelValidation.
️️✔️SemanticValidation succeeded [Detail] [Expand]
Validation passes for SemanticValidation.
️️✔️PoliCheck succeeded [Detail] [Expand]
Validation passed for PoliCheck.
️️✔️SpellCheck succeeded [Detail] [Expand]
Validation passes for SpellCheck.
️️✔️Lint(RPaaS) succeeded [Detail] [Expand]
Validation passes for Lint(RPaaS).
️️✔️PR Summary succeeded [Detail] [Expand]
Validation passes for Summary.
️️✔️Automated merging requirements met succeeded [Detail] [Expand]
Posted by Swagger Pipeline | How to fix these errors?

Swagger Generation Artifacts

️🔄ApiDocPreview inProgress [Detail]
Posted by Swagger Pipeline | How to fix these errors?

I don't see a baseline commit, so it's hard to know exactly what's new in this api-version. I also see several RPs in this review. Do you have any documentation that highlights what is new for us to review? We likely will not have time to review the entirety of this PR.

heaths avatar Jan 25 '24 23:01 heaths

Hi @danielthank! For review efficiency consideration, when creating a new API version, it is required to place API specs of the base version in the first commit, and push new version updates into successive commits. You can use OpenAPIHub to initialize the PR for adding a new version.
For more details refer to the wiki.

@heaths I will use this slides to introduce our new changes.

danielthank avatar Feb 01 '24 14:02 danielthank

Also, from the meeting, I see the confusion: you're treating each group of endpoints as a separate resource provider, which is atypical and confusing. You can put things in separate swaggers, but you should combine all swaggers as a single service in the top-level readme.md and ship together e.g., https://github.com/Azure/azure-rest-api-specs/blob/main/specification/keyvault/data-plane/readme.md#tag-package-75

heaths avatar Feb 02 '24 00:02 heaths

@heaths Thanks for the review! I have addressed all of the comments you left and just top-level readme.md is left now. I will consult with SDK members in our team and come up with a better solution. The following is the justification for the breaking changes. Please let me know what I need to do to get an approval.

Justification for breaking changes

  • common.json I am trying to follow Azure API guideline and fix the existing error response model. It won't cause breaking change because we have already been returning the error response that is aligned with the Azure API guideline.
  • cross version They are also caused by the error response model change. The error response for GA version APIs in production aligns with the error response model of Azure API guideline.

Justification for SpellCheck

  • The GeocodingResponse in the description is for referencing the model and the writer wrote it to make the docs more readable. I think it's better to leave it as it was.

danielthank avatar Feb 02 '24 06:02 danielthank

Let me try explain the problem of api-version.

case 1

User initiates with REST API /route/directions:asyncBatch?api-version=A. During polling, SDK would invoke /asyncBatch/operations/{operationId}?api-version=A (even if your operation-location header from /route returns api-version=B)

This means "AsyncBatchManagement" Swagger would support all api-versions from any of the service doing the async.

case 2

For now, your service does not have a resource model. So this does not apply. But when you have one, the reverse of case 1 would also happen.

User can start with REST API /asyncBatch/operations/{operationId}?api-version=A, which in the end (after LRO completed), could result in a resource in the other service e.g. /route/someresource?api-version=A

weidongxu-microsoft avatar Feb 02 '24 10:02 weidongxu-microsoft

Let me try explain the problem of api-version.

case 1

User initiates with REST API /route/directions:asyncBatch?api-version=A. During polling, SDK would invoke /asyncBatch/operations/{operationId}?api-version=A (even if your operation-location header from /route returns api-version=B)

This means "AsyncBatchManagement" Swagger would support all api-versions from any of the service doing the async.

case 2

For now, your service does not have a resource model. So this does not apply. But when you have one, the reverse of case 1 would also happen.

User can start with REST API /asyncBatch/operations/{operationId}?api-version=A, which in the end (after LRO completed), could result in a resource in the other service e.g. /route/someresource?api-version=A

Loop in @JeffreyRichter for api-version discussion.

weidongxu-microsoft avatar Feb 02 '24 10:02 weidongxu-microsoft

@weidongxu-microsoft Thank you for the review. Following the POST or DELETE LRO pattern guideline, we will

  1. Return the Operation-Location header with api-version the same as what users pass for job submission.
  2. The status monitor endpoint will be backward compatible, which means the client can replace the api-version in Operation-Location header with newer api-version and get the status and result successfully.

For case 2, considering we have already decided to use POST pattern, we won't get a resource model in the status monitor. It means SDK users will not pass any part of the result of status monitor to another function in the SDK.

danielthank avatar Feb 05 '24 03:02 danielthank

For case 2, the model which contains "outputBlobUrl" is kind of a versioned response already (just a very simple one, for now).

You can have v1 with only "outputBlobUrl" in e.g. route; and v2 with e.g. "outputBlobUrl" and "additionalProperty". Whether to return "additionalProperty" in the response would depends on the api-version that initiate the /asyncBatch/operations/{operationId}.

weidongxu-microsoft avatar Feb 06 '24 02:02 weidongxu-microsoft

@weidongxu-microsoft that's right. We have a model called AsyncBatchOperationResult in the swagger. There's nothing stopping us from extending the model in the future. image

danielthank avatar Feb 06 '24 06:02 danielthank

@heaths @weidongxu-microsoft @JeffreyRichter We are ready to merge this PR. Please let us know if there's anything we need to change. Thank you.

danielthank avatar Feb 06 '24 06:02 danielthank

Please fix SpellCheck CI.

Also review BreakingChange CI, verify they are expected or false alert. If expected, you need approval from breaking change board.

weidongxu-microsoft avatar Feb 06 '24 06:02 weidongxu-microsoft

@weidongxu-microsoft I have written justification of SpellCheck and BreakingChange in previous discussion.

danielthank avatar Feb 06 '24 16:02 danielthank

You must justify the breaking changes: https://github.com/Azure/azure-rest-api-specs/pull/27188/checks?check_run_id=21209757976

The entire service must version uniformly. Please update all swaggers so that they all share the same api-version value.

JeffreyRichter avatar Feb 06 '24 21:02 JeffreyRichter

For spellcheck, click the "how to fix" link. which points to https://github.com/Azure/azure-rest-api-specs/blob/main/documentation/ci-fix.md#spell-check

Basically, update cspell.json

weidongxu-microsoft avatar Feb 07 '24 01:02 weidongxu-microsoft

@pbrasil Please help to clarify the uniform api version concern raised by Jeffrey.

cc @jsedlak-microsoft

chcmsft avatar Feb 27 '24 08:02 chcmsft

@heaths @weidongxu-microsoft The breaking change pipeline fail is fixed. Can you help to review again and sign off if there are no further issue.

chcmsft avatar Apr 08 '24 04:04 chcmsft

New change after the last review

  • field linkedResource is replaced by outputStorageUrl. outputStorageUrl is the storage account blob storage url for storing the output result.

The entire service must version uniformly <= action raised by @JeffreyRichter The Azure Maps service comprises various category APIs, each maintaining its own API version. This situation has persisted for some time. While there is a plan to unify these versions, it will not be addressed in this PR.

chcmsft avatar Apr 08 '24 06:04 chcmsft

@weidongxu-microsoft all issues are addressed.

chcmsft avatar Apr 09 '24 09:04 chcmsft

@heaths I've addressed all the comments above. Do you have other concern?

chcmsft avatar Apr 09 '24 09:04 chcmsft

@jhendrixMSFT Can you help to merge? I've got weidong's approval

chcmsft avatar Apr 15 '24 02:04 chcmsft

@jhendrixMSFT Could you help merge this PR? We have got review board's approval.

danielthank avatar Apr 19 '24 00:04 danielthank

@JeffreyRichter has this been reviewed and approved? I don't see the necessary tags.

jhendrixMSFT avatar Apr 22 '24 14:04 jhendrixMSFT

@heaths was commenting in the APIView. Heath, are you ready to sign off on this PR?

JeffreyRichter avatar Apr 22 '24 19:04 JeffreyRichter

@JeffreyRichter I don't have time to re-review so if you guys are happy with it, go ahead and merge. I don't want to sign off since I didn't review, but I can't seem to dismiss my review. The top doesn't show I'm blocking:

image

And there's no dismiss option in the summary at the bottom like I've seen on other PRs:

image

The author can't dismiss it either, which I know is possible because people have dismissed mine before (for good or bad reasons).

Can you dismiss it? Otherwise, I can just click the "Approve" option from the last screenshot above with a note I didn't review it.

heaths avatar Apr 26 '24 20:04 heaths

To be clear, I'm only approving to unblock. Seems the option to dismiss my review is missing from this PR; though, I've seen it before. I just don't have time to re-review in full right now.

heaths avatar Apr 26 '24 21:04 heaths