Capability to copy (deep) nodes from one target to another target

[ThomasRooney]

trafficstars

The Actions Object has the ability to enable users to add/mutate/remove attributes on OpenAPI documents, but cannot handle the following scenarios:

1. Renaming paths items

I.e. given a path on $.paths["/item"], move it to `$.paths["/newitem"]:

From

paths:
  /item:
    summary: 'The root resource'
    get:
      summary: 'Retrieve the root resource'
      ...

To:

paths:
  /newitem:
    summary: 'The root resource'
    get:
      summary: 'Retrieve the root resource'
      ...

Doing this via a full add/remove of the /item requires inlining the schema into the overlay, and thus makes it brittle: any overlay which overrides a path item in this way effectively nukes the ability for a source document to change any of the information in "/item".

2. Extracting inline schemas into component schemas

Many OpenAPI frameworks produce inline json schemas for operations:

paths:
  /item:
    post:
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required:
                - name
                - photoUrls
              properties:
                id:
                  type: integer
                  format: int64
                name:
                  type: string

Enabling an overlay to extract that inline schema and replace it with $ref: #/components/schemas/Item would enable a category of simplifications for downstream tooling.

Proposed change to the Actions Object

Fixed Fields

Field Name	Type	Description
target	string	REQUIRED A JSONPath expression selecting nodes in the target document.
description	string	A description of the action. [[CommonMark]] syntax MAY be used for rich text representation.
update	Any	If the target selects an object node, the value of this field MUST be an object with the properties and values to merge with the selected node. If the target selects an array, the value of this field MUST be an entry to append to the array.
remove	boolean	A boolean value that indicates that the target object or array MUST be removed from the the map or array it is contained in. The default value is false.
destination	string	A JSONPath expression selecting destination nodes in the target document. If set, this indicates that each node in the target expression (after applying the update) should be appended as a child to the destination node. If not set, the target is mutated in-place.

This would enable scenario 1 via

actions:
   - target: $.paths
     update: 
       "/newitem": {}
   - target: $.paths["/item"]
     destination: $.paths["/newitem"]
     remove: true

And scenario 2 via something similar.

[lornajane]

This is a good use case! I think there might also be a wider consideration here about how to use parts of the OpenAPI description (in its current state at the time the action runs) as inputs to actions.

[kevinswiber]

There's a lot of overlap with https://github.com/OAI/Overlay-Specification/issues/30 and https://github.com/OAI/Overlay-Specification/pull/32.

[sjoubert]

Another similar use case to 1. (but with dynamic destination based on the target) would be to duplicate content entries for requestBody or responses. Consider an API that wants to support both JSON and YAML (input and/or output), currently we have to do:

paths:
  /item:
    post:
      requestBody:
        content:
          application/json:
            $ref: '#/components/schemas/Item'
          application/yaml:
            $ref: '#/components/schemas/Item'

But this means we may forget to add the yaml duplicate on new paths or methods. This would be nice to only specify one:

paths:
  /item:
    post:
      requestBody:
        content:
          application/json:
            $ref: '#/components/schemas/Item'

and overlay an action to duplicate all application/json into application/yaml at once.

[jeremyfiel]

Another use case for this was outlined in a slack convo. https://open-api.slack.com/archives/C023Y5YJ474/p1747407454482789

the jist is the ability to reuse target values in updates.

one of the big issues you may have is the ability to reuse a target. overlay's doesn't currently support that. So in the case of exclusiveMinimum / minimum or the maximum equivalent, there's no way to carry over those values. likewise, there's no way to repurpose the example values into examples.

In the context of OpenAPI conversion from 3.0.x. to 3.1+, there's no way to take the minimum value and repurpose it to the exclusiveMinimum value

openapi: 3.0.4
info:
  title: overlay test
  version: 0.0.1
paths:
  '/things':
    get:
      summary: a summary
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                type: number
                exclusiveMinimum: true
                minimum: 100
              example: 102

overlay: 1.0.1
info:
  title:
  version: 0.0.1
actions:
- target: $..[?(@.type == "number" && @.minimum && @.exclusiveMinimum == true)]
  description: 'any number schema with minimum and exclusiveMinimum: true'
  update:
    description: set the new value from the $target.minimum value
    exclusiveMinimum: $target.minimum
- target: $..[?(@.type == "number" && @.minimum && @.exclusiveMinimum == true)].minimum
  description: 'any number schema with minimum and exclusiveMinimum: true, remove minimum keyword'
  remove: true

expected result

openapi: 3.1.1
info:
  title: overlay test
  version: 0.0.1
paths:
  '/things':
    get:
      summary: a summary
      responses:
        '200':
          description: OK
          content:
            application/json:
              schema:
                type: number
                exclusiveMinimum: 100
              examples:
                min_example:
                  value: 102

[lornajane]

I re-titled this issue to cover the wider scope that's emerging from the discussion

[ThomasRooney]

Would love to have this in a 1.1.0 given it's non-breaking :)

[ThomasRooney]

Another similar use case to 1. (but with dynamic destination based on the target) would be to duplicate content entries for requestBody or responses. Consider an API that wants to support both JSON and YAML (input and/or output), currently we have to do:

paths: /item: post: requestBody: content: application/json: $ref: '#/components/schemas/Item' application/yaml: $ref: '#/components/schemas/Item' But this means we may forget to add the yaml duplicate on new paths or methods. This would be nice to only specify one:

paths: /item: post: requestBody: content: application/json: $ref: '#/components/schemas/Item' and overlay an action to duplicate all application/json into application/yaml at once.

I think we could support this pretty easily with some notion of an in-place-transform.

E.g. to go from

paths:
  /item:
    post:
      requestBody:
        content:
          application/json:
            $ref: '#/components/schemas/Item'

To:

paths:
  /item:
    post:
      requestBody:
        content:
          application/json:
            $ref: '#/components/schemas/Item'
          application/yaml:
            $ref: '#/components/schemas/Item'

Would be an action akin to (if using JQ syntax):

- target: $.paths.*.post.requestBody.content
  transform: '. + {"application/yaml": .["application/json"]}'

I think this would fit pretty nicely in addition to this proposal. It complicates the overlay file format but I think using destination to support both complicates the OP use-case.

[lornajane]

I'd like to propose an alternative approach to renaming, building from the discussions in #33 and #140 in support of this issue.

I'm not sure about using $ in the syntax so other suggestions are welcome!!

This example introduces a new syntax that can be use in the update value, that refers to the node that matched the target expression. This way, an existing part of an OpenAPI description can be used as an input when expressing the value to use when updating.

Rename a path

overlay: 1.1.0
info:
  title: Rename a path
  version: 1.0.0
actions:
  - target: '$'
    update:
      paths:
        '/newPet':
           $target.paths["/Pet"]
  - target: '$.paths["/Pet"]
     remove: true

Copy response example

(this snippet from an earlier example)

paths:
  /item:
    post:
      requestBody:
        content:
          application/json:
            $ref: '#/components/schemas/Item'
          application/yaml:
            $ref: '#/components/schemas/Item'

Could be achieved with an Overlay like this:

overlay: 1.1.0
info:
  title: Re-use a response for another content type
  version: 1.0.0
actions:
  - target: '$.paths["/item"].post.requestBody'
    update:
       content:
          application/yaml:
              $target.content["application/json"]

Feedback very welcome ...

[baywet]

Here is some alternative syntax from #146 proposal, updated for the example in the previous syntax

overlay: 1.1.0
info:
  title: Re-use a response for another content type
  version: 1.0.0
actions:
  - target: '$.paths["/item"].post.requestBody.content'
    update: {"application/yaml": true}
  - target: '$.paths["/item"].post.requestBody.content["application/yaml"]'
    update-from-source: '$.paths["/item"].post.requestBody.content["application/json"]'

Some more notes from the meeting: I've voiced a concern with coming up with a new syntax ($target) as A) it requires implementers to implement parsing of this new syntax instead of simply relying on the library they use for JSON Path B) it requires recursive "parsing" of the update object, which adds to the complexity. @lornajane mentioned that we don't need the added complexity of JSON Path to simply "do property access".