Prevent `redocly bundle` from creating new schema objects in `#/components/schemas` when using `--dereferenced` option

[outdooracorn]

Describe the bug

The redocly bundle command is adding schema objects to the #/components/schemas section of the bundled schema, even when using the --dereferenced option is used.

Closed issue #239 seems to be similar and points to using the --dereferenced option to solve this issue.

To Reproduce Steps to reproduce the behavior:

1. Given this redocly.yaml file
2. And these OpenAPI files
3. Run npx redocly bundle schema/openapi.json --dereferenced -o out/openapi.json
4. See that PropertyValuePair and Error schema objects are added to #/components/schemas in the bundled OpenAPI document

Expected behavior

I would expect only (dereferenced versions of) the schema objects present in schemas.json to be in the #/components/schemas section of the bundled and dereference OpenAPI document.

Logs

OpenAPI description

OpenAPI v3.1 - see example repo

Redocly Version(s)

"@redocly/cli": "^1.25.11",

Node.js Version(s)

v18.20.4 and v20.12.2

OS, environment

Kubuntu 22.04

Additional context

[tatomyr]

Have you tried using the --remove-unused-components option?

[outdooracorn]

Have you tried using the --remove-unused-components option?

I have. It removes some, but not all, of the components. In this example, #/components/schemas/Statement is left.

I also don't want to remove all the components. I would like to keep #/components/schema so that it displays the schemas section in Swagger UI.

[tatomyr]

Ah, so you want to bundle your files, preserve the existing components, and not add the new ones? Then it's more of an enhancement rather than a bug. I'm not sure it would be an easy change. It will require modifying the internals, as the bundler currently resolves external references into the components section regardless of the --dereferenced option because the component might be used in an indirect way (e.g. in a discriminator). However, you can try using a custom plugin with a decorator to filter schemas that were only present in the original components. It should probably look like this:

const filterSchemas = () => {
  let originalSchemas;
  return {
    Components: {
      leave(components) {
        for (const key in components.schemas || {}) {
          if (originalSchemas?.[key] === undefined) {
            delete components.schemas[key];
          }
        }
      },
      NamedSchemas: {
        enter(schemas) {
          originalSchemas = schemas;
        },
      },
    },
  };
};

Please let me know if that works!

[outdooracorn]

Ideally, we would like to bundle the files and only dereference/inline schemas that aren't already in #/components. Said differently, keep references to components already in #/components, and dereference/inline everything else.

Thanks for the custom decorator plugin code, we will try it out.

[gnvk]

We're currently using swagger-cli to bundle our OpenAPI specifications and we're trying to migrate to redocly. This issue seems to be a blocker for us that we can't even really work around.

I created a minimal, reproducible example to demonstrate the issue: https://github.com/gnvk/redocly_example In this example we have 3 OAS specifications (stock, crypto, openapi). The bundled files are in the generated directory. The files bundled by swagger-cli use the schema definitions already in the original yaml, e.g. components.schemas.StockTrade, while redocly creates a new schema components.schemas.trade-2 (and also displays a warning).

We also use oapi-codegen to generate Go code from the bundled schemas. Actually this is the main reason to bundle them in the first place. The original, swagger-cli bundled files have nice schema tags that yield meaningful Go structs: stock.Trade (stock package, Trade struct, generated from stock.yaml), openapi.StockTrade (openapi common package, StockTrade struct, generated from openapi.yaml). On the other hand, the redocly bundled files have these strange trade, trade-2, symbol, symbol-2 schemas, which will be generated to openapi.Trade2.

We even tried custom decorators to work around this but without success. We'd be happy with either of these:

1. Use the already existing schema in the original file (like swagger-cli does)
2. We can remove those schemas, but then the generated schemas should have proper names. trade in stock.yaml but stock_trade in openapi.yaml. If the schema is unique, its name should be simply the filename, but if it's not, the directory name should be used instead of number postfixes (as recommended by this comment).

Can anyone here recommend a solution, or at least a workaround?

[tatomyr]

Hi @gnvk. I don't see an easy workaround for your problem. To achieve that, we'd need to change the current behaviour of Redocly CLI which might impact existing users. Have you considered renaming the duplicated files into something like StockTrade.yaml or crypto-trade.yaml, so you'll get the corresponding names in the components?

[gnvk]

Have you considered renaming the duplicated files into something like StockTrade.yaml or crypto-trade.yaml, so you'll get the corresponding names in the components?

Yes, in fact I even did all the renames, just to realise that that messes up the generated files for the server components. For example the trade in stock-trade.yaml will become stock.StockTrade in Go, which is not something we want.

Basically, I can either generate the correct openapi.yaml by renaming the component files, as you suggested, or I can generate the correct "small" YAMLs (e.g. stock.yaml) by leaving them as they're now. With swagger-cli I can achieve both using the named schemas, which are currently ignored by redocly. This is why I feel this issue is a blocker for us.

[tatomyr]

@gnvk I see. Then the only workaround I can suggest is running a decorator on the stock.yaml to remove the needless stock- prefixes in components and refs. It seems to be quite fragile though.

Just in case you want to explore that path, here's the idea. First, you have to create a plugin like the following:

// plugin.mjs

export default () => {
  return {
    id: "my-plugin",
    decorators: {
      oas3: {
        "remove-prefix": ({ prefix }) => ({
          ref: {
            leave(refObject) {
              // after partially resolving refs, we check if it still has a $ref field
              if (refObject?.$ref) {
                // ...and if it does, we remove the prefix
                refObject.$ref = refObject.$ref.replace(prefix, "");
              }
            },
          },
          NamedSchemas: {
            leave(schemas) {
              // replace all keys that start with the prefix with the new key in components.schemas
              for (const key in schemas) {
                if (key.startsWith(prefix)) {
                  const newKey = key.replace(prefix, "");
                  schemas[newKey] = schemas[key];
                  delete schemas[key];
                }
              }
            },
          },
        }),
      },
    },
  };
};

Then, use it use it in your configuration file:

# redocly.yaml

plugins:
  - plugin.mjs

apis:
  openapi:
    root: api/openapi.yaml
    output: generated/redocly/openapi.yaml
  stock:
    root: api/stock.yaml
    output: generated/redocly/stock.yaml
    # set up the custom decorator specific to the 'stock' api
    decorators: 
      "my-plugin/remove-prefix": 
        prefix: "stock-"

You don't need to declare the components section in your API files anymore but you have to rename the files to contain the prefixes to be unique (like stock-trade.yaml &c.).

On redocly bundle this configuration will produce the same generated openapi.yaml but it will update the component schema names that start with the stock- prefix when generating the stock.yaml.

Please let me know it that helps.

BTW, @gnvk's case is rather a variation of https://github.com/Redocly/redocly-cli/issues/661, and is not directly related to the original issue.

[gnvk]

Thank you @tatomyr! With a tiny modification in the plugin script (we also need to replace the NamedParameters) your workaround seems to be working in my example and also in our real code base (though it was a giant change: +7,822 −7,057).