[Question] How to keep `components.schemas` when bundling openapi spec from multiple files?

[rucciva]

Hello!

First of all, thanks a lot for creating this library. it's awesome.

Currently i'm trying to follow https://pb33f.io/libopenapi/bundling/. It works when combining multiple files into single file, but i noticed that all components.schemas no longer exists and all references are inlined.

Is it possible to bundle multiple files into single file without inlining all references?

My use case is i plan to generate go code from the bundled single file using oapi-codegen but would like to avoid inline struct definition inside another struct, which happens when a schema contains object inside object as a side effect of inlining all references.

[daveshanley]

Hi,

Thanks for the compliments.

I have now had for this kind of a feature upgrade for the bundler a few times. Right now it simply inlines every single reference, regardless if they are local or external (because in reality, there is no concept, there is only a URI to a resource).

However the desire to suck in external references and re-compose the document to inline those references only, seems to be a popular one. Here is the same request in vacuum.

https://github.com/daveshanley/vacuum/issues/421

So the answer is no, it does not support that.... yet.

[rucciva]

Got it @daveshanley , thanks a lot for the answer.

[rucciva]

hi @daveshanley , i was curious and i've been playing with the Bundle function. i managed somehow to achieve what i wanted though not yet tested with enough specs.

i modified the function to be like this

func bundle(model *v3.Document, inline bool) ([]byte, error) {
	model.Components = &v3.Components{
		Schemas: orderedmap.New[string, *base.SchemaProxy](),
	}

	compact := func(idx *index.SpecIndex, root bool) {
		sequencedReferences := idx.GetRawReferencesSequenced()
		mappedReferences := idx.GetMappedReferences()
		for _, sequenced := range sequencedReferences {
			// if we're in the root document, don't bundle anything.
			refExp := strings.Split(sequenced.FullDefinition, "#/")
			if len(refExp) == 2 {
				if refExp[0] == sequenced.Index.GetSpecAbsolutePath() || refExp[0] == "" {
					if root && !inline {
						idx.GetLogger().Debug("[bundler] skipping local root reference",
							"ref", sequenced.Definition)
						continue
					}
				}
			}

			mappedReference := mappedReferences[sequenced.FullDefinition]
			if mappedReference == nil {
				continue
			}
			if mappedReference.Circular {
				if idx.GetLogger() != nil {
					idx.GetLogger().Warn("[bundler] skipping circular reference",
						"ref", sequenced.FullDefinition)
				}
				continue
			}

			ref := ""
			switch {
			case strings.HasPrefix(sequenced.Definition, "#/components/schemas"):
				ref = "#/components/schemas/" + sequenced.Name
				schema := &baselow.Schema{}
				schema.Build(context.Background(), sequenced.Node, sequenced.Index)
				model.Components.Schemas.Set(sequenced.Name, base.CreateSchemaProxy(base.NewSchema(schema)))
			}
			if ref == "" {
				continue
			}

			sequenced.Node.Content = base.CreateSchemaProxyRef(ref).GetReferenceNode().Content
		}
	}

	rolodex := model.Rolodex
	indexes := rolodex.GetIndexes()
	for _, idx := range indexes {
		compact(idx, false)
	}
	compact(rolodex.GetRootIndex(), true)

	// copy components into root node in case new references need to be resolved, e.g. reference inside `allOf`
	components, err := toYamlNode("components", *model.Components)
	if err != nil {
		return nil, fmt.Errorf("fail to convert components into `*node.Yaml`: %w", err)
	}
	for _, idx := range append(indexes, rolodex.GetRootIndex()) {
		idx.GetRootNode().Content = components.Content
	}

	return model.Render()
}

func toYamlNode(key string, v interface{}) (n *yaml.Node, err error) {
	b, err := yaml.Marshal(map[string]interface{}{
		key: v,
	})
	if err != nil {
		return nil, err
	}
	y := yaml.Node{}
	return &y, yaml.Unmarshal(b, &y)
}

so far it works with limitation that:

1. the reference name is unique (although i think adding prefix or postfix is possible to make it globally unique)
2. i need to structure the yaml of non-root file in a way that i now what kind of reference it is (schema, response, etc). for example if it schema then it needs to be under the following yaml node:

   components: 
       schemas: 
           ....
   

3. ~~when i do the following modification inside any children of PathItem inside non-root file, i encountered unable to locate reference anywhere in the rolodex~~ from:

   schema:
       $ref : "..."
   

   to:

   schema:
       allOf|anyOf:
           - $ref : "..."
   

my question is regarding no. 2. Can we somehow figure out whether a Reference is schema or response or any other kind? i know that the Reference has ParentNodeSchemaType but it somehow always hold empty string

p.s., the custom bundler is available in https://github.com/TelkomIndonesia/oapik?tab=readme-ov-file#bundle

[stcalica]

Having this issue when trying to resolve/merge/bundle the spec with something like the spec here: https://github.com/Dwolla/dwolla-openapi?tab=readme-ov-file It inlined all the references BUT CreateUnverifiedCustomerRequestBody and CreateVerifiedPersonalRequestBodyin this file.

I believe it inlined where it could but because the next set of references were inside it's own file it ignored it.

This could have been fine if it just added them as components instead.

[daveshanley]

I am working on this feature. A way to generate a 'composed' document, one where remote refs are pulls into local components, vs inlining over and over.

[daveshanley]

Composed bundling is now available in v0.22.0

https://github.com/pb33f/libopenapi/releases/tag/v0.22.0 https://pb33f.io/libopenapi/bundling/

Example: https://github.com/pb33f/libopenapi/blob/main/test_specs/nested_files/openapi.yaml

And the composed version output by the lib. https://github.com/pb33f/libopenapi/blob/main/test_specs/nested_files/openapi-bundled.yaml