filtering unused components - not working

[connorsadler]

Hi,

I am trying to filter a yaml openapi spec, to remove a path and remove unused schema components. The path filtering works fine, but the unused component removal does not work - unless I deliberately add an unused component to the original file (I did this by accident while experimenting)

Please see attached zip file for sample files and README.md with steps to reproduce issue I will also copy in README.md below

There is always the possibility that I am doing something wrong - apologies if that is the case here.

Thanks,

Connor

Mac OS openapi-format version: tried with 1.14.0 and 1.17.0 - same results

PROBLEM: The 'remove unused components' doesn't work - it leaves some items in there that it shouldn't

Steps to show problem:

1. edit spec: pet-sample-openapi-spec.yaml observe it has 2 paths, each path references a response schema object which uses a second schema object
2. filter the spec by running: ./dev-produce-filtered-spec.sh this uses "dev-produce-filtered-spec_customFilter.yaml" which only includes one of the paths (WORKS) but should removed unused "schemas" (DOESNT WORK)
3. edit result: pet-sample-openapi-spec_filtered.yaml observe it includes only one path (correct), but all the schema objects are included - some of which are unused so should have been removed
4. edit spec 2: pet-sample-openapi-spec-2.yaml observe it is the same as the first spec, but it has an unused component in schemas: ThisTypeIsNeverReferencedAnywhere
5. filter the spec 2 by running: ./dev-produce-filtered-spec-2.sh Note: In the output it says it's removed some unused components
6. edit result: pet-sample-openapi-spec_filtered-2.yaml observe the filtering has now worked - one path and all unused components have been removed

Archive.zip

[thim81]

hi @connorsadler

Thanks for providing the detailed reproduction steps.

When I find some time, I'll try to reproduce the behavior based on your archive.zip and steps.

I'll share update once we have them.

[connorsadler]

Thanks We can work around it for the moment by just including a dummy unused item in the original spec, which we are in control of

[thim81]

hi @connorsadler

Thanks to your clear reproduction steps and archive with your openapi files & config, I did some rework on the unused component tracking and removal logic. This improved unused component handling is included in the openapi-format v1.20.0 release.

You can see the result of your provided example in the openapi-format playground

or use your npx openapi-format pet-sample-openapi-spec.yaml -o pet-sample-openapi-spec_filtered.yaml --no-sort --filterFile dev-produce-filtered-spec_customFilter.yaml --verbose

(if you want to open your local config in the online playground, just add --playground npx openapi-format pet-sample-openapi-spec.yaml -o pet-sample-openapi-spec_filtered.yaml --no-sort --filterFile dev-produce-filtered-spec_customFilter.yaml --verbose --playground)

[thim81]

After validation, feel free to provide your feedback. If the issue is resolved, you can close this item.

[thim81]

hi @connorsadler

Did you have some time to try out the latest version? To see if your issue is resolved?

[wsocha]

Hi @thim81, I'm using [email protected] and I'm facing similar issue. When I want to filter out one path and all its related components, some of the schemas are not removed. Running filter again on the first run result eventually removes all the unused components.

example.zip

You can find sample spec with two paths and filter definition in the attached archive. The filter specify one path to be removed with operationId pets/getPets. Running yarn openapi-format api.json -o result.json --filterFile filter.yml command does not remove Pets schema, though.

[thim81]

hi @wsocha

Thanks for providing this clear case. I've used your example in the online playground

The Pets component should indeed also be removed:

I'll look into this case and see if we can find the cause of this.

[thim81]

hi @wsocha

I released 1.22.5 of openapi-format, which has improvements to handle the unused components.

The result for your example in the online playground:

Let me know, if the unused removal works as expected, so I can close the issue or explore other options.

[wsocha]

Hi @thim81! Now the filter works as expected and removes all unused components. Thank you very much for fixing the issue and releasing the fixed version swiftly.

[thim81]

hi @wsocha

Glad to see it finally solved, we did several iterations to improve the "unused" detection and removal. It is mostly because of the recursive nature, that is if you remove something, a component becomes unused, which gets removed, which causes other components to become unused ... ➿

I'll close this issue, if somebody else experiences filtering issues with unused components, let's start a new issue.