Feature: Advanced Filter Core Plugin

[mathis-m]

trafficstars

Advanced Filter Core Plugin

Preview

https://user-images.githubusercontent.com/11584315/105450159-30d59000-5c7a-11eb-8ba7-51b597f5dbf0.mp4

Introduction

Reflect common filtering, like we all know from IDE's.

const ui = SwaggerUIBundle({
    advancedFilter: {
      enabled: true
    },
    ....

The main focus of this PR is to implement a base for filtering that can be extended in an easy way using the plugin system.

Documentation

• Configuration
• Plug Points
• Types
  • AdvancedFilterConfiguration
  • MatcherOptions
  • Matchers
  • BaseMatcherState

Configuration

The advanced filter plugin can be configured using the global SwaggerUI configuration. To do so you can override the AdvancedFilterConfiguration defaults (see below), using the key advancedFilter.

advancedFilter: {
  phrase: "",
  enabled: false,
  matcherOptions: {
    matchCase: true,
    matchWholeWord: false
  },
  matchers: {
    operations: {
      isActive: true
    },
    tags: {
      isActive: true
    },
    definitions: {
      isActive: true
    }
  }
}

Plug Points

The Advanced Filter fully integrates into the swagger plugin system.

Modify spec selectors

In order to apply the filtered spec the advanced filter plugin overrides spec selectors:

...
statePlugins: {
  advancedFilter: {
  ...
  },
  spec: {
    selectors: {
      taggedOperations: (state) => ({ getSystem }) => {
        const { advancedFilterSelectors } = getSystem()
        if (advancedFilterSelectors.isEnabled() && advancedFilterSelectors.getPhrase() !== "") {
          const filteredSpec = advancedFilterSelectors.getFilteredSpec()
          state = state.set("resolvedSubtrees", filteredSpec)
          state = state.set("json", filteredSpec)
        }
        return taggedOperations(state)(getSystem())
      },
      definitions: (state) => ({ getSystem }) => {
        const { advancedFilterSelectors } = getSystem()
        if (advancedFilterSelectors.isEnabled() && advancedFilterSelectors.getPhrase() !== "") {
          const filteredSpec = advancedFilterSelectors.getFilteredSpec()
          state = state.set("resolvedSubtrees", filteredSpec)
          state = state.set("json", filteredSpec)
        }
        return definitions(state)
      },
    },
  },
}
...

In this way future matcher results can be reflected back into the spec state selection process too.

MatcherOption Conventions

MatcherOptions keys are mapped to a components via convention below, e.g. MatcherOption_matchCase:

MatcherOption_{key}

These components should renderer the options state. In case of a two-state state a toggle button could be used.

Matcher Conventions

In order to maintain the isActive state each matcher must have a component named via convention below, e.g. MatcherSelectOption_operations (the keys of Matchers are used):

MatcherSelectOption_{key}

Matcher fn

Each matcher has a corresponding fn to generate the filtered subset of the current spec for its context. The fn has following naming convention, e.g. advancedFilterMatcher_operations:

advancedFilterMatcher_{key}

The fn gets called with these arguments (spec, options, phrase, system) and should return a Map containing the subset of the filtered spec.

Creating a matcher plugin

Example for matching operation summary:

const escapeRegExp = (string) => {
  return string.replace(/[.*+?^${}()|[\]\\/]/g, "\\$&") // $& means the whole matched string
}
const opSummaryPlugin = (system) => ({
  components: {
    MatcherSelectOption_operationSummary: ({ getComponent, matcherKey }) => {
      const MatcherSelectOption = getComponent("MatcherSelectOption", true)
      return (
        <MatcherSelectOption matcherKey={matcherKey} label="operation summary" />
      )
    }
  },
  fn: {
    advancedFilterMatcher_operationSummary: (spec, options, phrase, { getSystem }) => {
      const system = getSystem()
      const expr = system.fn.getRegularFilterExpr(options, escapeRegExp(phrase))
      if (expr) {
        return system.fn.getMatchedOperationsSpec(
          (ops) => ops.map(path => path
            .filter(op =>  expr.test(op.get("summary")))
          ),
          spec, system,
        )
      }
    }
  }
})
const ui = SwaggerUIBundle({
  advancedFilter: {
    enabled: true,
    matchers: {
      operationSummary: {
        isActive: true
      }
    }
  },
  plugins: [opSummaryPlugin],
  ....

Types

AdvancedFilterConfiguration

This is the public interface model for configuring the advanced filter.

Property	Type	Default	Description
phrase	string	""	This is the state for the filter phrase. By default it is a empty string. Empty string will result in not filtering the spec.
enabled	boolean	false	By default the advanced filter is not enabled. When set to true the advanced filter components are rendered and filtering logic is executed.
matcherOptions	MatcherOptions	see type	This is the plug point for configuring matching behavior. This is the state that will be considered by each matcher.
matchers	Matchers	see type	This is the plug point to register matchers. A matcher is able to filter the current OpenAPI Specification, while respecting the matcher options provided.

MatcherOptions

This is a dictionary to store the state of each matcher option. Each key will be used to evaluate the corresponding component (see conventions).

Property	Type	Default	Description
matchCase	boolean	true	By default the matchers will match case-sensitive. If set to false it will rely on regex flag ignore case.
matchWholeWord	boolean	false	If set to true the matchers will only match full words. The matching logic is based on regex and will wrap the escaped phrase with \b.

Matchers

This is a dictionary to store the state of each matcher. Each key will be used to evaluate the corresponding matcher select option component (see conventions). This is the plug point to register matchers. A matcher is able to filter the current OpenAPI Specification, while respecting the matcher options provided. Each matcher has a context e.g. operations matcher - will match the operations and will return updated subset of the spec. The subsets returned by all matchers will be deep assigned with each other. The new filtered spec will be provided to the state in the advancedFilter namespace.

Property	Type	Default	Description
operations	BaseMatcherState	true	The operations matcher is capable of matching operation paths. In order to keep the spec clean it will add all tags of the filtered operations to the partial spec result. In addition it will add all top level requestBody and response schemas of the filtered operations to the partial spec result(to #/definitions or #/components/schemas). The filtered spec should include all matching opartions, only used tags and only the models that are used in the operations.
tags	BaseMatcherState	true	The tags matcher matches is capable of matching tags. The filtered spec will include all operations with a matching tag, all matching tags and all models / schemas used in filtered operations.
definitions	BaseMatcherState	true	The definitions matcher matches is capable of matching models / schemas by name / title. The filtered spec will include all models / schemas that have matching name / title.

BaseMatcherState

This is the base for all matchers.

Property	Type	Default	Description
isActive	boolean	see individual matcher state	This will enable filtering for the matchers context if set to true.

Current todo list:

• [x] @tim-lai plugin architecture review

Relations

#6744 Fixes #6648

[mathis-m]

@tim-lai build failing due to bundle size.

[anunay1]

When will this be available?

[alexrejto]

I am looking for this exact feature!

[mathis-m]

UX review is needed. After this I will refactoring accordingly and tests will be written.

[alexrejto]

Will this advanced filter be compatible will swagger-ui-react?

[mathis-m]

@alexrejto

Will this advanced filter be compatible will swagger-ui-react?

Yes since it will be released as core plugin it will be provided to swagger-ui-react too. I will ensure to add the possibility to configure it through <SwaggerUI advancedFilter={...} />

[wasdJens]

Hey, this feature looks really promising :) Are there any plans when this will be made available?

[captainkw]

hi @tim-lai just curious if this feature will be merged any time soon? My company uses Swagger for our apidocs and would very much appreciate this feature!

[captainkw]

hi @tim-lai just following up again on this PR. This would be a super nice feature that would be awesome for my current company to use. Multiple engineers have been asking for this.

[jstarrdewar]

@tim-lai seconding @captainkw . Our swagger docs are getting too long/complex to browse. It's now taking a decidedly nontrivial amount of time to hunt for a specific endpoint and trying out the preview above it is exactly what we need to solve the problem.

[tim-lai]

Hi all, there's a couple of tasks to complete to get this PR merged.

• [x] product review - OK
• [x] architecture review - OK
• [ ] UX review - pending confirmation, but tentative ok
• [ ] swagger-ui-react config enabled
• [ ] docker config enabled
• [ ] tests: overall feature can be toggled,
• [ ] tests: pattern matching combinations against one or more filter options
• [ ] documentation: migrate the excellent PR documentation to its own project file(s)

[mathis-m]

Hi all could someone or possible some of you review the documentation?

[mathis-m]

@char0n should I start working on this again? Would it be possible to release it with the 4.x as default?

[char0n]

Hi @mathis-m,

@char0n should I start working on this again? Would it be possible to release it with the 4.x as default?

This is quite a big change and a lot of invested work (great work). The scope of v4 was established before the effort on v4 started. v4 is limited to evolution of dependency tree and bug-fixes. More into about this in pre-release article.

IMHO the most appropriate time to restart this effort will be after the release/4.x branch lands on master.

[talkohavy]

Is there anything we regular folks can do to help?

P.S. Me personally (and i'm guessing many many other developers as well) just wanted a case-insensitive matcher, but boy... you really out done yourself! This is some fine fine work! I absolutely love it!

[captainkw]

hi can we get this merged already?

[ayushjaipuriyarht]

Please get this merged, this is very much needed.

[renatocron]

pretty please? 👉👈

[joaofrca]

sending some good vibes here and also wishing this gets merged too soon 🤞 ❤️

[mathis-m]

OP here,

I have stepped back from doing PRs at swagger due to time constraints...

It is pretty out of sync with current code and probably needs again some work.

As well as working on open points mentioned here by maintainers: https://github.com/swagger-api/swagger-ui/pull/6851#issuecomment-857228698

Anyone can try to rebase this onto latest code and start working to finish my work up.

@char0n Any chance it will be considered to be merged, in the near future, once some tackles it and finishes the work?

[Mathias02]

This appears to be a branch merge that falls under the responsibility of maintainers if I am not mistaken.

[mathis-m]

@robert-hebel-sb @glowcloud is there any interest from SmartBear to get this integrated?

Why am I trying to reactive this? Currently it's more efficient to filter the raw open api schema than using swagger ui for most of the commercial / big player apis available out there that provide open api schemas.

Yes => I will pick it up again, get it running on latest version, within next weeks

No => Close PR

(Please escalate/assign this internally if you feel you can't give an answer)

[MichakrawSB]

Hi @mathis-m , we sincerely apologize for the situation.

We’ve been making significant changes to our internal processes recently, and we hope these improvements will make the experience more satisfying for you and help prevent situations like this in the future.

This PR is now in the review queue with top priority, but because we’re in the middle of major changes to Swagger Editor (see: https://swagger.io/blog/swagger-editor-a-new-era-begins/), it may still take us a few more weeks to complete the review.

Once again, we’re truly sorry for the delay, and we hope that as we improve on our side, we can continue to count on your support and contributions to the community.

Cheers, Swagger Team