Optimize the wheel file contents for Lite

[whitphx]

The following files included in the wheel are not needed for Lite.

• gradio/_frontend_code
• gradio/templates
• gradio/test_data
• gradio/node
• gradio/*.pyi

[jpatters]

Instead of introducing a new key (regex_paths) I think it would be nice to add some consistency to the apis and use StringMatch like in VirtualService and Gateway. ie:

paths:
- exact: /v1
- regex: "\/.*/docs\/.*"
- prefix: /v2

[edgel]

This feature will be very useful to define the authz policy, without this feature the only way that I can image is to use the Mixer Policy with OPA (it is powerful but seems not a recommend way), so I suggest AuthorizationPolicy to support another header matcher mode (istio/pilot/pkg/security/authz/model/matcher/header.go):

• /exact: for exact string mach
• */suffix: for suffix string match
• prefix/*: for prefix string match
• *: for non-empty string match
• |^/regex/.*$|: for regex string match

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: httpbin-1
  namespace: foo

spec:
  selector:
    matchLabels:
      app: httpbin
      version: v1
  rules:
    - from:
        - source:
            principals: ["principal", "principal-prefix-*", "*-suffix-principal", "*"]
            requestPrincipals: ["requestPrincipals", "requestPrincipals-prefix-*", "*-suffix-requestPrincipals", "*"]
            namespaces: ["ns", "ns-prefix-*", "*-ns-suffix", "*"]
            ipBlocks: ["1.2.3.4", "5.6.0.0/16"]
      to:
        - operation:
            methods: ["method", "method-prefix-*", "*-suffix-method", "*"]
            hosts: ["exact.com", "*.suffix.com", "prefix.*", "*"]
            ports: ["80", "90"]
            paths: ["/exact", "/prefix/*", "*/suffix", "*", "|^/regex/.*$|"]

[GODBS]

This becomes important in Istio 1.5 now that the alpha Authentication Policy is being replaced with the Request Authentication and Peer Authentication. There is no other way to exclude paths for JWT then to use an Authorization Policy which does not allow regex.

[incfly]

Hmm indeed this seems a feature regression... I confirmed we don't have regex in the beta API (both req/authz). @yangminzhu you must know more on this. We do not include regex in the authz api is that for a specific reason or due to our negligence?

[yangminzhu]

yes, we dropped the regex support in the beta RequestAuthentication and didn't add it in the AuthorizationPolicy as we're not sure how many users will actually need the regex and what's the API should be.

Instead of introducing a new key (regex_paths) I think it would be nice to add some consistency to the apis and use StringMatch like in VirtualService and Gateway. ie:

paths:

• exact: /v1
• regex: "/./docs/."
• prefix: /v2

@jpatters thanks for the suggestion, we ever considered this a little bit but @liminw has some concerns that this will make the policy quite redudant.

/exact: for exact string mach /suffix: for suffix string match prefix/: for prefix string match : for non-empty string match |^/regex/.$|: for regex string match

@edgel this seems a more easier and incremental way to support the regex, but we need to think about the detail syntax.

@diemtvu @liminw It looks like many users are actually wanting the regex match in authZ policy, could you share some thoughts here, maybe we could support it like this: paths: ["/exact", "/prefix*", "*-suffix", "*", "regex(...)"]?

[myidpt]

We usually don't make a feature request as a P0. Previsouly this was an experimental feature and we don't expect users to use it in production.

[nak3]

+1 Knative on Istio have to use triggerRules.excludedPaths.{exact,prefix} for authentication policy as knative service gets health checking and metrics collection from external(knative system pods). So this feature is really necessary. Cloud Run's solution also suggested it here for now.

[incfly]

use triggerRules.excludedPaths.{exact,prefix} for authentication policy

new authorization policy has not_paths since 1.5 for negative match @nak3

[lgarrett-isp]

We usually don't make a feature request as a P0. Previsouly this was an experimental feature and we don't expect users to use it in production.

@myidpt

What is the recommended solution for the following REST pattern of sub-resources?

Suppose I build a CRUD REST service for Countries. Each country has a sub-resource City. Following REST convention, I believe this means paths like /countries /countries/{country id} /countries/{country id}/cities /countries/{country id}/cities/{city id}

I have a rule to ALLOW requests to /countries/MyCountry* if request.auth.claims["countries"] contains "MyCountry".

Some users who are authorized for /countries/MyCountry* are NOT authorized to POST cities (they are read-only users). My natural thought is to create a rule to DENY POST for /countries/*/cities/* when request.auth.claims["readonly"] is true.

I am not an expert REST developer but I had thought this was an extremely common pattern--can you help me understand why this pattern is uncommon (what are the better patterns for subresources?), or what led to the conclusion that users would not use this regex support for this purpose in production?

[DanSalt]

I am not an expert REST developer but I had thought this was an extremely common pattern--can you help me understand why this pattern is uncommon (what are the better patterns for subresources?), or what led to the conclusion that users would not use this regex support for this purpose in production?

@lgarrett-isp

Can't speak for everyone, but it's definitely the same issue for us. We've followed what we believe is the correct REST convention, and find ourselves unable to express this within the policy. We're trying to replace our custom Envoy-based sidecar with a full Istio implementation, but the lack of this support means we'll likely end up with some kind of hybrid, which is unfortunate. We definitely need this for Production use-cases.

[lgarrett-isp]

Yep, we have run head-first into the same; we are moving away from Istio for our REST services until it has more support for basic REST conventions.

[howardjohn]

@yangminzhu is this solved by https://github.com/istio/istio/issues/27984?

[yangminzhu]

@yangminzhu is this solved by #27984?

yes, but we're evaluating if glob matching would be good enough for most use cases.

---

@wereeder @jpatters @edgel @nak3 @lgarrett-isp @DanSalt @ajit-hybris @aguromano @ajitchahal @mcieplak @rolandkool

I'm wondering would the glob matching good enough for your use cases? It looks like most examples (e.g. \/.*swagger.* or /countries/*/cities/*) posted here can be supported by glob matching. I feel there should be some use cases that requires the full regex but not sure detail examples, do you have any examples for this? Or you think the glob matching is indeed good enough? Thanks

[anannaya]

@yangminzhu For My header is be like x-auth-request-groups : 'CN=EMSUDEMY,OU=Groups,O=cco.xxx.com, CN=all-xxxx-people,OU=xxxx Groups,O=cco.xxxx.com, CN=xxxxxxxxx,OU=xxxx Groups,O=cco.xxxx.com . I would like match *all-xxxx-people* Glob matching should be fine for my usecase.

[williamaronli]

there seems a discussion about this in the community concerning about the security risk @yangminzhu do you have any updates for this regex supporting?

[mstrYoda]

Is there any update for this issue? Will you support glob matching?

[previousdeveloper]

+1

[carnei-ro]

Hey guys, I'm running Istio 1.9.0. I tried to implement the countries/cities example:

Allow POST to /v1/countries/{countries_id} Deny POST to /v1/countries/{countries_id}/cities/{city_id}

  action: ALLOW
  rules:
      to:
        - operation:
            methods:
              - POST
            paths:
              - /v1/countries/*
            notPaths:
              - /v1/countries/*/cities/*

I can not make the example work as expected. I could find this: curl -X POST http://iam.consumer--account/v1/countries/baz/cities/foo: the request hits my service (NOT the behavior I was expecting) curl -X POST http://iam.consumer--account/v1/countries/*/cities/foo: the request is blocked by Envoy. It seems the "wildcard" is only accepted at the end.

IMHO The issue should remain open... It is not possible to implement "REST" policy in this way :(

Is it possible to allow writing a condition like safe_regex of envoy?

[mstrYoda]

Hey guys, I'm running Istio 1.9.0. I tried to implement the countries/cities example:

Allow POST to /v1/countries/{countries_id} Deny POST to /v1/countries/{countries_id}/cities/{city_id}

  action: ALLOW
  rules:
      to:
        - operation:
            methods:
              - POST
            paths:
              - /v1/countries/*
            notPaths:
              - /v1/countries/*/cities/*

I can not make the example work as expected. I could find this: curl -X POST http://iam.consumer--account/v1/countries/baz/cities/foo: the request hits my service (NOT the behavior I was expecting) curl -X POST http://iam.consumer--account/v1/countries/*/cities/foo: the request is blocked by Envoy. It seems the "wildcard" is only accepted at the end.

IMHO The issue should remain open... It is not possible to implement "REST" policy in this way :(

Is it possible to allow writing a condition like safe_regex of envoy?

Actually it is possible to write regex based path matching in envoy. We can write regex path matching as below in envoy:

Under the hood Istio uses almost the same configuration, all wildcard matchings uses regex under the hood: https://github.com/istio/istio/blob/master/pilot/pkg/security/authz/matcher/string.go#L29

It is very simple to support regex in AuthorizationPolicy. We can specify path matching regex as a field of custom resource or we can give path with prefix like "regex: /some/path/*/".

I can create a pr for that if Istio maintainers consider to support it.

[shaneqld]

I've run into this problem while migrating from Istio on GKE (1.4.10) to 1.9.1. In my case, the application required JWT auth on /v1beta1/forms/* except for a few "custom methods", e.g. /v1beta1/forms/*:submit.

In this case, /v1beta1/forms/*:submit is a subset of /v1beta1/forms/*, so glob matching isn't ideal. Personally I like to be explicit and I think regex is needed for that.

I like the proposal raised by @edgel as it doesn't require changing the API like using StringMatch would.

With regards to the format of the string, some thoughts:

• I assume something as suggested like |regex| or regex: regex would be very unlikely to break existing rules.
• Because wildcards can currently only be used at the start or end of strings, * could possibly be used to signify a matcher mode, give room for extensibility and not cause backwards compatibility problems. For example: r* prefix for regex (r*^/my/.*/regex$) or g* prefix for glob (g*/my/*/glob).

[ognjen-it]

+1 bump

It would be good if we implement regex!

[LEDfan]

Could this be reopened? I still see the value of this feature and would like to use it. Thanks!

[yangminzhu]

reopened the issue for more discussions and also unassigned myself as I'm not actively working on this.

[mstrYoda]

@yangminzhu Hi, I am volunteer to implement this feature, we really need it on production, can you assign it to me? Also we can work with volunteers who wants to be pair.

[yangminzhu]

@mstrYoda sorry for the late reply, just assigned the issue to you, thanks for helping on this!

[johnbuluba]

Hello, We are currently blocked by this issue as we have the need to match header values using regex.

@mstrYoda Is there any progress on this? Is there any way we can help solving this issue?

Also there seems to be another issue regarding the same subject [#25021], which one is the main issue that tracks the progress? Maybe we can close one of them as duplicate (cc @yangminzhu).

Thanks

[mstrYoda]

@johnbuluba Thank you for reaching me, I will create a pr for this soon, we heavily need this in production too :)

[yangminzhu]

@mstrYoda thanks for the help. Just want to let you know we tried to solve this before but there are some difficulties that we do not have good solutions, you might need to think about these (or write a short proposal) before sending the actual PR:

1. it's hard to support this without changing the API, technically we could introduce special syntax to represent the regex in the existing field but we want to avoid that as it is not backward compatible and generally a bad user experience in reusing the same field for different purposes.

2. some general concern about the regex's security and usability, most users probably only need wildcard matching.

[mstrYoda]

@mstrYoda thanks for the help. Just want to let you know we tried to solve this before but there are some difficulties that we do not have good solutions, you might need to think about these (or write a short proposal) before sending the actual PR:

1. it's hard to support this without changing the API, technically we could introduce special syntax to represent the regex in the existing field but we want to avoid that as it is not backward compatible and generally a bad user experience in reusing the same field for different purposes.
2. some general concern about the regex's security and usability, most users probably only need wildcard matching.

Hi @yangminzhu actually I just thinking to support it with same field by "regex:" prefix but like I said, it might bad user experience. I agree with u, probably supporting only wildcard matching is what actually needed here (to support child resource paths for restful apis).

[magoon]

Please add wildcard matching within the string, to augment the existing support for prefix/suffix matching. We just need to be able to handle routes like /order/*/details. I know there are a lot of requests for regular expressions, which would be nice, but please don't let that stop us from having basic wildcard support to Authorization Policy Rules.