api icon indicating copy to clipboard operation
api copied to clipboard

Add regex rewrite

Open SpecialYang opened this issue 4 years ago • 28 comments

In our gateway case, we definitely need this awesome feature to meet so many path routes. Maybe EnvoyFilter can support regex rewrite for data plane. But we are tired to name these routes that require regular rewriting. Only if we pick meaningful names for these routes, EnvoyFilter can match these route names and apply regex rewrite policy. Adding regex rewrite api for rewrite in HTTPRoute is the easy way to fix this case. I think someone needs this feature like me.

Based on this pr https://github.com/istio/api/pull/1566, introduce RegexMatchAndSubstitute which may be used in the future for similar cases. @howardjohn @hzxuzhonghu

Fixes https://github.com/istio/istio/issues/22290 and https://github.com/istio/istio/issues/24172.

SpecialYang avatar Jan 27 '21 15:01 SpecialYang

🤔 🐛 You appear to be fixing a bug in Go code, yet your PR doesn't include updates to any test files. Did you forget to add a test?

Courtesy of your friendly test nag.

istio-policy-bot avatar Jan 27 '21 15:01 istio-policy-bot

😊 Welcome @SpecialYang! This is either your first contribution to the Istio api repo, or it's been awhile since you've been here.

You can learn more about the Istio working groups, code of conduct, and contributing guidelines by referring to Contributing to Istio.

Thanks for contributing!

Courtesy of your friendly welcome wagon.

istio-policy-bot avatar Jan 27 '21 15:01 istio-policy-bot

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

google-cla[bot] avatar Jan 27 '21 15:01 google-cla[bot]

Hi @SpecialYang. Thanks for your PR.

I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

istio-testing avatar Jan 27 '21 15:01 istio-testing

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

google-cla[bot] avatar Jan 27 '21 15:01 google-cla[bot]

@googlebot I consent

SpecialYang avatar Jan 27 '21 15:01 SpecialYang

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

google-cla[bot] avatar Jan 27 '21 15:01 google-cla[bot]

I am personally fine with this, but wanted to echo @costinm concern: https://github.com/istio/api/pull/1566#issuecomment-767631005

At least we should bounce the idea with the K8S Services WG - and the other projects planning to support the K8S APIs. I assume since this didn't make it to 1.9 we can afford few weeks to discuss with the relevant WGs.

howardjohn avatar Jan 27 '21 15:01 howardjohn

lgtm

At least we should bounce the idea with the K8S Services WG

Is this the service-apis?

hzxuzhonghu avatar Jan 28 '21 02:01 hzxuzhonghu

I am personally fine with this, but wanted to echo @costinm concern: #1566 (comment)

At least we should bounce the idea with the K8S Services WG - and the other projects planning to support the K8S APIs. I assume since this didn't make it to 1.9 we can afford few weeks to discuss with the relevant WGs.

Same question. Why is it related to service APIs?

The regex rewrite policy for uri is basic feature in most gateway products. Our gateway envoy get runtime configs from istiod via xds protocol. We must apply many EnvoyFilter configs and pick up meaningful names for target routes. We want an easy way just like configuring the regex rewrite policy directly in HTTPRoute, which is clear.

As I know, current istiod use RE2 for all regex match cases. I think we can also use it in regex rewrite until istiod support different regex styles.

I'm sure many users will look forward to this feature. Thanks.

SpecialYang avatar Jan 28 '21 13:01 SpecialYang

Great. adding regular-based rewriting is necessary.

gengleilei avatar Jan 29 '21 10:01 gengleilei

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

google-cla[bot] avatar Feb 01 '21 03:02 google-cla[bot]

I am personally fine with this, but wanted to echo @costinm concern: #1566 (comment)

At least we should bounce the idea with the K8S Services WG - and the other projects planning to support the K8S APIs. I assume since this didn't make it to 1.9 we can afford few weeks to discuss with the relevant WGs.

Same question. Why is it related to service APIs?

The regex rewrite policy for uri is basic feature in most gateway products. Our gateway envoy get runtime configs from istiod via xds protocol. We must apply many EnvoyFilter configs and pick up meaningful names for target routes. We want an easy way just like configuring the regex rewrite policy directly in HTTPRoute, which is clear.

As I know, current istiod use RE2 for all regex match cases. I think we can also use it in regex rewrite until istiod support different regex styles.

I'm sure many users will look forward to this feature. Thanks.

@howardjohn @costinm PTAL. Thanks.

SpecialYang avatar Feb 01 '21 14:02 SpecialYang

Is this done? i desire this function!

wyler-max avatar Mar 01 '21 11:03 wyler-max

All (the pull request submitter and all commit authors) CLAs are signed, but one or more commits were authored or co-authored by someone other than the pull request submitter.

We need to confirm that all authors are ok with their commits being contributed to this project. Please have them confirm that by leaving a comment that contains only @googlebot I consent. in this pull request.

Note to project maintainer: There may be cases where the author cannot leave a comment, or the comment is not properly detected as consent. In those cases, you can manually confirm consent of the commit author(s), and set the cla label to yes (if enabled on your project).

ℹ️ Googlers: Go here for more info.

google-cla[bot] avatar Mar 01 '21 11:03 google-cla[bot]

@SpecialYang: The following test failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
release-notes_api 156fd06d0dae3dab1ed67f980c3a557f302ec850 link /test release-notes_api

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

istio-testing avatar Mar 01 '21 11:03 istio-testing

:frowning_face: Sorry, but only Googlers may change the label cla: yes.

google-cla[bot] avatar Mar 01 '21 11:03 google-cla[bot]

Is this done? i desire this function!

cc @howardjohn @hzxuzhonghu @costinm Is there any latest feedback here?

SpecialYang avatar Mar 01 '21 11:03 SpecialYang

@SpecialYang: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

istio-testing avatar Mar 04 '21 00:03 istio-testing

@howardjohn will this ever see the light? I opened https://github.com/istio/istio/issues/22290 one year and a half ago :(

kmai avatar Aug 03 '21 22:08 kmai

@SpecialYang @howardjohn @hzxuzhonghu Anyone working on this PR?

fatedier avatar Sep 26 '21 07:09 fatedier

@SpecialYang @howardjohn @hzxuzhonghu Anyone working on this PR?

I am working on this. But the community seems don't accept this api change yet. We add this feature in our internal version of istio and run in production. Hope community can rethink about this very common feature to fix this long-alive issue https://github.com/istio/istio/issues/22290.

SpecialYang avatar Sep 26 '21 08:09 SpecialYang

I would recommend giving feedback on https://github.com/kubernetes-sigs/gateway-api/pull/731, as Istio will implement this API.

howardjohn avatar Sep 27 '21 16:09 howardjohn

I would recommend giving feedback on kubernetes-sigs/gateway-api#731, as Istio will implement this API.

What's the relationship between gateway-api and istio virtual service? Do you mean that virtualservice would not implement complicated features, users who need these features should use gateway-api instead?

fatedier avatar Sep 28 '21 02:09 fatedier

What's the relationship between gateway-api and istio virtual service? Do you mean that virtualservice would not implement complicated features, users who need these features should use gateway-api instead?

The gateway-api can be seen as the next evolution of the Istio APIs (in the future), with a common API shared by many proxy implementations. By getting this feature into that API it would make it an obvious choice to also add it to VirtualService

howardjohn avatar Sep 28 '21 15:09 howardjohn

@SpecialYang @howardjohn I'm very sorry for mentioning, but what is the current status?

The process of discussion/implementing this feature seems not transparent at all. :( Is this PR not merged due to conflicts or there's some internal maintainers decision that this wouldn't be merged due to some ideological considerations? Or maybe you are just waiting for approval of https://github.com/kubernetes-sigs/gateway-api/pull/731 proposal?

dm3ch avatar Sep 29 '21 01:09 dm3ch

I can't believe this feature isn't in istio... why hasn't this 1 yo PR not been merged into the project yet?

dacloutier avatar Oct 25 '21 12:10 dacloutier

@howardjohn Is the necessary gateway api fields been added? If not should we just go ahead and add this support. This seems pretty reasonable request and some of our service owners are also asking for this feature.

ramaraochavali avatar Apr 15 '22 09:04 ramaraochavali

@SpecialYang , @istio-policy-bot , @googlebot . What happened in the end with this pull, are they going to implement it or not?

gabomarf avatar Dec 19 '22 14:12 gabomarf

@googlebot I consent

gabomarf avatar Feb 07 '23 14:02 gabomarf

🤔 🐛 You appear to be fixing a bug in Go code, yet your PR doesn't include updates to any test files. Did you forget to add a test?

Courtesy of your friendly test nag.

istio-policy-bot avatar Feb 07 '23 17:02 istio-policy-bot

Can you review this PR, please ?

@dcberg @duderino @linsun @louiscryan @nrjpoddar @smawson @istio/technical-oversight-committee

gabomarf avatar Feb 14 '23 18:02 gabomarf

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

istio-testing avatar Mar 13 '23 22:03 istio-testing