CLOUDFLAREAPI: FEATURE REQUEST: Support Cloudflare Redirects (via Product Rules instead of Page Rules)

[pro-sumer]

It's great that DNSControl supports redirects via Cloudflare Page Rules using CF_TEMP_REDIRECT and CF_REDIRECT!

Unfortunately we only get 3 rules on free plans. Cloudflare recently introduced Redirects via Product Rules (currently in beta) with more relaxed limits.

It would be great if you can support these new redirects as well.

[tlimoncelli]

Sounds like a useful thing to support. Can you suggest a name (like CF_BULK_REDIRECT()) and a syntax?

[pro-sumer]

Haven't experimented with this new functionality yet...

(the 3 Pages Rules were exactly enough so far and I have those in my DNSControl config file)

[hmoffatt]

The bulk redirects are account-wide, not per domain. It seems tricky for dnscontrol to manage them on this basis. (You also need a business or enterprise subscription to use wildcards which makes them a bit limiting).

The single redirects look like a useful alternative to page rules here though as they're per domain. You can match on a lot more than source URL though so the syntax would be complicated; you'd need to be able to specify a whole match expression (eg (http.request.uri.path eq "/*" and ip.geoip.country eq "AF")) and the target.

[aglasson]

I believe the below image is what OP is talking about. These seem to have been intended to supplanted page rules while page rules have retained their ability to do URL redirects.

My suggestion, without fully understanding the potential impacts or underlying Cloudflare intentions, is to change both CF_TEMP_REDIRECT and CF_REDIRECT to manage the new rule types instead. Obviously there needs to be some conversion handling process or manual intervention. A hypothetical conversion approach might be:

CF_TEMP_REDIRECT("foo.bar.co/*", "https://foofoo.bar.co/", {'cloudflare_ruletype_redirect': 'true'})`

You can match on a lot more than source URL though so the syntax would be complicated

While the new rules have a lot more functionality I don't see any reason why DNSControl needs to make use of any of the additional rule functionality, stick to doing only what it does now. I don't think DNSControl has ever intended to be a complete Cloudflare management system, if a cloudflare user wants more than simple redirects, manage those outside of DNSControl.

[tlimoncelli]

PAGE_RULES are going away: https://developers.cloudflare.com/rules/reference/page-rules-migration/

[pro-sumer]

PAGE_RULES are going away

Thank you for notifying! I'm going to convert my three page rules into redirect rules (in the Cloudflare web interface).

I will be a bit sad that I lose the "Configuraton-as-Code" that DNSControl provided, so I would like it if my feature request gets implemented at some point, but I would totally understand it if it gets rejected for being outside the scope of DNSControl.

[tlimoncelli]

I use a lot of redirects, so I don't plan on losing this functionality.

My plan is to investigate how we could have CF_TEMP_REDIRECT generate the same functionality with the new system.

I haven't worked out the details yet, but I hope to start working on this soon.

[allixsenos]

@tlimoncelli other kinds of rules are explicitly out of scope?

[tlimoncelli]

@allixsenos I'm not sure. I won't know until I start looking at the code.

To be honest, I'm not sure if I'm going to be able to work on this by the time page rules are disabled for free accounts (1-July-2024). If someone else has more time, I'd love help!

[tlimoncelli]

@jzhang-sre and I are working on this now. We hope to ship it in a release in time for people to convert.

[tlimoncelli]

I'm about to merge a PR that supports this. Sadly I'll be on holiday and without computer access between now and when Cloudflare ends support for Free-Tier customers (June 30, 2024). I'd rather get the (potentially buggy) code in people's hands than leave them completely stranded.

Hopefully the code will be stable by the time paid customers (July 31, 2024) lose support for Page Rules.

[pro-sumer]

Finally had time to try this, but all the patterns in my rules are (currently?) not supported?

domain.tld/.well-known/* -> https://social.domain.tld/.well-known/$1
domain.tld/users/* -> https://social.domain.tld/users/$1
domain.tld/@* -> https://social.domain.tld/@$1

So they are currently still manual changes in the Cloudflare dashboard, but I hope DNSControl can support these.

These are the "When" expression on the dashboard:

(starts_with(http.request.uri.path, "/.well-known") and http.host eq "domain.tld")
(starts_with(http.request.uri.path, "/users") and http.host eq "domain.tld") 
(starts_with(http.request.uri.path, "/@") and http.host eq "domain.tld") 

And they all have the same "Then" expression:

concat("https://social.domain.tld", http.request.uri.path)

[pro-sumer]

Additionally, I'm getting these errors for domains that don't have any redirects:

ERROR
Error getting corrections (cloudflare): failed fetching redirect rule list cloudflare: could not find entrypoint ruleset in the http_request_dynamic_redirect phase (10003)

[pro-sumer]

Would it be an idea to introduce a new CF_SINGLE_REDIRECT statement with 3 parameters?

• rule name
• "When" expression (which users can build & copy from the Cloudflare dashboard)
• "Then" expression (which users can build & copy from the Cloudflare dashboard)

(Handling " characters in expressions may be tricky? Or can they just be escaped?)

[tlimoncelli]

Finally had time to try this, but all the patterns in my rules are (currently?) not supported?

domain.tld/.well-known/* -> https://social.domain.tld/.well-known/$1
domain.tld/users/* -> https://social.domain.tld/users/$1
domain.tld/@* -> https://social.domain.tld/@$1

So they are currently still manual changes in the Cloudflare dashboard, but I hope DNSControl can support these.

These are the "When" expression on the dashboard:

(starts_with(http.request.uri.path, "/.well-known") and http.host eq "domain.tld")
(starts_with(http.request.uri.path, "/users") and http.host eq "domain.tld") 
(starts_with(http.request.uri.path, "/@") and http.host eq "domain.tld") 

And they all have the same "Then" expression:

concat("https://social.domain.tld", http.request.uri.path)

Could you provide the "when" and "then" expressions it should generate? I'll update the code as appropriate.

Thanks!

[pro-sumer]

Could you provide the "when" and "then" expressions it should generate? I'll update the code as appropriate.

I already kind of mentioned them in my reply (but the “When” expressions were not completely matching, so I hope I made things consistent here):

Rule 1

• Rule:

domain.tld/.well-known* -> https://social.domain.tld/.well-known$1

• When:

(starts_with(http.request.uri.path, "/.well-known") and http.host eq "domain.tld")

• Then:

concat("https://social.domain.tld", http.request.uri.path)

Rule 2

• Rule:

domain.tld/users* -> https://social.domain.tld/users$1

• When:

(starts_with(http.request.uri.path, "/users") and http.host eq "domain.tld") 

• Then:

concat("https://social.domain.tld", http.request.uri.path)

Rule 3

• Rule:

domain.tld/@* -> https://social.domain.tld/@$1

• When:

(starts_with(http.request.uri.path, "/@") and http.host eq "domain.tld") 

• Then:

concat("https://social.domain.tld", http.request.uri.path)

Observations

• All three rules seem to have the same pattern now, so only a single conversion needs to be added? (which makes sense since my entire goal is to redirect to the subdomain “social”)
• Maybe http.host can be used instead of the hardcoded “domain.tld” in “Then” expressions?

[tlimoncelli]

Thanks for spelling out the 3 cases. I created test cases for all three.

Yes, they are similar enough that I was able to cover all 3 with the same if/then.

Take a look at https://github.com/StackExchange/dnscontrol/pull/3024 and let me know if that works for you.

I'm going to take a stab at CF_SINGLE_REDIRECT if I have time this weekend.

[tlimoncelli]

I fixed @pro-sumer 's bugs and also found many more during deployment at stackoverflow.com. New PR: https://github.com/StackExchange/dnscontrol/pull/3031

A point release will go out tonight.

[pro-sumer]

Sorry, was too busy to check until now.

Seems to work fine with DNSControl version 4.12.1!

I don't like the long names, so looking forward to your CF_SIMPLE_REDIRECT implementation.

[tlimoncelli]

Glad it worked!

There will be a 4.12.2 soon to fix one more situation we found.

Yes, I don't like the long names either. It was a quick hack to make sure each name was unique. CF_SIMPLE_REDIRECT will let you pick your own name.

[tlimoncelli]

CC @cafferata

Hey @pro-sumer et all. Please check out https://github.com/StackExchange/dnscontrol/pull/3035 as it implements CF_SINGLE_REDIRECT, which lets you write any "when" rule and "then" expression.

[pro-sumer]

@tlimoncelli Am I using CF_SINGLE_REDIRECT wrong, or is there a bug in this new implementation in version 4.12.3?

See https://github.com/StackExchange/dnscontrol/issues/3039