URL Cache stores noMatch=bypass parameters

[Lagerregal]

There is an issue in class UrlEncoder, method "storeInUrlCache()". It puts all "originalUrlParameters" to the cache entry but should ignore the noMatch parameters.

This causes some errors: If you remove a parameter, an extbase controller parameter for example, from an url with the realurl config "noMatch=bypass", then realurl creates two (!) cache entries for the same URL.

For example this configuration:

$realurlConfig['fixedPostVars']['myPluginPage'] = [
    [
        'GETvar' => 'tx_myextbase_plugin[model]',
        'lookUpTable' =>[...]
    ],
    [
        'GETvar' => 'tx_myextbase_plugin[controller]',
        'noMatch' => 'bypass',
    ]
];

If you create a link to this page with or without the controller parameter, the URL will be the same. But we have two cache entries.

URL cache entry 1: speaking_url = my-page/my-extbase-model/ request_variables = {"id":"1","tx_myextbase_plugin[model]":"1","tx_myextbase_plugin[controller]":"MyController","cHash":"..."}

URL cache entry 2: speaking_url = my-page/my-extbase-model/ request_variables = {"id":"1","tx_myextbase_plugin[model]":"1","cHash":"..."}

Impact: Now we have the issue in the UrlDecoder if this URL is requested. The UrlEncoder expects one (!) cache entry, else it will check if the "L" paramter machtes the current language (which does not exist in this example). So no cache entry was found for this URL which ends up in a "cHash not found" Exception or other unexpected stuff. If you disable the "cHash not found" Exception the RealurlDecode also create a third (!) cache entry for this URL without the cHash.

[dmitryd]

No, realurl works correctly here. Its job to encode URLs as is according to the configuration. Your job is to make sure that you pass proper URLs. RealURL is not a toy. It does not forgive programmers' errors. Cache maps created URLs to a set of original parameters, so it is perfectly possibly that there are two URLs but different parameter sets if the developer is not careful. RealURL cannot help the developer with that.

No magic: you configured it like this, you provided the URL, you got the result. Do your job better and make sure you create only one set of URLs. Don't put your responsibilities to me :)

[pniederlag]

the job as a developer is to make life of other developers easy. Currently it is very easy to "f--- up" things with a broken realurl config.

@Lagerregal if you set noMatch => bypass then ANY VALUE of the parameter is dropped from the url. In general this does not make very much sense! Looks a little bit like you are trying to mimik the feature "skipDefaultArguments" of extbase plugins which allows to whipe controller and action from the url if they match the default value. hth

[dmitryd]

Currently it is very easy to "f--- up" things with a broken realurl config.

Well, as I said: realurl is not a toy. I believe it is possible to make a proper config using configuration reference. However if something is missing or unclear there, I am always open to improvement suggestions.

Regarding how people sometimes configure realurl, I often remember Weinberg's second law: “If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilisation.”

[dmitryd]

Looks a little bit like you are trying to mimik the feature "skipDefaultArguments" of extbase plugins which allows to whipe controller and action from the url if they match the default value. hth

Hmmm. This makes me think of a new postVar type: drop. If encountered on encoding, it will just do nothing (no segment created). If such type encountered on decoding, it will continue like it was not there. This way it will be easy to drop controller and action from the URL. The only problem than will be for Extbase to determine what's expected.

[dmitryd]

Keeping open because something has to be done about it... But no ideas yet what should be done.

[Lagerregal]

Hey, thanks for explaining me what I have to do as developer :-) Yeah such a drop feature would be great. Maybe there is a better solution for simple extbase list & detail plugins with really beautiful urls?

Such as "domain.com" = list action "domain.com/my-object/ = detail action for "My object"

If no "MyObject" parameter is given, use list action. Otherwise use detail action with the given parameter.

This behaviour is currently possible by hiding controller and action parameters via realurl "nomatch=bypass" and set the "default action = show" via typoscript, if a value for "MyObject" is given.

This way is a little bad... Does realurl provides a better solution for this "default usecase"?

[dmitryd]

Hey, thanks for explaining me what I have to do as developer :-)

Don't take it personally :) It is not me being arrogant or anything like that. The problem is that realurl way of doing things is very, very limited and it is very difficult to do it right. TYPO3 core does not offer any support and sometimes realurl has to go to desperate ways to get the information or recover form problems that arise from the lack of core support. Therefore realurl is very touchy about how people configure it and what kind of URLs they provide. There are all kind of use cases and ways people want realurl to work. Often meeting one request means breaking the other. So realurl (and I) have to find some middle ground that will serve well enough to everybody.

Does realurl provides a better solution for this "default usecase"?

Unfortunately, realurl does not. It only encodes and decodes URLs as they are given by the TYPO3 core. It does not know about Extbase, Fluid, PiBase, whatever. It gets /index.php?id=1&L=3&tx_something[param]=77439 and transforms it to something like /en-us/resume/john-appleseed/, that's all. It cannot add, or alter, or remove parameters. It is plain 1:1 translation from one form to another. It is caller's responsibility to make the original URL so that it translates to speaking URL nicely.

What you can do, is to:

1. Make links without controller/action
2. Add controller/action using TS conditions and config.defaultGetVars

For example, if you have a link like /index.php?id=123&tx_yourext[uid]=5678, you can add controller action this way:

[globalString = GP:tx_yourext|uid = /\d+/]
config.defaultGetVars.tx_myext {
  action = single
  controller = MyController
}
[global]

You can add that to your ext's TS setup. In such case users of your ext will not have to do it in their files. You only need to generate links without controller and action arguments.

[Lagerregal]

Hm, yes that's the way I configured it already. But I think this mix of configurations is pretty uncool:

1. removing parameters when the link is builded via the View (remove parameters in Fluid)
2. the url rewriting via realurl
3. add the parameters again in TypoScript

I understand how hard life can be as open source developer and have great respect for your work, tanks for realurl! But such a feature I described above would be really helpful :-) Would you merge a feature like that if I create a pull request or is this function outside of the realurl scope?

[dmitryd]

This will be difficult to implement correctly for both encoding and decoding. drop is easy but if the entry is not in the cache (like cache is cleared), than decoder will not be able to recreate the parameter.

Possible solution:

• On encoding check that the value in parameters strictly matches configured value, if yes - remove, if no - add to url according to other transformations (lookUpMap, noMatch, etc). This makes it checked before all others.
• On decoding... Here we have a problem. Either the value was removed or not. Not easy to guess. All transformations have to be performed in order to understand if it is the right value. And still it will be ambiguous...

Decoding is always the biggest problem. Given the url /path/to/page/news/news/detail/my-news-item/category/corporate-news we have to understand in some way that it is logically the same as /path/to/page/news/my-news-item/category/corporate-news or even the same as /path/to/page/news/detail/my-news-item/category/corporate-news. And after you process the path and find the postVar prefix, you have /news/detail/my-news-item/category/corporate-news. So how do you know if the first news is a controller name or a news item title? You can check forward but this is very complicated because you have to check every possible combination of transformations according to cconfig (process lookUpMaps, conditions, noMatches, valueDefaults, etc) and possibly even other postVars and may be their transformations to be sure.

See, what I mean by complex?