Redirected page layout differs from manual navigation layout

[Col-E]

Reproduction

Using in-private or clean install of Firefox:

1. Google <anything> site:reddit.com and click on a page that has a dozen or more comments
2. Observe that the redirect works, but you see the "More from /r/whatever" after three comment trees, and none of the comment trees are expanded by default
3. Observe that going to the URL bar and pressing [Enter] reloads the page and presents a layout without the "More from /r/whatever" and all comment trees are expanded by default
   • Reloading or pressing [F5] does not achieve the same effect. You will still see the same layout that includes the "More from /r/whatever".
   • This behavior is consistent with and without the "Reddit Enhancement Suite" extension enabled.
   • Any page that has been manually navigated to via the URL bar retains the proper layout.
   • Any page that has not been visited before shows the "More from /r/whatever" and collapsed comment trees.

Screenshot of observed behavior by initial redirect: firefox_t6m0nlEYYr.png

Screenshot of observed behavior after manual URL navigation: firefox_4S6cTdlCgR.gif

Expectation

The redirect should result in the same display as if you manually navigated to the old. domain.

[skippybosco]

I see from your screenshots that you are not logged into Reddit.

1. Do you see this behavior if you are logged in?

2. Do you see this behavior if you manually navigate to old.reddit.com or follow an existing bookmark to a URL or does it only happen when following a Google search link?

Reddit has recently implemented some server side enhancements that impact users in a logged out state that are beyond this plugins ability to override.

[Col-E]

1. When I am logged in I have my user preference set to display reddit in the old style. While signed in, if I click on a link that directs me to www.reddit.com then I get to the desired outcome. This applies with or without the use of this extension.
2. If I make a regular <a href="https://www.reddit.com/r/whatever/bla_bla_bla">link</a> and click on it, I get the desired outcome. The link is the same as what appears on google's search results (where I copied it from) but the referrer seems to affect the outcome.

Reddit has recently implemented some server side enhancements that impact users in a logged out state that are beyond this plugins ability to override.

This may be true in some circumstances, but I do not see those being relevant here. While I am logged out, if I navigate to a www.reddit.com link via the URL bar and press enter, I get the desired outcome. The difference is when the link is clicked from google, per the answers to your prior to questions.

In fact, if I do window.location.replace("<reddit-link>") on any arbitrary page then I get the desired outcome... unless I do it from google.com. If I do the same thing but I'm currently on google.com then the undesired behavior occurs.

[skippybosco]

This may be true in some circumstances, but I do not see those being relevant here.

The behavior you're describing appears to be server side changes on Reddit outside the control of this plugin (or any plugin). There have been a variety of flavors of this issue recently, for example:

https://github.com/tom-james-watson/old-reddit-redirect/issues/145

The issue appears that Reddit is making server side choices that impact user behavior for logged out sessions including manipulation of the URL as well as injecting internal ads (ie. "more from)..

What you describe sounds like they are testing ways to generate better tracking and conversion to user goals from search referrals, unfortunately outside the control of the plugin. The only known means by which to offset this behavior currently is to be logged in to Reddit.

[Col-E]

server side changes on Reddit

Even if this is the case, its only doing it for certain refering domains. When I said that I didn't see it being relevant here I meant that the circumstances that lead to this appear to be within the extension's capability to control, and thus work around. The circumstances leading to said changes are consistent. That implies you should be able to mitigate the issue if you can change those circumstances. It seems like that should be possible by swapping out the referer header so that reddit is never aware that the origin of the request is coming from a search engine such as google.com. The SO post seems to imply that you can change that value. I'm aware that this does not align with documentation about forbidden headers. I would like to see if its possible myself, but I don't know how to install an unsigned Firefox extension (Even with the about:config value xpinstall.signatures.required disabled) to dig more into the issue.

Further points indicating this is a Referer: www.google.com (or other search engine) based issue:

• In about:config changing Network.http.sendRefererHeader from 2 (default) to 0 also results in the desired outcome
  • Same for changing network.http.referer.XOriginPolicy from 0 (default) to 1
• Using an additional extension such as this one and deleting the Referer header from *.reddit.com/* requests also results in the desired outcome

[skippybosco]

I'll let @tom-james-watson chime in here, but header modifications, etc seems out of scope of the intent of this extension, and likely not fully compatible with the spirit of manifest 3. It is especially fragile given it is a hack to try and address a server side change that may be an A/B and/or temporary change by Reddit as it becomes a game of cat and mouse trying to keep up with the various tests they may implement.

[tom-james-watson]

It looks like with manifest V3 it should still be possible to remove a header:

https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/declarativeNetRequest/ModifyHeaderInfo#type

So in theory, it would be possible to prevent this behaviour. I'd not be against including this in the extension. It's similar to how we modify headers to prevent other anti-user behaviour like stopping returning HTML instead of raw images. I also have noticed this behaviour and find it annoying.

A PR would be welcome.

[skippybosco]

A PR would be welcome.

At this point would a PR just be for manifest 3 or would you want it implemented in v2 as well to be accepted?

[tom-james-watson]

Just V3. See https://github.com/tom-james-watson/old-reddit-redirect/issues/134.

[skippybosco]

I've looked into this a bit and it seems it will require a background service worker. That seems to be how other similar extensions are doing it as well.

Can see here:

https://github.com/requestly/modify-headers-manifest-v3/tree/master/dist

It appears a bit more complication and overhead given the edge case (ie. redirect from google when not logged in) in terms of the goals of the extensions scope to maintain simplicity.

That said, if the door is open for background service workers, then other options like click icon to disable/enable and other type capabilities are a consideration.

[tom-james-watson]

I just had a stab at this. Going off the docs, it looks like a new rule like this should work:

  {
    "id": 9,
    "priority": 2,
    "action": {
      "type": "modifyHeaders",
      "requestHeaders": [{ "header": "Referer", "operation": "remove" }]
    },
    "condition": {
      "urlFilter": "^https://old.reddit.com/*",
      "resourceTypes": ["main_frame"]
    }
  }

but it doesn't seem to have any effect.

The project you linked is just dynamically declaring a set of rules, but we don't need to do that - we just have a static json file.

If someone else wants to have a go at getting this to work and working out what's wrong with my rule, then a PR would be awesome.

[skippybosco]

I looked at a few different extensions that were stripping referrer and they all made use of service workers, they indicated it was a timing issue, but I didn't look further into it.

In your code is the referrer being removed before the page loads?

[tom-james-watson]

If someone can get this working and provide a PR, that would be awesome

[Adhjie]

I can't make PR but, does the referrer stripping could be implemented like how ClearUrls both extension/addon and userscript remove tracking elements from (strip any marketing parameters/referrers? [IDK, the exact terms]) the main URL?

Edit: Just wanted to ask, because ClearUrls has the ability to remove parts of the URL. Though, IDK, how different the usages are besides that and this extension.

[tom-james-watson]

It's not the URL that needs to change in this case - we just need to strip a header.