save-analytics-from-content-blockers

save-analytics-from-content-blockers copied to clipboard

It looks like GA made some changes to the JS on 17th Feb and this script in the current form no longer works.

It alters one GA with an incomplete record and makes it unusable so maybe worth considering for anyone yet to roll out.

Hello! Thank you for reporting this.

It alters one GA with an incomplete record and makes it unusable so maybe worth considering for anyone yet to roll out.

Do you know exactly what piece of code in that JavaScript stopped from being replaced? Any fix in mind? I'm a bit busy nowadays and will be really glad if you could debug & PR for instance.

Line 330 seems to be the offending one.

var KA={};var LA={};LA[S.cg]="cc";LA[S.dg]="ci";LA[S.eg]="cm";LA[S.fg]="cn";LA[S.gg]="cs";LA[S.hg]="ck";var BA=function(){var a="www";HA&&GA&&(a=GA);return"https://"+a+".google-analytics.com/*(Zw)*/*(Y29sbGVjdA)*"},AA=function(){var a;HA&&GA&&(a=GA);return"https://"+(a?a+".":"")+"mydomain.com/xxx/*(YW5hbHl0aWNzLmdvb2dsZS5jb20)*/*(Zw)*/*(Y29sbGVjdA)*"},

The original line of which:

var KA={};var LA={};LA[S.cg]="cc";LA[S.dg]="ci";LA[S.eg]="cm";LA[S.fg]="cn";LA[S.gg]="cs";LA[S.hg]="ck";var BA=function(){var a="www";HA&&GA&&(a=GA);return"https://"+a+".google-analytics.com/g/collect"},AA=function(){var a;HA&&GA&&(a=GA);return"https://"+(a?a+".":"")+"analytics.google.com/g/collect"},

No fix but then again, i've not really been able to check much in to this.

It appears this instance of google-analytics.com in the code needs a specialContentReplace due to the prefix it uses of https://"+a+"..

Can anyone suggest and test a fix please? I have no time to do it right now :(

Line 330 seems to be the offending one.

var KA={};var LA={};LA[S.cg]="cc";LA[S.dg]="ci";LA[S.eg]="cm";LA[S.fg]="cn";LA[S.gg]="cs";LA[S.hg]="ck";var BA=function(){var a="www";HA&&GA&&(a=GA);return"https://"+a+".google-analytics.com/*(Zw)*/*(Y29sbGVjdA)*"},AA=function(){var a;HA&&GA&&(a=GA);return"https://"+(a?a+".":"")+"mydomain.com/xxx/*(YW5hbHl0aWNzLmdvb2dsZS5jb20)*/*(Zw)*/*(Y29sbGVjdA)*"},

The original line of which:

var KA={};var LA={};LA[S.cg]="cc";LA[S.dg]="ci";LA[S.eg]="cm";LA[S.fg]="cn";LA[S.gg]="cs";LA[S.hg]="ck";var BA=function(){var a="www";HA&&GA&&(a=GA);return"https://"+a+".google-analytics.com/g/collect"},AA=function(){var a;HA&&GA&&(a=GA);return"https://"+(a?a+".":"")+"analytics.google.com/g/collect"},

No fix but then again, i've not really been able to check much in to this.

So we should replace this line and it will work again?

Not so much replacing the line, but the re-writing is not working so until that matching is fixed, it won't work.

This backend basically is capable of finding and replacing something in the script content via regexp. If you can find what to replace or fix, you can add a rule here (for extra cases), or add a path masking here, which applies to all scripts. Probably, you can remove path masking if it conflicts and replaces it with a "specialContentReplace" rule.

Nothing i've tried works so we've switched to using nginx as a proxy. Although a loss of user demographics, at least we have some stats now.

@iammattmartin is there anything preventing you from using dataunlocker.com?

Mainly our pref is keep things like this in-house so we're not reliant on a third party service being up.

Can someone explain what to do so I can write a fix and make PR? This issue is 1month+ old and there is no statistics for this period

Can someone explain what to do so I can write a fix and make PR? This issue is 1month+ old and there is no statistics for this period

1. Find a problem and how to fix it by changing the code
2. Fix it & submit PR
3. In PR description, note how to test the fix (since there are no automated test coverage)

I will be happy to review it. Thanks!

@VityaSchel we've started to use https://github.com/fr0der1c/google-analytics-proxy. Geo data does not come through but at least we have stats.

@iammattmartin do you know why Geo data does not go through the mentioned proxy? Did you find something better in the meantime?

EDIT: maybe due to Google not seeing the real user IP, so it considers the proxy IP. Wondering if there is a way to pass this to Google though :/

But I don't understand why since this line https://github.com/fr0der1c/google-analytics-proxy/blob/master/nginx.conf#L63

Explained in: https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#uip

Maybe you can try an updated proxy https://github.com/joshdk/google-analytics-proxy but I'm not sure it works with GA4 out of the box

@iammattmartin do you know why Geo data does not go through the mentioned proxy? Did you find something better in the meantime?

EDIT: maybe due to Google not seeing the real user IP, so it considers the proxy IP. Wondering if there is a way to pass this to Google though :/

But I don't understand why since this line https://github.com/fr0der1c/google-analytics-proxy/blob/master/nginx.conf#L63

Explained in: https://developers.google.com/analytics/devguides/collection/protocol/v1/parameters#uip

Maybe you can try an updated proxy https://github.com/joshdk/google-analytics-proxy but I'm not sure it works with GA4 out of the box

uip is ignored in GA4. I have no idea how DataUnlocker gets GA to see real user IP's.

It isn't and does work but requires a bit more work than in the scope of this project.

I've had mine working with uip and GA4 for a few months now.

It isn't and does work but requires a bit more work than in the scope of this project.

I've had mine working with uip and GA4 for a few months now.

It is neither documented nor working on our end

It does work but I don't have time to submit a PR and doc the change.

Er, your server IP doesn't happen to be recognized as UK? Because that's the situation we get just setting headers X-Forwarded-For and X-Real-Ip on backend and that's an awful lot of your users in one location.

uip is not documented and a Google analytics team member did say on the feature request for a way to add IP addresses on the GA4 MP that GA4 has no way to override user IP currently.

When it was using stock code, it was only showing the UK. After our changes, it now shows other countries.

The stats are correct though, this is a specific site for UK audience and matches our matomo stats.

When it was using stock code, it was only showing the UK. After our changes, it now shows other countries.

The stats are correct though, this is a specific site for UK audience and matches our matomo stats.

It's _uip not uip which isn't documented anywhere I could find, and analytics.google.com doesn't consistently respect it if at all while www.google-analytics.com does for certain. Both websites will show data in GA if you send the same parameters though.

Thanks but this is going off topic entirely and as I am not longer affected by this I am closing it as the repo owner isn't going to fix from the past convo.

@iammattmartin hey! PRs are welcome! It should be actually the one-liner PR adding a parameter here: https://github.com/dataunlocker/save-analytics-from-content-blockers/blob/494e0233c08e093cf3a51575c6b992f454f8d651/config.js#L83

Just make sure you test the change.

Sorry thats what I mean, this issue wasn't about that and got taken over by others adding stuff unrelated.

This issue was about the proxy breaking for what was reported at the start of the issue.

Got it. Do you have any fix in mind?

lmao this issue is looped basically needs one simple pr but no one wants to do it

It's not a case that no one wants to do it, some have tried but it doesn't work. Given you yourself said no one wants to do it but then asked what to do, seems pretty rich claiming no one wants to do it.