brave-browser Add refcode to select P3A metrics

STAR/Constellation has been deployed in production for quite some time, which allows us to add a refcode the select P3A metrics while maintaining privacy guarantees.

A refcode field will be added as the last STAR layer for some metrics, which will assist in retention analysis efforts.

cc @chrismore

Jan 12 '24 23:01 DJAndries

@chrismore @laurentiugruia @timchilds thoughts on which questions to add?

Feb 05 '24 22:02 DJAndries

@chrismore @laurentiugruia @timchilds thoughts on which questions to add?

We decided on this initial list:

Brave.Core.PagesLoaded2 Brave.Core.PagesReloaded Brave.Core.IsDefault Brave.Shields.UsageStatus Brave.Importer.ImporterSource.2 Brave.Core.LastTimeIncognitoUsed Brave.IOS.IsLikelyDefault Brave.VPN.LastUsageTime Brave.Sync.Status.2 Brave.Extensions.AdBlock

Many thanks!

Feb 16 '24 17:02 laurentiugruia

Verification PASSED on

Brave | 1.65.89 Chromium: 123.0.6312.58 (Official Build) beta (64-bit)
-- | --
Revision | 2975bd8d3905ed79de82dd9fa11b6e4019075dd6
OS | Windows 10 Version 22H2 (Build 19045.4170)

As per the comments https://github.com/brave/brave-browser/issues/35302#issuecomment-2023151461 performed regression testing and ensured that the metrics are sent to the endpoints star-randsrv.bsg.brave.com, p3a-json.brave.com, collector.bsg.brave.com as expected
Logged https://github.com/brave/brave-browser/issues/37169

collector.bsg.brave.com

Example	Example	Example	Example

p3a-json.brave.com

Example	Example	Example

star-randsrv.bsg.brave.com

Slow:

Example	Example	Example	Example

Express:

Example	Example	Example	Example

Typical:

Example	Example	Example	Example

Mar 25 '24 12:03 GeetaSarvadnya

@DJAndries: I have verified the issue using charles I am not seeing the refcode apart from the endpoint https://laptop-updates.brave.com, the refcode field is listed only under this endpoint. I don't see refcode for the metrics listed under https://github.com/brave/brave-browser/issues/35302#issuecomment-1948938784 in charles. I have verified https://star-randsrv.bsg.brave.com and I am not sure about the place to verify the refcode in STAR architecture. Could you please add the test plan in the issue? Thanks!

Example	Example

Mar 26 '24 07:03 GeetaSarvadnya

@DJAndries @GeetaSarvadnya assuming we're going to get a minor C123 upgrade today, we'll be releasing a minor 1.64.x tomorrow. If we can get the above resolved by tonight, we'll be able to get the above uplifted into 1.64.x as per https://github.com/brave/brave-core/pull/22767. If not, we'll need to get this into the next 1.64.x release which will most likely be on April 3.

Mar 26 '24 16:03 kjozwiak

Could you please add the test plan in the issue? Thanks!

sorry, just added a note. unfortunately, there's no way to verify the inclusion of the refcode in the STAR data using the client. the best we can do is regression testing.

If not, we'll need to get this into the next 1.64.x release which will most likely be on April 3.

this is okay, if that works out!

Mar 26 '24 20:03 DJAndries

sorry, just added a note. unfortunately, there's no way to verify the inclusion of the refcode in the STAR data using the client. the best we can do is regression testing.

@DJAndries Verify the refcode via https://laptop-updates.brave.com/ endpoint is fine in this case?

Mar 27 '24 03:03 GeetaSarvadnya

@GeetaSarvadnya we just need to verify that the p3a metrics are sent to the existing endpoints as per usual (via star-randsrv.bsg.brave.com, p3a-json.brave.com, collector.bsg.brave.com)

Mar 27 '24 16:03 DJAndries

Verification PASSED on Vivo X70 Pro version 12 running Bravemonoarm64.apk_1.65.96

As per the comments https://github.com/brave/brave-browser/issues/35302#issuecomment-2023151461 performed regression testing and ensured that the metrics were sent to the endpoints star-randsrv.bsg.brave.com, p3a-json.brave.com, collector.bsg.brave.com as expected

collector.bsg.brave.com

Example	Example	Example	Example

p3a-json.brave.com

Example	Example	Example

star-randsrv.bsg.brave.com

Slow:

Example	Example	Example	Example

Express:

Example	Example	Example	Example

Typical:

Example	Example	Example	Example

Apr 02 '24 07:04 GeetaSarvadnya

Removing the QA Pass labels as the above was uplifted into 1.64.x via https://github.com/brave/brave-core/pull/22767 and will need to be re-checked using 1.64.x. Used the verification(s) from https://github.com/brave/brave-browser/issues/35302#issuecomment-2017848090 & https://github.com/brave/brave-browser/issues/35302#issuecomment-2031266434 as the verifications needed to uplift into 1.64.x as per https://github.com/brave/brave-core/pull/22767#pullrequestreview-1975109807.

@GeetaSarvadnya as discussed on Slack, lets just spot check the above on 1.64.x. I don't think we need to run through all the test cases as we've verified the above pretty thoroughly via 1.65.x.

Apr 02 '24 21:04 kjozwiak

The above requires 1.64.116 or higher for 1.64.x verification 👍

Apr 02 '24 21:04 kjozwiak

Verification PASSED on

Brave | 1.64.116 Chromium: 123.0.6312.105 (Official Build) (64-bit)
-- | --
Revision | d9090cab16ea606108bdf6edd7466df58d9d76fd
OS | Windows 10 Version 22H2 (Build 19045.4170)

As per the comments https://github.com/brave/brave-browser/issues/35302#issuecomment-2023151461 performed regression testing and ensured that the metrics were sent to the endpoints star-randsrv.bsg.brave.com, p3a-json.brave.com, collector.bsg.brave.com as expected

Note: All the metrics are NOT sent at endpoint p3a-json.brave.com which is expected as the json p3a metrics reporting type will be deprecated soon

collector.bsg.brave.com

Example	Example	Example

p3a-json.brave.com

Example	Example	Example

star-randsrv.bsg.brave.com

Slow:

Example	Example	Example	Example

Express:

Example	Example	Example	Example

Typical:

Example	Example	Example	Example

Apr 03 '24 09:04 GeetaSarvadnya

Verification PASSED on Vivo X70 Pro version 12 running Bravemonoarm64.apk_1.64.116

As per the comments https://github.com/brave/brave-browser/issues/35302#issuecomment-2023151461 performed regression testing and ensured that the metrics were sent to the endpoints star-randsrv.bsg.brave.com, p3a-json.brave.com, collector.bsg.brave.com as expected

Note: All the metrics are NOT sent at endpoint p3a-json.brave.com which is expected as the json p3a metrics reporting type will be deprecated soon

collector.bsg.brave.com

Example	Example	Example	Example

p3a-json.brave.com

Example	Example	Example

star-randsrv.bsg.brave.com

Slow:

Example	Example	Example	Example

Express:

Example	Example	Example	Example

Typical:

Example	Example	Example	Example

Apr 03 '24 11:04 GeetaSarvadnya

brave-browser brave-browser copied to clipboard

Add refcode to select P3A metrics

brave-browser
brave-browser copied to clipboard