firefox-ios icon indicating copy to clipboard operation
firefox-ios copied to clipboard

Add FXIOS-4452 [v104] Submit commits to Glean probe-scraper

Open relud opened this issue 2 years ago • 9 comments

to validate and publish metrics, per https://mozilla.github.io/glean/book/user/adding-glean-to-your-project/enable-data-ingestion.html#validate-and-publish-metrics

After your product has been enabled, you must submit commits to probe scraper to validate and publish metrics. Metrics will only be published from branches defined in probe scraper's repositories.yaml, or the Git default branch if not explicitly configured. This should happen on every CI run to the specified branches. Nightly jobs will then automatically add published metrics to the Glean Dictionary and other data platform tools.

relud avatar Jun 21 '22 19:06 relud

@travis79 Is there context on this PR? Looks straightforward, but would like some clarity

jevans-mozilla avatar Jun 23 '22 17:06 jevans-mozilla

@travis79 Is there context on this PR? Looks straightforward, but would like some clarity

@relud is working on some improvements to the Probe Scraper data-tool that does a lot of the heavy lifting involved with data collection , ingestion, and storage, etc. This is a change that allows the app to take advantage of the new way we are wanting to do things. @relud could probably elaborate further if I have misrepresented anything.

tl;dr this change will allow firefox-ios to make use of the new version of a backend tool that is necessary for Glean.

travis79 avatar Jun 23 '22 17:06 travis79

nothing misrepresented, but from the same PR on another product:

The main benefit here for Glean products is that this CI job will allow people to see if changes to metrics files are invalid before they are merged.

Another benefit this will soon[^1] provide is that if a glean product does merge invalid changes, that doesn't block schema deploys for all glean products.

[^1]: probe scraper currently runs nightly scraping all of git history for every glean product. When that job fails, all schema deploys are blocked, usually until the offending commits are added to an ignore list in probe scraper. Pending a backend migration planned for 2022H2, this CI job will replace nightly probe scraper runs for this product. Until that migration happens, the nightly job will continue cause new metrics to show up in your analysis environment.

relud avatar Jun 23 '22 18:06 relud

Changes look good to me. It is adding a github action that will not interfere with our regular CI workflow.

isabelrios avatar Jun 24 '22 08:06 isabelrios

@isabelrios Can we get this merged?

badboy avatar Jul 29 '22 09:07 badboy

@badboy before landing could you please update the PR with latest main? The CI is failing because there has been changes that are needed to compile. Thanks and sorry it has taken so long to look into this PR

isabelrios avatar Jul 29 '22 10:07 isabelrios

Ah, no worried. Let me take a look.

badboy avatar Jul 29 '22 11:07 badboy

Ah, no worried. Let me take a look.

I can't, as this is in @relud's fork. @relud can you rebase it on top of main? Then we land this.

badboy avatar Jul 29 '22 11:07 badboy

@isabelrios this was rebased, and checks are now passing.

relud avatar Aug 03 '22 19:08 relud

Hello! Since this is green and @isabelrios approved in comments, I'll approve and merge it.

lmarceau avatar Aug 10 '22 18:08 lmarceau