Latest garie/lighthouse docker images drops performance score

[gschoenberger]

I am using the latest garie/lighthouse image to get the fix with dumb init to get rid of the chrome zombie processes: garie/lighthouse latest c135a34430bc 25 hours ago 606MB

Problem description: Fetched performance scores differ dramatically from the previous version.

Question: Did the default setting for emulated device change?

Previous: Score: 28 Emulated Nexus 5X, Simulated Fast 3G network

Current: Score: 3 Emulated Nexus 5X, Simulated Slow 4G network

Grafana is showing the drops after changing to the new image:

[gschoenberger]

I stumbled across https://github.com/GoogleChrome/lighthouse/blob/master/docs/throttling.md Maybe settings from lighthouse changed...

[boyney123]

Hey @gschoenberger ,

Yeah, this looks interesting, I updated to 4.0.0 to I bet something changed, I know we added some configuration to lighthouse now, you might be able to use that to configure it back to what it was?

Any idea's on how you would fix it?

[gschoenberger]

Currently I think it is not a problem with Garie, but lighthouse is just reporting lower scores. ATM it seems the values are correct but just differ in terms of lighthouse version.

My previous lighthouse container used lighthouse 3.0.3 with Chrome 69. The updated Garie now uses lighthouse 4.1.0 with Chrome 71, but I didn't expect the scores to drop from > 25 to < 5.

I compared two runtime settings and one with the chrome lighthouse extension:

Current Garie lighthouse: Score 4 Generated by Lighthouse 4.1.0 Chrome 71

Fetch time: Feb 5, 2019, 1:50 PM GMT+1
Device: Emulated Nexus 5X
Network throttling: 150 ms TCP RTT, 1,638.4 Kbps throughput (Simulated)
CPU throttling: 4x slowdown (Simulated)
User agent (host): Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/72.0.3626.81 Safari/537.36
User agent (network): Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3559.0 Mobile Safari/537.36
CPU/Memory Power: 752 

Current Chrome lighthouse extension: Score 7 Generated by Lighthouse 4.0.0 Chrome 71

Fetch time: Feb 5, 2019, 2:08 PM GMT+1
Device: Emulated Nexus 5X
Network throttling: 150 ms TCP RTT, 1,638.4 Kbps throughput (Simulated)
CPU throttling: 4x slowdown (Simulated)
User agent (host): Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.81 Safari/537.36
User agent (network): Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3559.0 Mobile Safari/537.36
CPU/Memory Power: 520

Previuos Garie lighthouse: Score 28 Generated by Lighthouse 3.0.3 Chrome 69

Fetch time: Feb 5, 2019, 12:30 PM GMT+1
Device: Emulated Nexus 5X
Network throttling: 150 ms TCP RTT, 1,638.4 Kbps throughput (Simulated)
CPU throttling: 4x slowdown (Simulated)
User agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/69.0.3497.100 Safari/537.36