rustc-perf icon indicating copy to clipboard operation
rustc-perf copied to clipboard

weird perf run duration, ~30 mins shorter

Open klensy opened this issue 4 months ago • 4 comments

If you look at status page timeline:

PR	Kind	SHA	Status	Run end	Duration	Errors
#145582	Master	64a99db105f45	Finished	8/31/2025, 10:50:44 AM	1h 39m 31s	
#146043	Master	cd60c60d9f6bf	Finished	8/31/2025, 7:09:24 AM	1h 07m 24s	
#146029	Master	523d3999dcd4b	Finished	8/31/2025, 1:54:13 AM	1h 36m 24s	
#145479	Master	0f506968010fa	Finished	8/30/2025, 10:03:50 PM	1h 34m 18s	
#146026	Master	e95db591a4550	Finished	8/30/2025, 7:00:12 PM	1h 40m 16s	

you can see that 146043 completed in almost 30 mins faster. This looks wrong, and i don't see any error messages (maybe something was skipped?). Probably this was other runner? In that case it will be nice to have runner column (even for old status page)

klensy avatar Aug 31 '25 11:08 klensy

There is only a single runner at the moment. The normal duration is 1h 7 minutes, the rest of the perf. runs were actually unexpectedly longer for some reason.

Kobzol avatar Aug 31 '25 12:08 Kobzol

Oh, it's inverse situation, fun.

klensy avatar Aug 31 '25 12:08 klensy

...and time almost back


PR	Kind	SHA	Status	Run end	Duration	Errors
#143290	Master	c0bb3b98bb7aa	Finished	9/1/2025, 6:11:41 PM	1h 06m 51s	
#146082	Try	    0b2a374f3806c	Finished	9/1/2025, 4:22:19 PM	1h 07m 01s	
#146077	Master	84a17470220e7	Finished	9/1/2025, 3:13:14 PM	1h 16m 23s	
#146072	Master	be4e9b77ab550	Finished	9/1/2025, 12:17:26 PM	1h 30m 54s	
#145663	Master	828e45ad11ce4	Finished	9/1/2025, 9:05:32 AM	1h 25m 54s	

klensy avatar Sep 01 '25 15:09 klensy

It seems to be a combination of DB timeouts and slow S3 uploads. Maybe some network issue at Hetzner.

Kobzol avatar Sep 01 '25 15:09 Kobzol