[Infrastructure/Benchmarking] Track overtime average time and memory usage to build SDK docs
Not sure how this would be set up or if it would even be particularly useful due to typical differences between runs, machines, loads, etc.
It seems there's been even more issues related to dartdoc's performance recently, so it would be interesting if we could track this with some relatively empirical data. Perhaps that would be by building the SDK, Flutter, or any larger package and tracking those results over time. Perhaps it could even been connected to this functionality: https://github.com/dart-lang/dartdoc/issues/2513
We are able to extract similar numbers from our build data ("build api docs" also does an upload which muddies the waters a little bit):
SELECT
start_time,
input.gitiles_commit.id,
(
SELECT
TIMESTAMP_DIFF(s.end_time, s.start_time, SECOND)
FROM
UNNEST(steps) AS s
WHERE
s.name = "build api docs") AS duration
FROM
`cr-buildbucket.dart.builds`
WHERE
builder.builder = "dart-sdk-linux"
AND start_time > TIMESTAMP_TRUNC(CURRENT_TIMESTAMP, WEEK)
ORDER BY start_time
LIMIT 10
Gives (the hashes are Dart SDK commit hashes):
1 | 2021-03-07 17:20:34.001283 UTC | ce3f671f7ad7480811a6b19bc4b2cf46e2ab4708 | 291 |
2 | 2021-03-07 21:50:03.567183 UTC | 0506b1b2d30947688b4a338accde0f015548c68a | 290 |
3 | 2021-03-07 23:24:52.558727 UTC | 75308fc26b7c08c26029cd143720eb05b5370abc | 291 |
4 | 2021-03-08 06:59:00.905004 UTC | 14b07bc81cc01d83e6a4f4def49a4d29eea66957 | 301 |
5 | 2021-03-08 08:27:41.624047 UTC | 645035bbb9c8fa595f3ba9f25284f7f85141255e | 294 |
6 | 2021-03-08 09:49:23.242292 UTC | 5794e716ab7a93fcd44fedece128ab3a65f482a0 | 301 |
7 | 2021-03-08 11:11:27.713726 UTC | 2d6073bbf8949644bae570471b62057584769ea5 | 314 |
8 | 2021-03-08 12:33:59.426629 UTC | 01b0947a6d54c878a809236a8890c162b840c8f0 | 300 |
9 | 2021-03-08 14:21:42.450650 UTC | ae37ff098a2bc3dd16f3e7d528cafff678013517 | 300 |
10 | 2021-03-08 15:47:10.035046 UTC | a84ee9662c0ac4c1f5fd7162cc763f1572140e81 | 298