CCF
CCF copied to clipboard
Published
20 hours ago •
microsoft
microsoft
Jsruntime memory limit
jsruntime-memory-limit@51595 aka 20221019.16 vs main ewma over 20 builds from 51166 to 51568
Click to see table
main
| build_id | build_number | tpcc_virtual_cft^ | tpcc_virtual_cft_mem | ls_virtual_cft^ | ls_virtual_cft_mem | tpcc_sgx_cft^ | tpcc_sgx_cft_mem | ls_jwt_virtual_cft^ | ls_jwt_virtual_cft_mem | ls_js_virtual_cft^ | ls_js_virtual_cft_mem | ls_sgx_cft^ | ls_sgx_cft_mem | ls_full_js_virtual_cft^ | ls_full_js_virtual_cft_mem | ls_jwt_sgx_cft^ | ls_jwt_sgx_cft_mem | ls_js_jwt_virtual_cft^ | ls_js_jwt_virtual_cft_mem | ls_js_sgx_cft^ | ls_js_sgx_cft_mem | ls_full_js_sgx_cft^ | ls_full_js_sgx_cft_mem | hist_sgx_cft^ | ls_js_jwt_sgx_cft^ | ls_js_jwt_sgx_cft_mem | RB put (/s)^ | CHAMP put (/s)^ | RB get (/s)^ | CHAMP get (/s)^ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 51166 | 20221013.3 | 11012.9 | 0 | 40320.4 | 0 | 6307.98 | 8.32349e+07 | 10266.9 | 0 | 4293.88 | 0 | 19594.2 | 1.71746e+07 | 3468.82 | 0 | 5599.85 | 1.53396e+07 | 3209.83 | 0 | 2420.73 | 9.57242e+06 | 2029.2 | 1.08831e+07 | 21994 | 1871.34 | 9.31027e+06 | 889265 | 1.36769e+06 | 9.20851e+06 | 3.54325e+07 |
| 51192 | 20221013.15 | 11236.4 | 0 | 41714.3 | 0 | 6302.95 | 8.24484e+07 | 10479.4 | 0 | 4385.86 | 0 | 19661.9 | 1.69124e+07 | 3563.95 | 0 | 5600.18 | 1.53396e+07 | 3339.92 | 0 | 2431.32 | 9.57242e+06 | 2033.47 | 9.57242e+06 | 23541.9 | 1878.69 | 9.04813e+06 | 902875 | 1.38135e+06 | 9.27532e+06 | 3.58042e+07 |
| 51218 | 20221014.2 | 10047.5 | 0 | 39098.4 | 0 | 6405.26 | 8.55942e+07 | 10049.4 | 0 | 4310.54 | 0 | 19215.6 | 1.74367e+07 | 3457.64 | 0 | 5690.56 | 1.53396e+07 | 3097.68 | 0 | 2424.67 | 9.83456e+06 | 2036.1 | 9.31027e+06 | 22728.6 | 1879.17 | 9.04813e+06 | 892491 | 1.37838e+06 | 9.07399e+06 | 3.55556e+07 |
| 51225 | 20221014.5 | 10867.1 | 0 | 39581.7 | 0 | 6372.16 | 8.32349e+07 | 10231 | 0 | 4666.19 | 0 | 20080.7 | 1.66503e+07 | 3424.22 | 0 | 5542.74 | 1.56017e+07 | 3231.91 | 0 | 2420.92 | 9.83456e+06 | 2035.03 | 9.31027e+06 | 29800.6 | 1867.36 | 9.31027e+06 | 917146 | 1.37633e+06 | 9.13466e+06 | 3.53707e+07 |
| 51268 | 20221014.24 | 10451.7 | 0 | 39975 | 0 | 6175.05 | 8.29727e+07 | 10156.6 | 0 | 4589.1 | 0 | 19398.9 | 1.69124e+07 | 3598.76 | 0 | 5576.74 | 1.53396e+07 | 3219.98 | 0 | 2422.77 | 9.57242e+06 | 2057.44 | 9.57242e+06 | 26259.1 | 1870.29 | 9.04813e+06 | 900135 | 1.40408e+06 | 9.39445e+06 | 3.65714e+07 |
| 51312 | 20221014.44 | 10700 | 0 | 40972.9 | 0 | 6144.6 | 8.32349e+07 | 10237.2 | 0 | 4689.8 | 0 | 19380 | 1.69124e+07 | 3562.02 | 0 | 5583.26 | 1.53396e+07 | 3414.58 | 0 | 2409.54 | 9.57242e+06 | 2026.31 | 9.31027e+06 | 25408.5 | 1865.88 | 9.04813e+06 | 891980 | 1.37652e+06 | 9.26688e+06 | 3.44781e+07 |
| 51323 | 20221017.3 | 11697.2 | 0 | 43984.1 | 0 | 6253.13 | 8.3497e+07 | 10061.3 | 0 | 4390.77 | 0 | 19369.6 | 1.6126e+07 | 3444.48 | 0 | 5600.96 | 1.56017e+07 | 3280.7 | 0 | 2434.56 | 9.57242e+06 | 2039.86 | 9.31027e+06 | 23812.4 | 1907.97 | 9.04813e+06 | 891786 | 1.36989e+06 | 9.20028e+06 | 3.58042e+07 |
| 51356 | 20221017.18 | 10476.1 | 0 | 42021.6 | 0 | 6294.69 | 8.29727e+07 | 10516.1 | 0 | 4260.13 | 0 | 19384.8 | 1.6126e+07 | 3424.86 | 0 | 5564.24 | 1.56017e+07 | 3349.05 | 0 | 2427.01 | 9.83456e+06 | 2032.63 | 9.57242e+06 | 28408.8 | 1879.5 | 9.31027e+06 | 902005 | 1.36871e+06 | 9.25445e+06 | 3.58042e+07 |
| 51369 | 20221017.23 | 10815.5 | 0 | 43886 | 0 | 6182.92 | 8.3497e+07 | 10496.8 | 0 | 4265.57 | 0 | 20141.6 | 1.63882e+07 | 3591.34 | 0 | 5526.98 | 1.50774e+07 | 3287.98 | 0 | 2428.79 | 9.57242e+06 | 2033.49 | 1.35046e+07 | 31812.7 | 1869.38 | 9.04813e+06 | 892565 | 1.36669e+06 | 9.15508e+06 | 3.48887e+07 |
| 51401 | 20221017.36 | 11574 | 0 | 42017.4 | 0 | 6401.49 | 8.32349e+07 | 10681.1 | 0 | 4325.12 | 0 | 19869.8 | 1.6126e+07 | 3465.53 | 0 | 5662.52 | 1.53396e+07 | 3295.72 | 0 | 2430.97 | 9.57242e+06 | 2063.81 | 9.31027e+06 | 24574.1 | 1835.87 | 9.31027e+06 | 903431 | 1.36769e+06 | 9.25436e+06 | 3.55556e+07 |
| 51413 | 20221018.3 | 11013.5 | 0 | 43526 | 0 | 6278.03 | 8.21863e+07 | 10297.9 | 0 | 4667.91 | 0 | 19757.1 | 1.6126e+07 | 3685.52 | 0 | 5570.73 | 1.56017e+07 | 3264.67 | 0 | 2430.11 | 1.00967e+07 | 2027.11 | 9.31027e+06 | 31329.2 | 1869.16 | 9.04813e+06 | 902039 | 1.34692e+06 | 9.26693e+06 | 3.57411e+07 |
| 51429 | 20221018.9 | 11110.8 | 0 | 40278 | 0 | 6224.69 | 8.29727e+07 | 10337.7 | 0 | 4266.39 | 0 | 19543 | 1.63882e+07 | 3578.13 | 0 | 5628.46 | 1.58639e+07 | 3221.39 | 0 | 2438.33 | 9.57242e+06 | 2036.01 | 9.31027e+06 | 26283.5 | 1849.43 | 9.31027e+06 | 907997 | 1.36879e+06 | 9.21269e+06 | 3.49488e+07 |
| 51448 | 20221018.17 | 10970.3 | 0 | 40387.1 | 0 | 6358.96 | 8.29727e+07 | 10141 | 0 | 4816.31 | 0 | 20736.7 | 1.6126e+07 | 3583.18 | 0 | 6546.68 | 1.58639e+07 | 3365.91 | 0 | 2491.75 | 9.83456e+06 | 2114.99 | 9.57242e+06 | 27143.6 | 2043.52 | 9.04813e+06 | 896039 | 1.36332e+06 | 9.17555e+06 | 3.58042e+07 |
| 51462 | 20221018.22 | 11528.7 | 0 | 40554.5 | 0 | 6364.65 | 8.21863e+07 | 10737.9 | 0 | 4313.46 | 0 | 20872.7 | 1.63882e+07 | 3591.18 | 0 | 6477.21 | 1.53396e+07 | 3248.26 | 0 | 2504.11 | 9.57242e+06 | 2088.99 | 9.31027e+06 | 27237.4 | 2108.59 | 9.04813e+06 | 905588 | 1.38284e+06 | 9.26269e+06 | 3.58669e+07 |
| 51484 | 20221018.31 | 11594.7 | 0 | 40594.8 | 0 | 6322.76 | 8.29727e+07 | 10515.9 | 0 | 4413.7 | 0 | 20910.3 | 1.6126e+07 | 3471.44 | 0 | 6470.15 | 1.58639e+07 | 3375.31 | 0 | 2541.74 | 1.00967e+07 | 2082.94 | 9.57242e+06 | 28596.7 | 1993.75 | 9.31027e+06 | 886076 | 1.34268e+06 | 9.00216e+06 | 3.5128e+07 |
| 51516 | 20221018.43 | 11003.7 | 0 | 41607.1 | 0 | 6379.93 | 8.37592e+07 | 10624.7 | 0 | 4282.55 | 0 | 20709.1 | 1.71746e+07 | 3524.01 | 0 | 6410.82 | 1.53396e+07 | 3224.96 | 0 | 2492.92 | 9.83456e+06 | 2081.91 | 9.57242e+06 | 23047.8 | 1992.92 | 9.04813e+06 | 885966 | 1.395e+06 | 9.17974e+06 | 3.58669e+07 |
| 51542 | 20221018.51 | 11060 | 0 | 40657.4 | 0 | 6396.41 | 8.3497e+07 | 10727.4 | 0 | 4326.02 | 0 | 20985.3 | 1.6126e+07 | 3603.46 | 0 | 6446.03 | 1.58639e+07 | 3255.85 | 0 | 2499.1 | 9.83456e+06 | 2154.7 | 9.57242e+06 | 25612.3 | 1998.32 | 9.04813e+06 | 881385 | 1.36551e+06 | 9.27532e+06 | 3.58663e+07 |
| 51552 | 20221018.55 | 8720.44 | 0 | 34911.7 | 0 | 6425.99 | 8.3497e+07 | 9261.82 | 0 | 4085.74 | 0 | 20823.3 | 1.63882e+07 | 3347.54 | 0 | 6505.01 | 1.58639e+07 | 3197.54 | 0 | 2485.32 | 1.00967e+07 | 2095.18 | 9.31027e+06 | 24649.6 | 2010.82 | 9.31027e+06 | 917639 | 1.37977e+06 | 9.27952e+06 | 3.63121e+07 |
| 51560 | 20221019.3 | 11252.2 | 0 | 37041 | 0 | 6359.85 | 8.3497e+07 | 10152.8 | 0 | 4368.85 | 0 | 20749 | 1.63882e+07 | 3403.13 | 0 | 6418.94 | 1.58639e+07 | 3365.06 | 0 | 2490.69 | 9.83456e+06 | 2105.81 | 9.31027e+06 | 28778.6 | 2036.1 | 9.04813e+06 | 908117 | 1.37624e+06 | 9.41168e+06 | 3.66362e+07 |
| 51568 | 20221019.7 | 10865.4 | 0 | 37837.3 | 0 | 5745.73 | 8.37592e+07 | 10050.4 | 0 | 4574.09 | 0 | 17946.8 | 1.71746e+07 | 3426.37 | 0 | 6045.94 | 1.53396e+07 | 3507.21 | 0 | 2356.59 | 9.83456e+06 | 1999.41 | 1.08831e+07 | 21129.7 | 1975.93 | 9.31027e+06 | 905101 | 1.37374e+06 | 9.29629e+06 | 3.54933e+07 |
jsruntime-memory-limit
| build_id | build_number | tpcc_virtual_cft^ | tpcc_virtual_cft_mem | ls_virtual_cft^ | ls_virtual_cft_mem | ls_jwt_virtual_cft^ | ls_jwt_virtual_cft_mem | ls_js_virtual_cft^ | ls_js_virtual_cft_mem | ls_full_js_virtual_cft^ | ls_full_js_virtual_cft_mem | ls_js_jwt_virtual_cft^ | ls_js_jwt_virtual_cft_mem | tpcc_sgx_cft^ | tpcc_sgx_cft_mem | ls_sgx_cft^ | ls_sgx_cft_mem | hist_sgx_cft^ | ls_jwt_sgx_cft^ | ls_jwt_sgx_cft_mem | ls_js_sgx_cft^ | ls_js_sgx_cft_mem | ls_full_js_sgx_cft^ | ls_full_js_sgx_cft_mem | ls_js_jwt_sgx_cft^ | ls_js_jwt_sgx_cft_mem | RB put (/s)^ | CHAMP put (/s)^ | RB get (/s)^ | CHAMP get (/s)^ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 51504 | 20221018.37 | 11281.8 | 0 | 41348.6 | 0 | 10524.5 | 0 | 4422.56 | 0 | 3558.66 | 0 | 3312.93 | 0 | 6419.32 | 8.45456e+07 | 20857.8 | 1.66503e+07 | 22886.2 | 6429.65 | 1.56017e+07 | 2441.86 | 9.83456e+06 | 2074.63 | 9.31027e+06 | 1988.36 | 9.04813e+06 | 900530 | 1.38275e+06 | 9.21688e+06 | 3.58669e+07 |
| 51506 | 20221018.38 | 11116.4 | 0 | 41070.8 | 0 | 10045 | 0 | 4315.12 | 0 | 3543.71 | 0 | 3209.49 | 0 | 6421.74 | 8.29727e+07 | 20735.1 | 1.69124e+07 | 28697.9 | 6506.47 | 1.58639e+07 | 2479.87 | 9.83456e+06 | 2079.24 | 9.31027e+06 | 1994.9 | 9.04813e+06 | 899778 | 1.37744e+06 | 9.27948e+06 | 3.58669e+07 |
| 51566 | 20221019.5 | 10937.4 | 0 | 43148.9 | 0 | 10595.7 | 0 | 4352.13 | 0 | 3509.57 | 0 | 3238.32 | 0 | 6495.13 | 8.42835e+07 | 20841.4 | 1.71746e+07 | 23349 | 6541.17 | 1.53396e+07 | 2467.86 | 9.57242e+06 | 2075.49 | 9.57242e+06 | 1980.02 | 9.04813e+06 | 894239 | 1.38611e+06 | 9.26688e+06 | 3.63759e+07 |
| 51567 | 20221019.6 | 11216.5 | 0 | 42745.8 | 0 | 10157.9 | 0 | 4359.9 | 0 | 3408.81 | 0 | 3464.92 | 0 | 6458 | 8.21863e+07 | 21382.3 | 1.66503e+07 | 25132.3 | 6487.74 | 1.56017e+07 | 2492.69 | 9.83456e+06 | 2092.73 | 9.57242e+06 | 1991.46 | 9.04813e+06 | 875913 | 1.3793e+06 | 9.25018e+06 | 3.58042e+07 |
| 51595 | 20221019.16 | 10642.7 | 0 | 41608.1 | 0 | 10162.6 | 0 | 4362.77 | 0 | 3422.41 | 0 | 3182.04 | 0 | 5662.88 | 8.37592e+07 | 17665.3 | 1.66503e+07 | 27524.9 | 6007.38 | 1.56017e+07 | 2415.12 | 9.83456e+06 | 1991.17 | 9.57242e+06 | 1935.05 | 9.04813e+06 | 909931 | 1.35664e+06 | 9.23763e+06 | 3.56174e+07 |
I tested this in a hacky way using logging in cpp code and a small test in python infra that submits this new proposal. It does reflect the new values between runtimes after submitting the proposal. I could perhaps write a function to expose these values to python infra, but I'm not sure if it's worth it. If anyone feels strongly about this, please let me know.