Use new ci-perf-kit with running-ng

[qinsoon]

[github-actions[bot]]

JikesRVM

• binding_trunk: 5f1b40c86f6e5721739d03d22452ac84932e68c4
• trunk: f006c71f14214dd390213b3daa7ce3af7e7d0133
• binding_branch: 5f1b40c86f6e5721739d03d22452ac84932e68c4
• branch: a4c157ff93bcbf0494cdc377561586fa74fc6adb

NoGC (wrench-2021-09-14-Tue-213533)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
antlr	749.00 ±50.82	749.00 ±50.82	749.0	772.00 ±50.82	772.00 ±50.82	772.0	+3.07%	+3.07% :red_square:
avrora	2140.00 ±152.47	2140.00 ±152.47	2140.0	2172.00 ±165.18	2172.00 ±165.18	2172.0	+1.50%	+1.50% :red_square:
fop	1344.50 ±133.42	1344.50 ±133.42	1344.5	1340.00 ±63.53	1340.00 ±63.53	1340.0	-0.33%	-0.33%
hsqldb	822.50 ±57.18	822.50 ±57.18	822.5	820.50 ±108.00	820.50 ±108.00	820.5	-0.24%	-0.24%
luindex	648.00 ±12.71	648.00 ±12.71	648.0	650.50 ±82.59	650.50 ±82.59	650.5	+0.39%	+0.39%

SemiSpace (wrench-2021-09-14-Tue-213632)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
antlr	782.00 ±63.53	782.00 ±63.53	782.0	808.50 ±412.95	808.50 ±412.95	808.5	+3.39%	+3.39% :red_square:
avrora	2206.50 ±362.13	2206.50 ±362.13	2206.5	2215.50 ±6.35	2215.50 ±6.35	2215.5	+0.41%	+0.41%
eclipse	10271.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	10271.0	10504.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	10504.0	+2.27%	+nan%
fop	1478.50 ±362.13	1478.50 ±362.13	1478.5	1617.50 ±247.77	1617.50 ±247.77	1617.5	+9.40%	+9.40% :red_square:
hsqldb	604.00 ±648.02	604.00 ±648.02	604.0	628.50 ±590.84	628.50 ±590.84	628.5	+4.06%	+4.06% :red_square:
jython	12398.00 ±31269.97	12398.00 ±31269.97	12398.0	:x:	:x:	:x:	+nan%	+nan%
luindex	726.00 ±406.60	726.00 ±406.60	726.0	716.00 ±393.89	716.00 ±393.89	716.0	-1.38%	-1.38% :green_square:
pmd	922.50 ±324.01	922.50 ±324.01	922.5	945.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	945.0	+2.44%	+nan%
sunflow	1049.00 ±762.37	1049.00 ±762.37	1049.0	1003.00 ±495.54	1003.00 ±495.54	1003.0	-4.39%	-4.39% :green_square:
xalan	642.00 ±114.36	642.00 ±114.36	642.0	632.00 ±50.82	632.00 ±50.82	632.0	-1.56%	-1.56% :green_square:

MarkSweep (wrench-2021-09-14-Tue-215119)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	2504.00 ±1156.26	2504.00 ±1156.26	2504.0	2512.50 ±1924.99	2512.50 ±1924.99	2512.5	+0.34%	+0.34%
fop	4515.00 ±3316.32	4515.00 ±3316.32	4515.0	4685.50 ±285.89	4685.50 ±285.89	4685.5	+3.78%	+3.78% :red_square:
jython	23344.00 ±6442.05	23344.00 ±6442.05	23344.0	23193.50 ±463.78	23193.50 ±463.78	23193.5	-0.64%	-0.64%
luindex	893.50 ±57.18	893.50 ±57.18	893.5	991.50 ±1264.27	991.50 ±1264.27	991.5	+10.97%	+10.97% :red_square:

[github-actions[bot]]

OpenJDK

• binding_trunk: 383abe61aea473c80bfcf80d3340193fd796db68
• trunk: f006c71f14214dd390213b3daa7ce3af7e7d0133
• binding_branch: 383abe61aea473c80bfcf80d3340193fd796db68
• branch: a4c157ff93bcbf0494cdc377561586fa74fc6adb

NoGC (wrench-2021-09-14-Tue-220233)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	3189.00 ±152.47	3189.00 ±152.47	3189.0	3135.50 ±196.95	3135.50 ±196.95	3135.5	-1.68%	-1.68% :green_square:
batik	1893.00 ±12.71	1893.00 ±12.71	1893.0	1915.00 ±304.95	1915.00 ±304.95	1915.0	+1.16%	+1.16% :red_square:
biojava	21149.00 ±825.90	21149.00 ±825.90	21149.0	21214.50 ±1886.87	21214.50 ±1886.87	21214.5	+0.31%	+0.31%
cassandra	8221.00 ±470.13	8221.00 ±470.13	8221.0	8204.00 ±279.54	8204.00 ±279.54	8204.0	-0.21%	-0.21%
eclipse	16287.00 ±825.90	16287.00 ±825.90	16287.0	16125.00 ±152.47	16125.00 ±152.47	16125.0	-0.99%	-0.99%
fop	1050.00 ±25.41	1050.00 ±25.41	1050.0	1049.50 ±31.77	1049.50 ±31.77	1049.5	-0.05%	-0.05%
graphchi	12779.00 ±635.31	12779.00 ±635.31	12779.0	12756.50 ±324.01	12756.50 ±324.01	12756.5	-0.18%	-0.18%
h2	10521.00 ±330.36	10521.00 ±330.36	10521.0	10432.00 ±292.24	10432.00 ±292.24	10432.0	-0.85%	-0.85%
h2o	5980.00 ±12.71	5980.00 ±12.71	5980.0	5923.00 ±660.72	5923.00 ±660.72	5923.0	-0.95%	-0.95%
jme	7088.00 ±0.00	nan ±nan :warning: 2 removed	7088.0	7087.00 ±12.71	7087.00 ±12.71	7087.0	-0.01%	+nan%
jython	6931.50 ±19.06	6931.50 ±19.06	6931.5	6959.50 ±158.83	6959.50 ±158.83	6959.5	+0.40%	+0.40%
luindex	7281.00 ±12.71	7281.00 ±12.71	7281.0	7301.50 ±82.59	7301.50 ±82.59	7301.5	+0.28%	+0.28%
lusearch	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
pmd	2242.00 ±139.77	2242.00 ±139.77	2242.0	2273.50 ±44.47	2273.50 ±44.47	2273.5	+1.40%	+1.40% :red_square:
sunflow	3421.00 ±38.12	3421.00 ±38.12	3421.0	3442.50 ±273.18	3442.50 ±273.18	3442.5	+0.63%	+0.63%
tomcat	6166.50 ±311.30	6166.50 ±311.30	6166.5	6185.50 ±95.30	6185.50 ±95.30	6185.5	+0.31%	+0.31%
xalan	625.50 ±82.59	625.50 ±82.59	625.5	628.00 ±25.41	628.00 ±25.41	628.0	+0.40%	+0.40%
zxing	1127.00 ±101.65	1127.00 ±101.65	1127.0	1107.00 ±139.77	1107.00 ±139.77	1107.0	-1.77%	-1.77% :green_square:

SemiSpace (wrench-2021-09-14-Tue-231143)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	3302.50 ±184.24	3302.50 ±184.24	3302.5	3376.00 ±381.19	3376.00 ±381.19	3376.0	+2.23%	+2.23% :red_square:
batik	1954.00 ±50.82	1954.00 ±50.82	1954.0	1952.50 ±6.35	1952.50 ±6.35	1952.5	-0.08%	-0.08%
biojava	24132.50 ±387.54	24132.50 ±387.54	24132.5	24083.50 ±819.55	24083.50 ±819.55	24083.5	-0.20%	-0.20%
cassandra	10820.50 ±527.31	10820.50 ±527.31	10820.5	10776.00 ±978.38	10776.00 ±978.38	10776.0	-0.41%	-0.41%
eclipse	15454.50 ±235.06	15454.50 ±235.06	15454.5	15466.50 ±679.78	15466.50 ±679.78	15466.5	+0.08%	+0.08%
fop	1312.50 ±6.35	1312.50 ±6.35	1312.5	1314.00 ±38.12	1314.00 ±38.12	1314.0	+0.11%	+0.11%
graphchi	12671.50 ±425.66	12671.50 ±425.66	12671.5	12729.50 ±641.66	12729.50 ±641.66	12729.5	+0.46%	+0.46%
h2	10892.00 ±1829.69	10892.00 ±1829.69	10892.0	10946.00 ±571.78	10946.00 ±571.78	10946.0	+0.50%	+0.50%
h2o	8309.50 ±819.55	8309.50 ±819.55	8309.5	:x:	:x:	:x:	+nan%	+nan%
jme	7081.50 ±6.35	7081.50 ±6.35	7081.5	:x:	:x:	:x:	+nan%	+nan%
jython	48929.00 ±13214.45	48929.00 ±13214.45	48929.0	49990.00 ±10774.86	49990.00 ±10774.86	49990.0	+2.17%	+2.17% :red_square:
luindex	7275.00 ±203.30	7275.00 ±203.30	7275.0	7331.50 ±412.95	7331.50 ±412.95	7331.5	+0.78%	+0.78%
lusearch	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
pmd	3376.50 ±146.12	3376.50 ±146.12	3376.5	3383.00 ±25.41	3383.00 ±25.41	3383.0	+0.19%	+0.19%
sunflow	8192.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	8192.0	8216.00 ±393.89	8216.00 ±393.89	8216.0	+0.29%	+nan%
tomcat	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
xalan	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
zxing	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%

GenCopy (wrench-2021-09-15-Wed-002205)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	3349.50 ±31.77	3349.50 ±31.77	3349.5	3353.00 ±114.36	3353.00 ±114.36	3353.0	+0.10%	+0.10%
batik	2007.00 ±12.71	2007.00 ±12.71	2007.0	2010.50 ±19.06	2010.50 ±19.06	2010.5	+0.17%	+0.17%
biojava	21367.50 ±425.66	21367.50 ±425.66	21367.5	21427.50 ±1238.85	21427.50 ±1238.85	21427.5	+0.28%	+0.28%
cassandra	10662.00 ±190.59	10662.00 ±190.59	10662.0	10654.50 ±527.31	10654.50 ±527.31	10654.5	-0.07%	-0.07%
eclipse	18066.50 ±324.01	18066.50 ±324.01	18066.5	17885.00 ±2134.64	17885.00 ±2134.64	17885.0	-1.00%	-1.00% :green_square:
fop	1349.50 ±57.18	1349.50 ±57.18	1349.5	1353.00 ±0.00	nan ±nan :warning: 2 removed	1353.0	+0.26%	+nan%
graphchi	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
h2	11006.50 ±362.13	11006.50 ±362.13	11006.5	10832.00 ±2147.35	10832.00 ±2147.35	10832.0	-1.59%	-1.59% :green_square:
h2o	10388.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	10388.0	:x:	:x:	:x:	+nan%	+nan%
jme	7064.50 ±19.06	7064.50 ±19.06	7064.5	7064.50 ±19.06	7064.50 ±19.06	7064.5	+0.00%	+0.00%
jython	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
luindex	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
lusearch	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
pmd	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
sunflow	8398.00 ±393.89	8398.00 ±393.89	8398.0	8454.00 ±114.36	8454.00 ±114.36	8454.0	+0.67%	+0.67%
tomcat	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
xalan	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
zxing	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%

MarkSweep (wrench-2021-09-15-Wed-012930)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	4127.00 ±304.95	4127.00 ±304.95	4127.0	4073.00 ±25.41	4073.00 ±25.41	4073.0	-1.31%	-1.31% :green_square:
batik	4584.00 ±127.06	4584.00 ±127.06	4584.0	4540.00 ±254.12	4540.00 ±254.12	4540.0	-0.96%	-0.96%
biojava	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
cassandra	35546.00 ±3036.78	35546.00 ±3036.78	35546.0	35221.50 ±6257.81	35221.50 ±6257.81	35221.5	-0.91%	-0.91%
eclipse	39254.50 ±1924.99	39254.50 ±1924.99	39254.5	38498.00 ±952.97	38498.00 ±952.97	38498.0	-1.93%	-1.93% :green_square:
fop	4383.50 ±247.77	4383.50 ±247.77	4383.5	4351.00 ±304.95	4351.00 ±304.95	4351.0	-0.74%	-0.74%
graphchi	26928.00 ±914.85	26928.00 ±914.85	26928.0	26605.50 ±247.77	26605.50 ±247.77	26605.5	-1.20%	-1.20% :green_square:
h2	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
h2o	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
jme	7705.00 ±50.82	7705.00 ±50.82	7705.0	7665.50 ±44.47	7665.50 ±44.47	7665.5	-0.51%	-0.51%
jython	52600.00 ±1041.91	52600.00 ±1041.91	52600.0	52419.50 ±3284.55	52419.50 ±3284.55	52419.5	-0.34%	-0.34%
luindex	9424.00 ±1168.97	9424.00 ±1168.97	9424.0	9390.00 ±228.71	9390.00 ±228.71	9390.0	-0.36%	-0.36%
lusearch	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
pmd	26207.50 ±540.01	26207.50 ±540.01	26207.5	26078.50 ±6.35	26078.50 ±6.35	26078.5	-0.49%	-0.49%
sunflow	58685.50 ±13195.39	58685.50 ±13195.39	58685.5	59038.00 ±12375.84	59038.00 ±12375.84	59038.0	+0.60%	+0.60%
tomcat	59280.50 ±3830.92	59280.50 ±3830.92	59280.5	58973.50 ±184.24	58973.50 ±184.24	58973.5	-0.52%	-0.52%
xalan	13353.00 ±1994.87	13353.00 ±1994.87	13353.0	13431.00 ±1346.86	13431.00 ±1346.86	13431.0	+0.58%	+0.58%
zxing	2600.50 ±514.60	2600.50 ±514.60	2600.5	2657.00 ±432.01	2657.00 ±432.01	2657.0	+2.17%	+2.17% :red_square:

Immix (wrench-2021-09-15-Wed-035310)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	3216.50 ±184.24	3216.50 ±184.24	3216.5	3259.50 ±235.06	3259.50 ±235.06	3259.5	+1.34%	+1.34% :red_square:
batik	1924.50 ±31.77	1924.50 ±31.77	1924.5	1919.50 ±44.47	1919.50 ±44.47	1919.5	-0.26%	-0.26%
biojava	20844.50 ±781.43	20844.50 ±781.43	20844.5	20933.00 ±2058.41	20933.00 ±2058.41	20933.0	+0.42%	+0.42%
cassandra	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
eclipse	15851.50 ±95.30	15851.50 ±95.30	15851.5	15797.00 ±152.47	15797.00 ±152.47	15797.0	-0.34%	-0.34%
fop	1151.00 ±0.00	nan ±nan :warning: 2 removed	1151.0	1155.50 ±19.06	1155.50 ±19.06	1155.5	+0.39%	+nan%
graphchi	12780.50 ±235.06	12780.50 ±235.06	12780.5	12754.50 ±717.90	12754.50 ±717.90	12754.5	-0.20%	-0.20%
h2	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
h2o	:x:	:x:	:x:	7180.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	7180.0	+nan%	+nan%
jme	7078.50 ±6.35	7078.50 ±6.35	7078.5	7074.50 ±19.06	7074.50 ±19.06	7074.5	-0.06%	-0.06%
jython	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
luindex	7160.50 ±1734.40	7160.50 ±1734.40	7160.5	7077.00 ±279.54	7077.00 ±279.54	7077.0	-1.17%	-1.17% :green_square:
lusearch	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
pmd	2699.00 ±228.71	2699.00 ±228.71	2699.0	2729.50 ±362.13	2729.50 ±362.13	2729.5	+1.13%	+1.13% :red_square:
sunflow	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
tomcat	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
xalan	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
zxing	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%

[github-actions[bot]]

OpenJDK

• binding_trunk: 145acad3d617ca29d1fa54f64f7e2c83ebd65452
• trunk: 34acc5472942671f89d18d8313af46f9c4c7783e
• binding_branch: 145acad3d617ca29d1fa54f64f7e2c83ebd65452
• branch: ea7da779561d79eb7fef5c316c13d5cdc7705655

NoGC (wrench-2021-09-23-Thu-115352)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	3183.00 ±12.71	3183.00 ±12.71	3183.0	3217.50 ±133.42	3217.50 ±133.42	3217.5	+1.08%	+1.08% :red_square:
batik	1905.00 ±165.18	1905.00 ±165.18	1905.0	1894.50 ±31.77	1894.50 ±31.77	1894.5	-0.55%	-0.55%
biojava	21281.50 ±412.95	21281.50 ±412.95	21281.5	21119.00 ±1423.09	21119.00 ±1423.09	21119.0	-0.76%	-0.76%
cassandra	8214.00 ±177.89	8214.00 ±177.89	8214.0	8204.50 ±69.88	8204.50 ±69.88	8204.5	-0.12%	-0.12%
eclipse	16121.00 ±216.01	16121.00 ±216.01	16121.0	16201.50 ±641.66	16201.50 ±641.66	16201.5	+0.50%	+0.50%
fop	1057.50 ±6.35	1057.50 ±6.35	1057.5	1053.00 ±25.41	1053.00 ±25.41	1053.0	-0.43%	-0.43%
graphchi	12713.50 ±57.18	12713.50 ±57.18	12713.5	12719.00 ±177.89	12719.00 ±177.89	12719.0	+0.04%	+0.04%
h2	10399.50 ±489.19	10399.50 ±489.19	10399.5	10462.00 ±775.08	10462.00 ±775.08	10462.0	+0.60%	+0.60%
h2o	6033.00 ±622.60	6033.00 ±622.60	6033.0	6030.50 ±2052.05	6030.50 ±2052.05	6030.5	-0.04%	-0.04%
jme	7086.00 ±12.71	7086.00 ±12.71	7086.0	7084.50 ±6.35	7084.50 ±6.35	7084.5	-0.02%	-0.02%
jython	6948.00 ±25.41	6948.00 ±25.41	6948.0	6943.50 ±19.06	6943.50 ±19.06	6943.5	-0.06%	-0.06%
luindex	7285.00 ±0.00	nan ±nan :warning: 2 removed	7285.0	7460.00 ±2426.89	7460.00 ±2426.89	7460.0	+2.40%	+nan%
lusearch	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
pmd	2266.00 ±152.47	2266.00 ±152.47	2266.0	2256.00 ±216.01	2256.00 ±216.01	2256.0	-0.44%	-0.44%
sunflow	3432.00 ±241.42	3432.00 ±241.42	3432.0	3408.00 ±50.82	3408.00 ±50.82	3408.0	-0.70%	-0.70%
tomcat	6152.00 ±25.41	6152.00 ±25.41	6152.0	6167.50 ±31.77	6167.50 ±31.77	6167.5	+0.25%	+0.25%
xalan	634.50 ±6.35	634.50 ±6.35	634.5	636.00 ±38.12	636.00 ±38.12	636.0	+0.24%	+0.24%
zxing	1124.00 ±12.71	1124.00 ±12.71	1124.0	1108.50 ±69.88	1108.50 ±69.88	1108.5	-1.38%	-1.38% :green_square:

SemiSpace (wrench-2021-09-23-Thu-130307)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	3272.00 ±25.41	3272.00 ±25.41	3272.0	3215.50 ±120.71	3215.50 ±120.71	3215.5	-1.73%	-1.73% :green_square:
batik	1885.00 ±88.94	1885.00 ±88.94	1885.0	1872.00 ±38.12	1872.00 ±38.12	1872.0	-0.69%	-0.69%
biojava	20840.50 ±870.38	20840.50 ±870.38	20840.5	21000.50 ±1111.79	21000.50 ±1111.79	21000.5	+0.77%	+0.77%
cassandra	8552.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	8552.0	8580.50 ±387.54	8580.50 ±387.54	8580.5	+0.33%	+nan%
eclipse	15147.50 ±69.88	15147.50 ±69.88	15147.5	15178.00 ±864.02	15178.00 ±864.02	15178.0	+0.20%	+0.20%
fop	1078.00 ±25.41	1078.00 ±25.41	1078.0	1078.50 ±44.47	1078.50 ±44.47	1078.5	+0.05%	+0.05%
graphchi	12618.50 ±2179.11	12618.50 ±2179.11	12618.5	12525.50 ±57.18	12525.50 ±57.18	12525.5	-0.74%	-0.74%
h2	8953.00 ±266.83	8953.00 ±266.83	8953.0	8840.00 ±622.60	8840.00 ±622.60	8840.0	-1.26%	-1.26% :green_square:
h2o	5959.50 ±273.18	5959.50 ±273.18	5959.5	:x:	:x:	:x:	+nan%	+nan%
jme	7075.00 ±12.71	7075.00 ±12.71	7075.0	7074.00 ±38.12	7074.00 ±38.12	7074.0	-0.01%	-0.01%
jython	7718.00 ±25.41	7718.00 ±25.41	7718.0	7725.00 ±381.19	7725.00 ±381.19	7725.0	+0.09%	+0.09%
luindex	7078.50 ±171.53	7078.50 ±171.53	7078.5	7056.50 ±108.00	7056.50 ±108.00	7056.5	-0.31%	-0.31%
lusearch	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
pmd	2554.50 ±311.30	2554.50 ±311.30	2554.5	2533.00 ±203.30	2533.00 ±203.30	2533.0	-0.84%	-0.84%
sunflow	5451.50 ±412.95	5451.50 ±412.95	5451.5	5343.50 ±870.38	5343.50 ±870.38	5343.5	-1.98%	-1.98% :green_square:
tomcat	:x:	:x:	:x:	9921.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	9921.0	+nan%	+nan%
xalan	2443.50 ±44.47	2443.50 ±44.47	2443.5	2431.50 ±171.53	2431.50 ±171.53	2431.5	-0.49%	-0.49%
zxing	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%

GenCopy (wrench-2021-09-23-Thu-141149)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	3268.00 ±63.53	3268.00 ±63.53	3268.0	3251.50 ±235.06	3251.50 ±235.06	3251.5	-0.50%	-0.50%
batik	1956.50 ±19.06	1956.50 ±19.06	1956.5	1953.50 ±31.77	1953.50 ±31.77	1953.5	-0.15%	-0.15%
biojava	21383.50 ±196.95	21383.50 ±196.95	21383.5	21283.00 ±368.48	21283.00 ±368.48	21283.0	-0.47%	-0.47%
cassandra	9926.50 ±69.88	9926.50 ±69.88	9926.5	9946.50 ±794.14	9946.50 ±794.14	9946.5	+0.20%	+0.20%
eclipse	17906.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	17906.0	17982.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	17982.0	+0.42%	+nan%
fop	1154.00 ±25.41	1154.00 ±25.41	1154.0	1156.50 ±31.77	1156.50 ±31.77	1156.5	+0.22%	+0.22%
graphchi	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
h2	10514.50 ±82.59	10514.50 ±82.59	10514.5	10485.00 ±304.95	10485.00 ±304.95	10485.0	-0.28%	-0.28%
h2o	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
jme	7069.50 ±31.77	7069.50 ±31.77	7069.5	7068.00 ±0.00	nan ±nan :warning: 2 removed	7068.0	-0.02%	+nan%
jython	7685.00 ±38.12	7685.00 ±38.12	7685.0	7688.50 ±69.88	7688.50 ±69.88	7688.5	+0.05%	+0.05%
luindex	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
lusearch	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
pmd	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
sunflow	6374.00 ±495.54	6374.00 ±495.54	6374.0	6428.00 ±368.48	6428.00 ±368.48	6428.0	+0.85%	+0.85%
tomcat	10180.00 ±406.60	10180.00 ±406.60	10180.0	:x:	:x:	:x:	+nan%	+nan%
xalan	2480.50 ±44.47	2480.50 ±44.47	2480.5	2492.00 ±139.77	2492.00 ±139.77	2492.0	+0.46%	+0.46%
zxing	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%

GenImmix (wrench-2021-09-23-Thu-151702)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	3330.00 ±241.42	3330.00 ±241.42	3330.0	3320.00 ±152.47	3320.00 ±152.47	3320.0	-0.30%	-0.30%
batik	1958.00 ±12.71	1958.00 ±12.71	1958.0	1951.50 ±19.06	1951.50 ±19.06	1951.5	-0.33%	-0.33%
biojava	21430.50 ±311.30	21430.50 ±311.30	21430.5	21370.00 ±76.24	21370.00 ±76.24	21370.0	-0.28%	-0.28%
cassandra	10063.00 ±355.77	10063.00 ±355.77	10063.0	9975.00 ±88.94	9975.00 ±88.94	9975.0	-0.87%	-0.87%
eclipse	17687.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	17687.0	17676.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	17676.0	-0.06%	+nan%
fop	1203.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	1203.0	1176.00 ±114.36	1176.00 ±114.36	1176.0	-2.24%	+nan%
graphchi	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
h2	10789.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	10789.0	10846.50 ±171.53	10846.50 ±171.53	10846.5	+0.53%	+nan%
h2o	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
jme	7071.00 ±38.12	7071.00 ±38.12	7071.0	7072.50 ±6.35	7072.50 ±6.35	7072.5	+0.02%	+0.02%
jython	7647.00 ±381.19	7647.00 ±381.19	7647.0	7620.50 ±209.65	7620.50 ±209.65	7620.5	-0.35%	-0.35%
luindex	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
lusearch	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
pmd	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
sunflow	5834.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	5834.0	5587.00 ±50.82	5587.00 ±50.82	5587.0	-4.23%	+nan%
tomcat	8233.50 ±578.13	8233.50 ±578.13	8233.5	8247.50 ±2217.23	8247.50 ±2217.23	8247.5	+0.17%	+0.17%
xalan	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
zxing	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%

MarkSweep (wrench-2021-09-23-Thu-162228)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	3727.50 ±44.47	3727.50 ±44.47	3727.5	3749.50 ±527.31	3749.50 ±527.31	3749.5	+0.59%	+0.59%
batik	5010.00 ±304.95	5010.00 ±304.95	5010.0	4875.00 ±127.06	4875.00 ±127.06	4875.0	-2.69%	-2.69% :green_square:
biojava	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
cassandra	30944.00 ±635.31	30944.00 ±635.31	30944.0	31555.50 ±2191.82	31555.50 ±2191.82	31555.5	+1.98%	+1.98% :red_square:
eclipse	39349.00 ±1143.56	39349.00 ±1143.56	39349.0	38485.50 ±1238.85	38485.50 ±1238.85	38485.5	-2.19%	-2.19% :green_square:
fop	3709.50 ±1543.80	3709.50 ±1543.80	3709.5	3611.50 ±1226.15	3611.50 ±1226.15	3611.5	-2.64%	-2.64% :green_square:
graphchi	29037.00 ±1029.20	29037.00 ±1029.20	29037.0	28471.50 ±120.71	28471.50 ±120.71	28471.5	-1.95%	-1.95% :green_square:
h2	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
h2o	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
jme	7907.00 ±241.42	7907.00 ±241.42	7907.0	7804.00 ±38.12	7804.00 ±38.12	7804.0	-1.30%	-1.30% :green_square:
jython	40150.50 ±400.25	40150.50 ±400.25	40150.5	40061.00 ±8995.99	40061.00 ±8995.99	40061.0	-0.22%	-0.22%
luindex	9192.00 ±495.54	9192.00 ±495.54	9192.0	9057.50 ±247.77	9057.50 ±247.77	9057.5	-1.46%	-1.46% :green_square:
lusearch	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
pmd	20006.50 ±6.35	20006.50 ±6.35	20006.5	20214.50 ±362.13	20214.50 ±362.13	20214.5	+1.04%	+1.04% :red_square:
sunflow	52825.50 ±2814.42	52825.50 ±2814.42	52825.5	52738.50 ±374.83	52738.50 ±374.83	52738.5	-0.16%	-0.16%
tomcat	24924.50 ±2039.35	24924.50 ±2039.35	24924.5	24979.50 ±2407.83	24979.50 ±2407.83	24979.5	+0.22%	+0.22%
xalan	6091.50 ±69.88	6091.50 ±69.88	6091.5	6131.00 ±127.06	6131.00 ±127.06	6131.0	+0.65%	+0.65%
zxing	1791.50 ±146.12	1791.50 ±146.12	1791.5	1787.00 ±0.00	nan ±nan :warning: 2 removed	1787.0	-0.25%	+nan%

Immix (wrench-2021-09-23-Thu-183227)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
avrora	3219.00 ±152.47	3219.00 ±152.47	3219.0	3227.50 ±184.24	3227.50 ±184.24	3227.5	+0.26%	+0.26%
batik	1904.50 ±6.35	1904.50 ±6.35	1904.5	1908.50 ±6.35	1908.50 ±6.35	1908.5	+0.21%	+0.21%
biojava	20496.00 ±406.60	20496.00 ±406.60	20496.0	20202.50 ±273.18	20202.50 ±273.18	20202.5	-1.43%	-1.43% :green_square:
cassandra	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
eclipse	15451.00 ±749.67	15451.00 ±749.67	15451.0	15539.00 ±216.01	15539.00 ±216.01	15539.0	+0.57%	+0.57%
fop	1052.50 ±6.35	1052.50 ±6.35	1052.5	1054.00 ±38.12	1054.00 ±38.12	1054.0	+0.14%	+0.14%
graphchi	12652.00 ±38.12	12652.00 ±38.12	12652.0	12665.00 ±38.12	12665.00 ±38.12	12665.0	+0.10%	+0.10%
h2	11498.00 ±787.78	11498.00 ±787.78	11498.0	11488.50 ±108.00	11488.50 ±108.00	11488.5	-0.08%	-0.08%
h2o	5139.00 ±nan :warning: 1/2 failed	nan ±nan :warning: 1 removed	5139.0	:x:	:x:	:x:	+nan%	+nan%
jme	7076.00 ±88.94	7076.00 ±88.94	7076.0	7072.50 ±44.47	7072.50 ±44.47	7072.5	-0.05%	-0.05%
jython	7880.50 ±95.30	7880.50 ±95.30	7880.5	7831.00 ±330.36	7831.00 ±330.36	7831.0	-0.63%	-0.63%
luindex	6992.00 ±228.71	6992.00 ±228.71	6992.0	6940.50 ±285.89	6940.50 ±285.89	6940.5	-0.74%	-0.74%
lusearch	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
pmd	2420.00 ±88.94	2420.00 ±88.94	2420.0	2372.00 ±127.06	2372.00 ±127.06	2372.0	-1.98%	-1.98% :green_square:
sunflow	:x:	:x:	:x:	:x:	:x:	:x:	+nan%	+nan%
tomcat	15901.50 ±4466.23	15901.50 ±4466.23	15901.5	14738.00 ±4307.40	14738.00 ±4307.40	14738.0	-7.32%	-7.32% :green_square:
xalan	1945.50 ±146.12	1945.50 ±146.12	1945.5	2035.00 ±317.66	2035.00 ±317.66	2035.0	+4.60%	+4.60% :red_square:
zxing	1155.00 ±254.12	1155.00 ±254.12	1155.0	1131.50 ±171.53	1131.50 ±171.53	1131.5	-2.03%	-2.03% :green_square:

[github-actions[bot]]

JikesRVM

• binding_trunk: 9ff108a4c08d45a800454fb6b070dd6e3059dfa0
• trunk: 34acc5472942671f89d18d8313af46f9c4c7783e
• binding_branch: 9ff108a4c08d45a800454fb6b070dd6e3059dfa0
• branch: ea7da779561d79eb7fef5c316c13d5cdc7705655

NoGC (wrench-2021-09-23-Thu-194714)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
antlr	784.00 ±38.12	784.00 ±38.12	784.0	811.00 ±228.71	811.00 ±228.71	811.0	+3.44%	+3.44% :red_square:
avrora	2147.00 ±63.53	2147.00 ±63.53	2147.0	2178.00 ±292.24	2178.00 ±292.24	2178.0	+1.44%	+1.44% :red_square:
fop	1347.00 ±127.06	1347.00 ±127.06	1347.0	1385.50 ±451.07	1385.50 ±451.07	1385.5	+2.86%	+2.86% :red_square:
hsqldb	839.50 ±57.18	839.50 ±57.18	839.5	793.00 ±25.41	793.00 ±25.41	793.0	-5.54%	-5.54% :green_square:
luindex	683.00 ±139.77	683.00 ±139.77	683.0	675.00 ±63.53	675.00 ±63.53	675.0	-1.17%	-1.17% :green_square:

SemiSpace (wrench-2021-09-23-Thu-194814)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
antlr	701.50 ±31.77	701.50 ±31.77	701.5	737.00 ±76.24	737.00 ±76.24	737.0	+5.06%	+5.06% :red_square:
avrora	2164.00 ±12.71	2164.00 ±12.71	2164.0	2170.50 ±57.18	2170.50 ±57.18	2170.5	+0.30%	+0.30%
eclipse	9808.00 ±190.59	9808.00 ±190.59	9808.0	9853.50 ±400.25	9853.50 ±400.25	9853.5	+0.46%	+0.46%
fop	1273.00 ±50.82	1273.00 ±50.82	1273.0	1278.00 ±114.36	1278.00 ±114.36	1278.0	+0.39%	+0.39%
hsqldb	553.00 ±50.82	553.00 ±50.82	553.0	669.00 ±127.06	669.00 ±127.06	669.0	+20.98%	+20.98% :red_square:
jython	2421.50 ±44.47	2421.50 ±44.47	2421.5	2437.50 ±705.19	2437.50 ±705.19	2437.5	+0.66%	+0.66%
luindex	585.00 ±228.71	585.00 ±228.71	585.0	610.50 ±69.88	610.50 ±69.88	610.5	+4.36%	+4.36% :red_square:
pmd	683.50 ±57.18	683.50 ±57.18	683.5	645.00 ±101.65	645.00 ±101.65	645.0	-5.63%	-5.63% :green_square:
sunflow	783.50 ±19.06	783.50 ±19.06	783.5	831.50 ±69.88	831.50 ±69.88	831.5	+6.13%	+6.13% :red_square:
xalan	411.50 ±19.06	411.50 ±19.06	411.5	392.50 ±196.95	392.50 ±196.95	392.5	-4.62%	-4.62% :green_square:

MarkSweep (wrench-2021-09-23-Thu-195350)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
antlr	1915.00 ±1346.86	1915.00 ±1346.86	1915.0	1845.00 ±292.24	1845.00 ±292.24	1845.0	-3.66%	-3.66% :green_square:
avrora	2375.50 ±133.42	2375.50 ±133.42	2375.5	2383.00 ±50.82	2383.00 ±50.82	2383.0	+0.32%	+0.32%
eclipse	22419.00 ±3163.84	22419.00 ±3163.84	22419.0	22513.50 ±285.89	22513.50 ±285.89	22513.5	+0.42%	+0.42%
fop	3253.00 ±1092.73	3253.00 ±1092.73	3253.0	3466.50 ±4733.06	3466.50 ±4733.06	3466.5	+6.56%	+6.56% :red_square:
hsqldb	6164.00 ±1168.97	6164.00 ±1168.97	6164.0	6253.50 ±3411.62	6253.50 ±3411.62	6253.5	+1.45%	+1.45% :red_square:
jython	13087.50 ±336.71	13087.50 ±336.71	13087.5	12956.00 ±127.06	12956.00 ±127.06	12956.0	-1.00%	-1.00% :green_square:
luindex	763.50 ±184.24	763.50 ±184.24	763.5	768.00 ±228.71	768.00 ±228.71	768.0	+0.59%	+0.59%
pmd	2802.50 ±438.36	2802.50 ±438.36	2802.5	3492.00 ±10673.21	3492.00 ±10673.21	3492.0	+24.60%	+24.60% :red_square:
sunflow	5386.50 ±82.59	5386.50 ±82.59	5386.5	5342.50 ±590.84	5342.50 ±590.84	5342.5	-0.82%	-0.82%
xalan	1572.00 ±50.82	1572.00 ±50.82	1572.0	1539.50 ±146.12	1539.50 ±146.12	1539.5	-2.07%	-2.07% :green_square:

[mmtkgc-bot]

Running benchmarks for JikesRVM... JikesRVM

• binding_trunk: b50ff1d1c871ee78dfb56a7e3342bda7f24c76e2
• trunk: 519e8f3c912017d1c0464e95195b0e7716cdd8a1
• binding_branch: b50ff1d1c871ee78dfb56a7e3342bda7f24c76e2
• branch: 221fb804d45bbdb270ba788dd75406204085c2b7

NoGC (wrench-2021-12-16-Thu-133004)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
antlr	804.90 ±17.11	804.90 ±17.11	806.0	810.20 ±15.70	810.20 ±15.70	815.5	+0.66%	+0.66%
avrora	2161.30 ±11.15	2161.30 ±11.15	2159.0	2169.65 ±13.17	2169.65 ±13.17	2174.0	+0.39%	+0.39%
fop	1353.00 ±6.72	1353.00 ±6.72	1349.5	1344.75 ±6.34	1344.75 ±6.34	1345.0	-0.61%	-0.61%
hsqldb	836.65 ±14.21	836.65 ±14.21	832.0	828.10 ±10.60	828.10 ±10.60	824.5	-1.02%	-1.02% :green_square:
luindex	704.50 ±24.30	704.50 ±24.30	677.0	710.95 ±22.75	710.95 ±22.75	699.5	+0.92%	+0.92%

SemiSpace (wrench-2021-12-16-Thu-134358)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
antlr	709.85 ±6.47	709.85 ±6.47	709.0	712.35 ±7.39	712.35 ±7.39	709.5	+0.35%	+0.35%
avrora	2183.20 ±13.81	2183.20 ±13.81	2182.5	2196.40 ±23.58	2196.40 ±23.58	2190.0	+0.60%	+0.60%
eclipse	10071.95 ±221.56	10071.95 ±221.56	9861.5	9843.75 ±30.80 :warning: 4/20 failed	9843.75 ±30.80	9831.0	-2.27%	-2.27% :green_square:
fop	1278.35 ±10.34	1278.35 ±10.34	1271.5	1284.45 ±8.87	1284.45 ±8.87	1286.5	+0.48%	+0.48%
hsqldb	633.05 ±21.09	633.05 ±21.09	652.0	626.95 ±22.98	626.95 ±22.98	657.5	-0.96%	-0.96%
jython	2449.10 ±44.37	2431.79 ±27.09 :warning: 1 removed	2424.0	2440.05 ±18.25	2440.05 ±18.25	2436.0	-0.37%	+0.34%
luindex	608.80 ±8.04	608.80 ±8.04	611.5	605.45 ±8.04	605.45 ±8.04	607.0	-0.55%	-0.55%
pmd	677.45 ±7.49	677.45 ±7.49	680.0	668.50 ±8.89	668.50 ±8.89	669.0	-1.32%	-1.32% :green_square:
sunflow	776.30 ±19.18	776.30 ±19.18	781.0	778.45 ±17.94	778.45 ±17.94	776.5	+0.28%	+0.28%
xalan	396.00 ±4.12	396.00 ±4.12	395.5	405.75 ±4.19	405.75 ±4.19	404.0	+2.46%	+2.46% :red_square:

MarkSweep (wrench-2021-12-16-Thu-144914)

Benchmark	Trunk(ms)			Branch(ms)			Diff
	mean	mean without outliers	median	mean	mean without outliers	median	mean	mean without outliers
antlr	2054.50 ±77.00	2054.50 ±77.00	1989.0	2092.80 ±84.96	2092.80 ±84.96	2094.0	+1.86%	+1.86% :red_square:
avrora	2359.35 ±19.09	2359.35 ±19.09	2358.5	2367.25 ±13.34	2367.25 ±13.34	2368.5	+0.33%	+0.33%
eclipse	24668.05 ±194.50	24668.05 ±194.50	24531.0	24740.35 ±179.08	24740.35 ±179.08	24583.0	+0.29%	+0.29%
fop	4166.85 ±28.31	4166.85 ±28.31	4166.0	4153.45 ±29.79	4153.45 ±29.79	4157.5	-0.32%	-0.32%
hsqldb	6373.25 ±93.78	6373.25 ±93.78	6311.5	6330.85 ±82.96	6303.16 ±62.80 :warning: 1 removed	6306.0	-0.67%	-1.10% :green_square:
jython	15682.40 ±272.86	15682.40 ±272.86	15839.5	15996.90 ±150.96	15996.90 ±150.96	15937.0	+2.01%	+2.01% :red_square:
luindex	702.65 ±48.58	702.65 ±48.58	767.5	687.20 ±52.88	687.20 ±52.88	742.0	-2.20%	-2.20% :green_square:
pmd	3686.10 ±170.69	3686.10 ±170.69	3561.0	3738.25 ±180.59	3738.25 ±180.59	3528.0	+1.41%	+1.41% :red_square:
sunflow	6509.05 ±35.63	6509.05 ±35.63	6499.0	6513.50 ±35.26	6513.50 ±35.26	6518.5	+0.07%	+0.07%
xalan	1925.50 ±34.34	1914.11 ±26.14 :warning: 1 removed	1911.0	1968.75 ±57.04	1968.75 ±57.04	1939.5	+2.25%	+2.85% :red_square:

[mmtkgc-bot]

Running benchmarks for OpenJDK...

[qinsoon]

Superseded by https://github.com/mmtk/mmtk-core/pull/1005.