node
node copied to clipboard
Published
20 hours ago •
nodejs
nodejs
assert: refactor to avoid unsafe array iteration
Checklist
- [x]
make -j4 test(UNIX), orvcbuild test(Windows) passes - [x] commit message follows commit guidelines
Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/868/
These changes appear to have some fairly large performance regressions:
assert/ok.js n=100000 *** -77.20 % ±2.93% ±3.94% ±5.21%
assert/throws.js method='doesNotThrow' n=10000 *** -41.40 % ±5.50% ±7.37% ±9.71%
Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/871/
These changes appear to have some fairly large performance regressions:
I've remove the changes responsible for the regressions, it looks better now.
Benchmark results
confidence improvement accuracy (*) (**) (***)
assert/deepequal-buffer.js method='deepEqual' strict=0 len=1000 n=20000 1.00 % ±3.84% ±5.11% ±6.66%
assert/deepequal-buffer.js method='deepEqual' strict=0 len=100 n=20000 -2.52 % ±3.69% ±4.91% ±6.39%
assert/deepequal-buffer.js method='deepEqual' strict=1 len=1000 n=20000 -0.05 % ±4.32% ±5.75% ±7.49%
assert/deepequal-buffer.js method='deepEqual' strict=1 len=100 n=20000 -0.32 % ±3.85% ±5.12% ±6.67%
assert/deepequal-buffer.js method='notDeepEqual' strict=0 len=1000 n=20000 -3.52 % ±4.77% ±6.36% ±8.29%
assert/deepequal-buffer.js method='notDeepEqual' strict=0 len=100 n=20000 -4.13 % ±4.35% ±5.79% ±7.53%
assert/deepequal-buffer.js method='notDeepEqual' strict=1 len=1000 n=20000 -0.30 % ±3.47% ±4.62% ±6.01%
assert/deepequal-buffer.js method='notDeepEqual' strict=1 len=100 n=20000 2.27 % ±4.58% ±6.10% ±7.94%
assert/deepequal-map.js method='deepEqual_mixed' strict=0 len=500 n=500 -0.56 % ±1.79% ±2.39% ±3.11%
assert/deepequal-map.js method='deepEqual_mixed' strict=1 len=500 n=500 0.19 % ±1.63% ±2.17% ±2.83%
assert/deepequal-map.js method='deepEqual_objectOnly' strict=0 len=500 n=500 * -1.51 % ±1.49% ±1.98% ±2.58%
assert/deepequal-map.js method='deepEqual_objectOnly' strict=1 len=500 n=500 -0.23 % ±1.79% ±2.39% ±3.11%
assert/deepequal-map.js method='deepEqual_primitiveOnly' strict=0 len=500 n=500 -2.59 % ±4.13% ±5.49% ±7.15%
assert/deepequal-map.js method='deepEqual_primitiveOnly' strict=1 len=500 n=500 * -4.75 % ±4.60% ±6.13% ±8.01%
assert/deepequal-map.js method='notDeepEqual_mixed' strict=0 len=500 n=500 -0.69 % ±4.12% ±5.49% ±7.15%
assert/deepequal-map.js method='notDeepEqual_mixed' strict=1 len=500 n=500 -4.91 % ±5.35% ±7.14% ±9.33%
assert/deepequal-map.js method='notDeepEqual_objectOnly' strict=0 len=500 n=500 -0.01 % ±2.13% ±2.84% ±3.70%
assert/deepequal-map.js method='notDeepEqual_objectOnly' strict=1 len=500 n=500 0.62 % ±2.09% ±2.78% ±3.62%
assert/deepequal-map.js method='notDeepEqual_primitiveOnly' strict=0 len=500 n=500 0.25 % ±4.35% ±5.81% ±7.61%
assert/deepequal-map.js method='notDeepEqual_primitiveOnly' strict=1 len=500 n=500 -1.49 % ±3.11% ±4.15% ±5.42%
assert/deepequal-object.js method='deepEqual' strict=0 size=1000 n=5000 3.36 % ±5.64% ±7.50% ±9.76%
assert/deepequal-object.js method='deepEqual' strict=0 size=100 n=5000 -0.92 % ±6.39% ±8.50% ±11.06%
assert/deepequal-object.js method='deepEqual' strict=0 size=50000 n=5000 0.09 % ±1.03% ±1.37% ±1.80%
assert/deepequal-object.js method='deepEqual' strict=1 size=1000 n=5000 -3.24 % ±4.11% ±5.47% ±7.12%
assert/deepequal-object.js method='deepEqual' strict=1 size=100 n=5000 1.74 % ±7.32% ±9.77% ±12.77%
assert/deepequal-object.js method='deepEqual' strict=1 size=50000 n=5000 -0.72 % ±1.10% ±1.47% ±1.93%
assert/deepequal-object.js method='notDeepEqual' strict=0 size=1000 n=5000 6.76 % ±9.67% ±12.88% ±16.77%
assert/deepequal-object.js method='notDeepEqual' strict=0 size=100 n=5000 -0.28 % ±7.63% ±10.15% ±13.22%
assert/deepequal-object.js method='notDeepEqual' strict=0 size=50000 n=5000 0.16 % ±1.59% ±2.12% ±2.76%
assert/deepequal-object.js method='notDeepEqual' strict=1 size=1000 n=5000 -2.35 % ±6.49% ±8.64% ±11.24%
assert/deepequal-object.js method='notDeepEqual' strict=1 size=100 n=5000 0.06 % ±8.16% ±10.86% ±14.14%
assert/deepequal-object.js method='notDeepEqual' strict=1 size=50000 n=5000 * 1.90 % ±1.81% ±2.41% ±3.13%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Array' strict=0 len=20000 n=25 primitive='array' * 4.97 % ±4.67% ±6.22% ±8.10%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Array' strict=0 len=20000 n=25 primitive='number' -0.08 % ±4.27% ±5.69% ±7.41%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Array' strict=0 len=20000 n=25 primitive='object' -0.11 % ±3.62% ±4.82% ±6.27%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Array' strict=0 len=20000 n=25 primitive='string' -1.97 % ±3.83% ±5.10% ±6.64%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Array' strict=1 len=20000 n=25 primitive='array' 2.72 % ±3.07% ±4.09% ±5.33%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Array' strict=1 len=20000 n=25 primitive='number' -0.21 % ±5.21% ±6.94% ±9.04%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Array' strict=1 len=20000 n=25 primitive='object' -2.90 % ±4.68% ±6.24% ±8.15%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Array' strict=1 len=20000 n=25 primitive='string' 0.61 % ±4.69% ±6.25% ±8.14%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Set' strict=0 len=20000 n=25 primitive='array' -0.65 % ±5.18% ±6.90% ±8.99%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Set' strict=0 len=20000 n=25 primitive='number' 3.52 % ±5.32% ±7.10% ±9.28%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Set' strict=0 len=20000 n=25 primitive='object' -3.31 % ±5.71% ±7.60% ±9.89%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Set' strict=0 len=20000 n=25 primitive='string' 1.64 % ±7.08% ±9.42% ±12.26%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Set' strict=1 len=20000 n=25 primitive='array' 2.74 % ±6.99% ±9.31% ±12.13%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Set' strict=1 len=20000 n=25 primitive='number' -0.40 % ±6.24% ±8.32% ±10.87%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Set' strict=1 len=20000 n=25 primitive='object' 0.05 % ±4.92% ±6.55% ±8.54%
assert/deepequal-prims-and-objs-big-array-set.js method='deepEqual_Set' strict=1 len=20000 n=25 primitive='string' 0.60 % ±6.34% ±8.44% ±10.99%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Array' strict=0 len=20000 n=25 primitive='array' 3.34 % ±3.66% ±4.87% ±6.35%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Array' strict=0 len=20000 n=25 primitive='number' 3.41 % ±5.30% ±7.06% ±9.19%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Array' strict=0 len=20000 n=25 primitive='object' 1.72 % ±4.87% ±6.49% ±8.45%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Array' strict=0 len=20000 n=25 primitive='string' 0.05 % ±4.73% ±6.29% ±8.19%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Array' strict=1 len=20000 n=25 primitive='array' 3.63 % ±5.29% ±7.05% ±9.19%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Array' strict=1 len=20000 n=25 primitive='number' -0.69 % ±4.01% ±5.33% ±6.94%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Array' strict=1 len=20000 n=25 primitive='object' -1.54 % ±5.28% ±7.04% ±9.20%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Array' strict=1 len=20000 n=25 primitive='string' 3.66 % ±5.06% ±6.75% ±8.83%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Set' strict=0 len=20000 n=25 primitive='array' 2.49 % ±7.38% ±9.82% ±12.78%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Set' strict=0 len=20000 n=25 primitive='number' 3.21 % ±5.12% ±6.82% ±8.89%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Set' strict=0 len=20000 n=25 primitive='object' -1.28 % ±5.52% ±7.34% ±9.55%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Set' strict=0 len=20000 n=25 primitive='string' ** 8.46 % ±5.07% ±6.75% ±8.79%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Set' strict=1 len=20000 n=25 primitive='array' 2.00 % ±6.77% ±9.02% ±11.74%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Set' strict=1 len=20000 n=25 primitive='number' -2.12 % ±4.89% ±6.50% ±8.47%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Set' strict=1 len=20000 n=25 primitive='object' -1.17 % ±7.13% ±9.49% ±12.35%
assert/deepequal-prims-and-objs-big-array-set.js method='notDeepEqual_Set' strict=1 len=20000 n=25 primitive='string' 0.65 % ±7.27% ±9.67% ±12.59%
assert/deepequal-prims-and-objs-big-loop.js method='deepEqual' strict=0 n=20000 primitive='array' 1.53 % ±5.47% ±7.28% ±9.47%
assert/deepequal-prims-and-objs-big-loop.js method='deepEqual' strict=0 n=20000 primitive='number' * -5.71 % ±4.59% ±6.12% ±7.99%
assert/deepequal-prims-and-objs-big-loop.js method='deepEqual' strict=0 n=20000 primitive='object' 1.69 % ±4.23% ±5.64% ±7.34%
assert/deepequal-prims-and-objs-big-loop.js method='deepEqual' strict=0 n=20000 primitive='string' -1.09 % ±4.11% ±5.47% ±7.12%
assert/deepequal-prims-and-objs-big-loop.js method='deepEqual' strict=1 n=20000 primitive='array' 1.94 % ±5.24% ±6.97% ±9.07%
assert/deepequal-prims-and-objs-big-loop.js method='deepEqual' strict=1 n=20000 primitive='number' 1.40 % ±4.71% ±6.26% ±8.15%
assert/deepequal-prims-and-objs-big-loop.js method='deepEqual' strict=1 n=20000 primitive='object' -2.20 % ±4.38% ±5.83% ±7.59%
assert/deepequal-prims-and-objs-big-loop.js method='deepEqual' strict=1 n=20000 primitive='string' -1.32 % ±4.73% ±6.30% ±8.21%
assert/deepequal-prims-and-objs-big-loop.js method='notDeepEqual' strict=0 n=20000 primitive='array' -1.37 % ±5.89% ±7.83% ±10.20%
assert/deepequal-prims-and-objs-big-loop.js method='notDeepEqual' strict=0 n=20000 primitive='number' -0.66 % ±4.78% ±6.36% ±8.27%
assert/deepequal-prims-and-objs-big-loop.js method='notDeepEqual' strict=0 n=20000 primitive='object' 2.00 % ±6.05% ±8.07% ±10.52%
assert/deepequal-prims-and-objs-big-loop.js method='notDeepEqual' strict=0 n=20000 primitive='string' 0.67 % ±4.69% ±6.25% ±8.17%
assert/deepequal-prims-and-objs-big-loop.js method='notDeepEqual' strict=1 n=20000 primitive='array' 4.62 % ±6.56% ±8.73% ±11.37%
assert/deepequal-prims-and-objs-big-loop.js method='notDeepEqual' strict=1 n=20000 primitive='number' 1.08 % ±4.82% ±6.41% ±8.34%
assert/deepequal-prims-and-objs-big-loop.js method='notDeepEqual' strict=1 n=20000 primitive='object' -3.34 % ±5.29% ±7.04% ±9.16%
assert/deepequal-prims-and-objs-big-loop.js method='notDeepEqual' strict=1 n=20000 primitive='string' 1.71 % ±4.65% ±6.19% ±8.07%
assert/deepequal-set.js method='deepEqual_mixed' strict=0 len=500 n=500 0.55 % ±2.29% ±3.04% ±3.96%
assert/deepequal-set.js method='deepEqual_mixed' strict=1 len=500 n=500 1.51 % ±6.06% ±8.08% ±10.56%
assert/deepequal-set.js method='deepEqual_objectOnly' strict=0 len=500 n=500 0.17 % ±1.33% ±1.77% ±2.31%
assert/deepequal-set.js method='deepEqual_objectOnly' strict=1 len=500 n=500 0.74 % ±1.25% ±1.66% ±2.17%
assert/deepequal-set.js method='deepEqual_primitiveOnly' strict=0 len=500 n=500 1.66 % ±4.83% ±6.43% ±8.38%
assert/deepequal-set.js method='deepEqual_primitiveOnly' strict=1 len=500 n=500 -1.50 % ±6.10% ±8.12% ±10.57%
assert/deepequal-set.js method='notDeepEqual_mixed' strict=0 len=500 n=500 -1.89 % ±2.97% ±3.96% ±5.15%
assert/deepequal-set.js method='notDeepEqual_mixed' strict=1 len=500 n=500 -6.72 % ±8.62% ±11.55% ±15.21%
assert/deepequal-set.js method='notDeepEqual_objectOnly' strict=0 len=500 n=500 -0.24 % ±2.26% ±3.00% ±3.91%
assert/deepequal-set.js method='notDeepEqual_objectOnly' strict=1 len=500 n=500 -0.36 % ±1.97% ±2.62% ±3.41%
assert/deepequal-set.js method='notDeepEqual_primitiveOnly' strict=0 len=500 n=500 1.23 % ±3.73% ±4.97% ±6.50%
assert/deepequal-set.js method='notDeepEqual_primitiveOnly' strict=1 len=500 n=500 0.78 % ±3.91% ±5.20% ±6.77%
assert/deepequal-typedarrays.js len=100 method='deepEqual' strict=0 n=500 type='Float32Array' * -3.33 % ±3.29% ±4.38% ±5.70%
assert/deepequal-typedarrays.js len=100 method='deepEqual' strict=0 n=500 type='Float64Array' -0.86 % ±4.30% ±5.72% ±7.44%
assert/deepequal-typedarrays.js len=100 method='deepEqual' strict=0 n=500 type='Int8Array' 0.42 % ±4.71% ±6.29% ±8.22%
assert/deepequal-typedarrays.js len=100 method='deepEqual' strict=0 n=500 type='Uint8Array' 0.92 % ±5.78% ±7.71% ±10.08%
assert/deepequal-typedarrays.js len=100 method='deepEqual' strict=0 n=500 type='Uint8ClampedArray' 2.60 % ±6.47% ±8.65% ±11.34%
assert/deepequal-typedarrays.js len=100 method='deepEqual' strict=1 n=500 type='Float32Array' -0.16 % ±5.67% ±7.56% ±9.86%
assert/deepequal-typedarrays.js len=100 method='deepEqual' strict=1 n=500 type='Float64Array' 1.68 % ±5.25% ±6.99% ±9.10%
assert/deepequal-typedarrays.js len=100 method='deepEqual' strict=1 n=500 type='Int8Array' 0.48 % ±6.13% ±8.15% ±10.61%
assert/deepequal-typedarrays.js len=100 method='deepEqual' strict=1 n=500 type='Uint8Array' -1.32 % ±4.46% ±5.95% ±7.76%
assert/deepequal-typedarrays.js len=100 method='deepEqual' strict=1 n=500 type='Uint8ClampedArray' -1.73 % ±5.66% ±7.53% ±9.80%
assert/deepequal-typedarrays.js len=100 method='notDeepEqual' strict=0 n=500 type='Float32Array' -0.04 % ±5.06% ±6.75% ±8.82%
assert/deepequal-typedarrays.js len=100 method='notDeepEqual' strict=0 n=500 type='Float64Array' 0.75 % ±4.23% ±5.64% ±7.38%
assert/deepequal-typedarrays.js len=100 method='notDeepEqual' strict=0 n=500 type='Int8Array' 2.99 % ±6.99% ±9.31% ±12.14%
assert/deepequal-typedarrays.js len=100 method='notDeepEqual' strict=0 n=500 type='Uint8Array' 0.11 % ±6.09% ±8.10% ±10.55%
assert/deepequal-typedarrays.js len=100 method='notDeepEqual' strict=0 n=500 type='Uint8ClampedArray' 1.34 % ±7.68% ±10.22% ±13.30%
assert/deepequal-typedarrays.js len=100 method='notDeepEqual' strict=1 n=500 type='Float32Array' 1.63 % ±5.52% ±7.34% ±9.56%
assert/deepequal-typedarrays.js len=100 method='notDeepEqual' strict=1 n=500 type='Float64Array' 0.55 % ±5.55% ±7.39% ±9.62%
assert/deepequal-typedarrays.js len=100 method='notDeepEqual' strict=1 n=500 type='Int8Array' -3.45 % ±6.88% ±9.15% ±11.91%
assert/deepequal-typedarrays.js len=100 method='notDeepEqual' strict=1 n=500 type='Uint8Array' -1.68 % ±5.58% ±7.43% ±9.68%
assert/deepequal-typedarrays.js len=100 method='notDeepEqual' strict=1 n=500 type='Uint8ClampedArray' -2.25 % ±4.84% ±6.44% ±8.39%
assert/deepequal-typedarrays.js len=5000 method='deepEqual' strict=0 n=500 type='Float32Array' -2.01 % ±4.02% ±5.38% ±7.05%
assert/deepequal-typedarrays.js len=5000 method='deepEqual' strict=0 n=500 type='Float64Array' 1.88 % ±1.99% ±2.65% ±3.45%
assert/deepequal-typedarrays.js len=5000 method='deepEqual' strict=0 n=500 type='Int8Array' -2.50 % ±6.38% ±8.49% ±11.05%
assert/deepequal-typedarrays.js len=5000 method='deepEqual' strict=0 n=500 type='Uint8Array' -2.58 % ±5.53% ±7.37% ±9.61%
assert/deepequal-typedarrays.js len=5000 method='deepEqual' strict=0 n=500 type='Uint8ClampedArray' 0.14 % ±6.07% ±8.10% ±10.59%
assert/deepequal-typedarrays.js len=5000 method='deepEqual' strict=1 n=500 type='Float32Array' 2.24 % ±4.96% ±6.60% ±8.60%
assert/deepequal-typedarrays.js len=5000 method='deepEqual' strict=1 n=500 type='Float64Array' -1.16 % ±6.79% ±9.04% ±11.76%
assert/deepequal-typedarrays.js len=5000 method='deepEqual' strict=1 n=500 type='Int8Array' -0.73 % ±5.59% ±7.46% ±9.76%
assert/deepequal-typedarrays.js len=5000 method='deepEqual' strict=1 n=500 type='Uint8Array' 0.09 % ±6.06% ±8.06% ±10.50%
assert/deepequal-typedarrays.js len=5000 method='deepEqual' strict=1 n=500 type='Uint8ClampedArray' 2.81 % ±4.10% ±5.45% ±7.10%
assert/deepequal-typedarrays.js len=5000 method='notDeepEqual' strict=0 n=500 type='Float32Array' 1.28 % ±4.91% ±6.54% ±8.55%
assert/deepequal-typedarrays.js len=5000 method='notDeepEqual' strict=0 n=500 type='Float64Array' ** -5.56 % ±4.15% ±5.52% ±7.20%
assert/deepequal-typedarrays.js len=5000 method='notDeepEqual' strict=0 n=500 type='Int8Array' -2.71 % ±7.04% ±9.39% ±12.28%
assert/deepequal-typedarrays.js len=5000 method='notDeepEqual' strict=0 n=500 type='Uint8Array' -2.72 % ±5.80% ±7.74% ±10.12%
assert/deepequal-typedarrays.js len=5000 method='notDeepEqual' strict=0 n=500 type='Uint8ClampedArray' 5.31 % ±6.42% ±8.57% ±11.24%
assert/deepequal-typedarrays.js len=5000 method='notDeepEqual' strict=1 n=500 type='Float32Array' -4.42 % ±5.06% ±6.74% ±8.78%
assert/deepequal-typedarrays.js len=5000 method='notDeepEqual' strict=1 n=500 type='Float64Array' -0.47 % ±5.74% ±7.64% ±9.97%
assert/deepequal-typedarrays.js len=5000 method='notDeepEqual' strict=1 n=500 type='Int8Array' -4.16 % ±6.32% ±8.43% ±11.02%
assert/deepequal-typedarrays.js len=5000 method='notDeepEqual' strict=1 n=500 type='Uint8Array' -5.76 % ±6.50% ±8.68% ±11.38%
assert/deepequal-typedarrays.js len=5000 method='notDeepEqual' strict=1 n=500 type='Uint8ClampedArray' -2.55 % ±4.63% ±6.16% ±8.02%
assert/ok.js n=100000 * -4.86 % ±3.69% ±4.90% ±6.38%
assert/throws.js method='doesNotThrow' n=10000 -2.61 % ±5.92% ±7.88% ±10.26%
assert/throws.js method='throws_RegExp' n=10000 -2.84 % ±3.70% ±4.92% ±6.40%
assert/throws.js method='throws_TypeError' n=10000 * -4.54 % ±3.85% ±5.12% ±6.68%
Be aware that when doing many comparisons the risk of a false-positive
result increases. In this case there are 136 comparisons, you can thus
expect the following amount of false-positive results:
6.80 false positives, when considering a 5% risk acceptance (*, **, ***),
1.36 false positives, when considering a 1% risk acceptance (**, ***),
0.14 false positives, when considering a 0.1% risk acceptance (***)
++ mv output030121-160502.csv /w/bnch-comp
Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/872/
These changes come with some perf regressions:
confidence improvement accuracy (*) (**) (***)
assert/ok.js n=100000 *** -6.77 % ±3.69% ±4.92% ±6.44%
assert/throws.js method='doesNotThrow' n=10000 *** -14.90 % ±5.67% ±7.56% ±9.86%
This may or may not be considered an acceptable trade-off.