Smoothing and average calculation and display is weird/broken/unintuitiv

[trapicki]

The current average calculation seems to reflect the method to calculate the smoothing. The smoothing seems to be calculated as a incremental average with some weighting factor that weights the beginning much stronger then the end of time series. This leads to strange effects:

• If a time series starts with a low value, even high values later will not allow the last averaged value to exceed a certain limit. This in turn leads to very slowly progressing quality indicators (e.g. speed) when you started slow with a letter. This is extremely frustrating.
• In the statistics, the scale for averages is changing depending on the smoothing. The scale only shows the values included in the smoothed data set, not the original values. Thus it's impossible to understand the graphs when changing the smoothing slider changes the scale.
• On high smoothing values, the history flattens out, which makes it more and more useless.
• In the per-letter speed graphs, the target speed sometimes show up, sometimes not. Confusing.
• For some values, smoothing makes no sense, for example "number of letters".

Suggestions:

• Calculate the average speed used for the quality judgement of a letter via an IIR, with more weight of recent data instead of old data.
• Always show the exact data points in the diagrams (maybe faded). The averaged data can be shown as a segmented or splined line on top of that.
• For diagrams, the weight should be the number of samples included by averaging, maybe with a nice rolloff (Gaussian, or Consine-Rolloff) weighting. For trends, calculate multiple values like "last 3 sessions", "last day", "since beginning" for better interpretation of the values.
• Keep the scale constant for different smoothing factors.
• Allow to always include 0 (speed) or 100 (accuracy) in the diagram.
• Please consider logarithmic diagrams. For accuracy, going from 90% to 95% is probably as hard as going from 95% to 97.5%. For speed, increasing by 10% should always be the same effort. This should be reflected in the diagrams.

[hjhicks]

I came here to write this exact issue. The data it displays can sometimes be flatout wrong. Here are a few of my thoughts:

• I'd like to see raw data with no manipulations made to it.
• I'd like the axes on the graph to have a constant scale. It drives me crazy when the per-letter graphs suddenly completely change ranges and scales after a new lesson. I'd really like to be able to look at two graphs and easily be able to compare them.

[geroldmeisinger]

I'm learning a new keyboard layout with keybr (great software btw!) and have the same issue. After two weeks I still don't know what's going on and at this point I'm too afraid to ask.

• the scale of the graph for each letter is changing all the time. I assume there is some weighting which only uses the last x data points depending on the history (sometimes just 5, sometimes 30). I would like to keep it constant (always show last 30 data points).
• what does the WPM value mean for an individual letter? is it the time it takes to press the letter after another letter multiplied by the average word length? "As you type, the algorithm collects your typing statistics, such as the time-to-type metric for each individual key." but how exactly? I couldn't find any documentation on this value. and it doesn't feel like the actual value is reflected in the graph? when I think I had a very good run it seems only "slightly better" than the previous entry.
• multiple times I had the case where it takes about 50 lessons to unlock a new letter (which is expected) only to unlock the next 2-3 letters in a single lesson each. I assume there is not enough data to calculate a stable average and I made some lucky hits. I think there should be a minimum of 5 lessons for each letter to get enough data first. if a new letter wasn't unlocked in the first try the graph complains about "Need more data to compute the remaining lessons to unlock this letter." and suddenly it takes much longer to "clear out" a bad start. it seems the first lesson is weighted more? or it's just harder to "correct an average" with more data. here is one of this letter:

Apparently I lucked out with 50wpm in the first lesson. it's now constantly decreasing because my overall average across all letters is about 30wpm (on the new layout). still it's nowhere near the real value. consequently the letter is and stays green for much longer than expected. I don't understand what those points mean but they cannot be the raw wpm value given the distance of V on my layout. the reverse happens with a bad first lesson. the letter stays red for much longer than expected.

• I don't understand what the data points in "smoothed" graph mean. there is the trendline which is the overall average and all the datapoints are "weaker" averages. so it's smoothed datapoints on a smoothed line? it's also kind of strange that if you move the smoothness slider all the way to the right it evens out on the first datapoint(?), the first datapoint is always fixed(?). shouldn't it even out on the average?

if you go to profile -> download data this is what you get:

[
	{
		"layout": "en-us",
		"textType": "generated",
		"timeStamp": "2024-11-24T20:02:21.000Z",
		"length": 126,
		"time": 99337,
		"errors": 12,
		"speed": 76.10457332111902,
		"histogram": [
			{
				"codePoint": 32,
				"hitCount": 22,
				"missCount": 0,
				"timeToType": 215
			},
			{
				"codePoint": 97,
				"hitCount": 21,
				"missCount": 1,
				"timeToType": 660
			},
			{
				"codePoint": 101,
				"hitCount": 28,
				"missCount": 1,
				"timeToType": 623
			},
			{
				"codePoint": 105,
				"hitCount": 6,
				"missCount": 1,
				"timeToType": 954
			},
			{
				"codePoint": 108,
				"hitCount": 9,
				"missCount": 3,
				"timeToType": 998
			},
			{
				"codePoint": 110,
				"hitCount": 15,
				"missCount": 2,
				"timeToType": 590
			},
			{
				"codePoint": 114,
				"hitCount": 24,
				"missCount": 4,
				"timeToType": 979
			}
		]
	},
        ...

[trapicki]

Regarding the current algorithm:

It seems like it actually is IIR (Infinite Impulse Response), progressive filter that is used at least for the Profile charts. With the automatic scaling leading to such strange results:

The smoothness filter for the display in https://github.com/aradzie/keybr.com/blob/30d1b8514cfebe78b282b48b871f1bdb8ac028d1/packages/keybr-math/lib/filter.ts#L11 walks through values starting from the first and updating the running smoothed value by weighting everything up to the current sample and the current sample. That is the very definition of an IIR. The result is that the first data point is unaffected, and the other points are progressively smoothed out.

This would need some change to look at a limited number of values. The seemingly strange effect that the first values seem unfilter could be fixed by back-projecting the progression for the first values that do not have a history. But that would be only for cosmetics.

[trapicki]

The code for the chart plotting and scaling can be found in https://github.com/aradzie/keybr.com/tree/master/packages/keybr-chart/lib, for example for Speed it is https://github.com/aradzie/keybr.com/blob/30d1b8514cfebe78b282b48b871f1bdb8ac028d1/packages/keybr-chart/lib/SpeedChart.tsx#L13 The Speed range of the chart is calculated from the smoothed values. This could be changed so that a more stable scale is used, like the range from 0 to the maximum Speed.

[makerio90]

iirc a IIR filter with a smothness of 0 will result in unfiltered data. but the unfixed data ranges can only be done with code modification. ill see if i can come up with a userscript.

[florian-13]

I think far better than the currently used smoothing method would be the "Moving Average". For this you simple simply display the average of the surrounding n values as the point value (higher values of n would mean higher smoothing)