iptsd
iptsd copied to clipboard
linux-surface
Detect spam touches and then reset
This is from https://github.com/linux-surface/linux-surface/issues/417
It is not humanly possible to touch the screen like 20 times in 1 second. Once a threshold is reached, iptsd-reset-sensor should run automatically. This will prevent the cursor from going out of control and from having the system be unusable.
+1 to this. Taking in acocunt that iptsd-reset-sensor kills the touch on KDE neon , at least on my Surface 4 pro
Idk but reasonable is >=0.1 second per touch
so if it exceed than 10 touches per second then it should be treated as non-sense touching.
IMO
Which parameter is this in /etc/iptsd.conf? I have found that it has parameters, but could not figure out what they are doing.
This is not a parameter in the configuration, this is a suggestion for a new feature.
I like the idea, but there is one issue: With the HID based drivers, there is actually no standard interface for resetting the sensor. There are certain actions that will trigger a reset for sure on devices using the ipts kernel driver, but I am not sure if the same applies to the ithc and spi-hid drivers.
@quo Do you happen to know if there is a way to reset the sensor over HID on newer devices?
The easiest thing for now would probably be to switch to singletouch mode and back to multitouch mode once the threshold is reached. This will definitly trigger a reset on devices before SP7, and it might have the same effect on the newer ones.
I have seen this thing in /etc/ipts.conf [Config] BlockOnPalm = true TouchThreshold = 20 StabilityThreshold = 0.1
Have been played around with this, and some ipmrovement can be experienced, but ghost touches happen too, especially when the screen gets hot (high cpu loads).
But if the driver knows the single touch mode, can it be switched? Or can the developers create a parameter in the config to activate? Because many of who are suffering from heating and ghost touches, may be happy with single touch mode, which is stable; i have seen that no matter how hot is the tablet, in the BIOS and boot screen the touch is reliable.
Thanks for your help :)
You mentioned that there are iptc and spi-hid drivers, may be that they are more stable or pure single-touch? How can i activate those? (If someone knows.)
You mentioned that there are iptc and spi-hid drivers, may be that they are more stable or pure single-touch? How can i activate those? (If someone knows.)
Those are for drivers for different devices. ITHC is for SP7+ and later, SPI-HID is for the Pro X (and the AMD devices, as soon as we get it running there).
You mentioned that there are iptc and spi-hid drivers, may be that they are more stable or pure single-touch? How can i activate those? (If someone knows.)
Those are for drivers for different devices. ITHC is for SP7+ and later, SPI-HID is for the Pro X (and the AMD devices, as soon as we get it running there).
Oh yeah :D I hoped that there are other solutions :D Sry :D
Had time to test with Windows with the previous faulty screen. No or minimal ghost touches, no matter how hot is the screen and system. Maybe the Windows driver is in single-touch mode. At least it did not do anything with two fingers. (But on Linux it doesn't also.) After Windows, installed Debian, ghost touches came back. Tried to install the Git version (Debian bullseye 11.5), and did not work at all.
Which parameter is this in /etc/iptsd.conf? I have found that it has parameters, but could not figure out what they are doing.
idk if you mentioned my above comment? if yes...
that comment Idk but reasonable is >=0.1 second per touch ... I mentioned the implementation given by author of this issue (see 1st comment of this issue)
which is not about iptsd parameter but how I think about his/her implementation
i have seen that no matter how hot is the tablet, in the BIOS and boot screen the touch is reliable.
sorry this related to the issue but not directly @horvathcsabalaszlo
Maybe the Windows driver is in single-touch mode.
@horvathcsabalaszlo does that Windows operating in normal mode (not WinRE not safemode and touch driver is normal)
if yes then Windows should put in multi-touch mode, because everyday life on normal windows boot, it must able to using multi touch.
Oh, understand. So i thought that in single mode the sensor is more stable, or it ignores 2nd touches by itself.
Windows ran on the default driver (i uninstalled it from the test stuff since), Win10 with the driver it downloads by default.
I have tried some pages, Google Maps, but the two-finger zoom did not work. On Linux, it works.
@horvathcsabalaszlo ok I found the same member in linux-surface matrix post the message again about trick to try to avoid ghost touch on SP4 (yeah that previous trick also mean the same device same ghost problem)
maybe you could keep track there too.
btw let give my opinion.
or it ignores 2nd touches by itself.
This is not much necessary to know just FYI, I think it is depends on algorithm of driver, I mean it is not 100% ignore 2nd touches, it may (or maynot) received all touches but only allow only 1 to be outputed to system.
For exmaple, when press 2nd touch while 1st touch persist, some driver/config decline 2nd and still using 1st, some driver/config discard 1st and using 2nd, and some driver/config being in confused state, I mean this state is too technical to explain, but the syntomp is that the both touch is shown but not simutaneously, I mean it fast switching from 1st to 2nd and the another way back)
I have tried some pages, Google Maps, but the two-finger zoom did not work.
wait, have you using driver from Surface? it should work on Windows, or have I missed something.
Thanks :)
"wait, have you using driver from Surface? it should work on Windows, or have I missed something." No, i used the driver Windows uses by default (what is downloaded from MS Update). Was surprising to me.
"This is not much necessary to know just FYI, I think it is depends on algorithm of driver, I" Yes! This is why BIOS screen is a mystery, because it's rock solid. No ghosts. No matter how hot...
Windows uses by default (what is downloaded from MS Update). Was surprising to me.
Ok then you have to reinstall driver from Surface instead, this will may make multitouch work on Windows. (include that maybe normally without ghost touch or if have then idk if less worse than on Linux driver or not)
This is why BIOS screen is a mystery, because it's rock solid. No ghosts. No matter how hot...
IMO it is not mystery, the HW design may not designed to to deal with noise, so the raw output may have problem with noise. I think EFI drvier work well on BIOS/EFI bootloader because the EFI driver is produced by MS which deal with noise.
So I think Windows driver (from Surface) should do too, that why I tell that it is driver or iptsd problem on cutting out noise.
The problem of ghost touches on linux is not new, and the cause is pretty much clear: iptsd does not have any of the noise reduction or calibration that the windows driver has. Same applies for the singletouch mode, our processing is way more basic than that.
Please keep this on topic, this is about fixing the ghost touches by resetting the sensor automatically, not about anything else. We dont need yet another issue about the ghost touches itself.
"Ok then you have to reinstall driver from Surface instead" Where i can find this driver? Or what are you referring as "from Surface"?
Hm, maybe would good to have that driver inspected, sadly i'm only having the basics of programming :(
@StolD : Would be good to have a test with the reset method; as i tried, if ghosts appear, issue the iptsd stop and start command from SSH, it would help only for seconds. :( So would be interesting what effect it would have if it would be automated.
Which iptsd are you using? Out of the other issues you have opened, I gather that you are using the old iptsd from the repo. That doesnt do anything in regards to singletouch or multitouch. The driver will always be in multitouch mode. It also won't trigger any resets when you start or stop the service. On the old iptsd, that is done by calling iptsd-reset-sensor. So that is what you need to run over SSH.
$ apt list|grep iptsd
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
iptsd/unknown,now 0.5.1-1 amd64 [installed]
This is the current one for Debian/Ubuntu 20.04/KDE Neon from the linux-surface repo. But i may upgrade to 22.04 so i may be try the newer ones.
iptsd-reset-sensor previously completely stopped the touch; now it doesn't, will give it a try :) (Or for example, schedule it, or put a shortcut to the tray.)
Thanks! :)
Other suggestion; would worth implementing a simple filter on the touches? I turned on the visualisation of the touches, and observed that if i press the screen for a long time when it's warm, the ghosts are appearing randomly on random points. (It's much rarer when they come without touching the screen.) May it be possible to restrict the detection to an area around the first detected touch? So it won't ruin dragging, but may prevent many of the ghost touches. (I mean if a coordinate is coming that is outside of the r=... area of a detected touch, simply drop and only process the area of the previously processed touch.)
May it be possible to restrict the detection to an area around the first detected touch?
That would be hard to get right. Either you allow for too much distance and still get noise, or you effectively make it a singletouch implementation.
The best way to fix it would probably be temporal stabilization. iptsd would have to keep all contacts around from the last 3 frames and then eliminate every contact that is not present in all three. With the finger tracking we have this should be doable, but I didn't try implementing it yet.
EDIT: An alternative approach is defining a minimal size for inputs. Currently we only have a maximal size, everything above gets declared as a palm.
Yes, it would be a single, but would be work. (In addition, would be switched from config, so if one is struggling with crazy screen, can switch it on.)
"The best way to fix it would probably be temporal stabilization. " Yes, its also sounds good! Many sw-controlled inputs use this method :)
"With the finger tracking we have this should be doable" Combining the two... :) The base is that you mention, defining an area :)
But the last suggestion from you, the use of standalone driver without userspace daemon, seems OK :) So if it will be stable in the future, my Sf4 will be 100% usable, and i think it would help to many other owners.
Thanks for your help :)
The best way to fix it would probably be temporal stabilization
I am worried about numerical stability and diverging errors. The last time someone suggested a Kalman filter I ran into these kinds of problems with it. That or I most likely don't understand the concept.
last 3 frames
Would this improve or worsen rhythm gaming?
An alternative approach is defining a minimal size for inputs
Please do not force this with an option to disable it! My main problem with the old singletouch driver was the threshold being so high that I have to press down my fingerprint not just my fingertip. This minimal size is also similar to the blurring I originally had in my algorithm which I removed because it breaks distinguishing fingers close together.
With the finger tracking we have this should be doable
I am already hoping for such a feature but for the opposite reason. I have a problem where I scroll with my finger and it is momentarily detected as lifted, down, and lifted in a instant. This causes any buttons below to be clicked and is a hazard for mistakes.
reasonable is >=0.1 second per touch
This threshold is way too low. It would be fine if it's a threshold for the number of detected ghost touches but not for normal touches. Some applications like games may need you to rapidly press and release all 10 fingers at once. Doing that once would trigger your threshold if it is not aggregated by any window. Average human reaction speed is 200ms, so that 50 touches per second, bumped up to 55 if you add a safety 10%.
I was thinking of a 3 second window. Based on what said implies, this is too long. A 1 second window would probably also feel slow. Material design animations are 300ms. I would use something on the higher side so that the long sensor reset does not trigger randomly. 700ms seems like a good compromise between those two constraints.
I turned on the visualisation of the touches, and observed that if i press the screen for a long time when it's warm, the ghosts are appearing randomly on random points
A visualization of the underlying heatmap can be found at https://github.com/linux-surface/linux-surface/issues/417#issuecomment-809897595
iptsd does not have any of the noise reduction or calibration that the windows driver has
It's not necessarily the Windows driver that does the noise reduction, it could be the hardware. It certainly seems to be able to calibrate the hardware as I do not observe long waits for the sensor to reset. The problem is that on Linux the hardware suddenly decides to increase the heatmap sensitivity when there is a lot of input. iptsd could even implement this by subtracting out the average, which if I recall correctly qzed's prototype did, but again this would decrease the sensitivity too much for me.
@danielzgtg : Exactly the same i experienced that you show on the videos. touches on some points, but the points at one time are on almost the same place. Pressing the screen with two fingers stopped it sometimes.
And all of these settings should be configurable from iptsd.conf .
@horvathcsabalaszlo
Where i can find this driver? Or what are you referring as "from Surface"?
from Surface mean that driver you have to download from Microsoft Surface website, however you dont have to do this if you install Windows on Surface FROM "factory reset image for Surface"
Hm, maybe would good to have that driver inspected
pls make sure to be not confused, the driver I mean for usage with Windows, (to understand context I have said: I confirm you to using driver from Microsoft Surface website or driver in factory reset image is to make sure that multitouch work in Windows, since it is not normal to got only single touch mode in Windows)
btw to forcibly answer the question, Windows driver may not be good thing to inspect since it is closed source, only thing you get is binary / raw data, and some config text, without proper reverse engineering would get very less benefit from inspecting it, (I mean it is ddifccult to do RevEng)
pls stayed tune for next comment (already commented)
Some applications like games may need you to rapidly press and release all 10 fingers at once.
I forget about this so that mean I need to multiply number of maximum touches to 10 touches per second, it becomes 100 touches per seconds for device support max 10 touches.
I forget about this so that mean I need to multiply number of maximum touches to 10 touches per second, it becomes 100 touches per seconds for device support max 10 touches.
ok so instead of determine over 1 second
just determine over 100ms I mean max 10 touches per 100ms it is better then filter ~50-55 per second
I am worried about numerical stability and diverging errors. The last time someone suggested a Kalman filter I ran into these kinds of problems with it. That or I most likely don't understand the concept.
The finger tracker assigns every input a number. Because it considers the distance between current input and previous input, these numbers should stay stable between putting the finger on the display and lifting it (one exception is two inputs very close to each other where it might jump, but IIRC that is not where noise appears).
My idea would be run after the finger tracking, and look like this:
Frame 1 Frame 2 Frame 3
1 1 1
2 2 2
3
The inputs 1 and 2 are present in all three frames and therefore stable. The input 3 only appears in one frame, and immideately disappears, so it is not stable and dropped.
One problem with this approach is if noise appears in adjacent frames and simply jumps around. But we can fix that by defining an upper limit for movement between frames (we currently only have a lower one: https://github.com/linux-surface/iptsd/blob/master/src/contacts/finder.cpp#L211).
Would this improve or worsen rhythm gaming?
An input would be ignored for the first two frames, so there would be a small latency. After that, the latency should be gone. And the sampling rate of the touchscreen is very high, so I dont think it would be a practical issue. But its all theory of course.
Please do not force this with an option to disable it!
Dont worry, I will add parameters for every size based comparison, and you can use the iptsd-calibrate tool to tweak them to your fingers (or set them to 0 / 1000 to disable them).
iptsd could even implement this by subtracting out the average, which if I recall correctly qzed's prototype did, but again this would decrease the sensitivity too much for me.
iptsd has always done this, even with the basic algorithms: https://github.com/linux-surface/iptsd/blob/master/src/contacts/basic/heatmap.cpp#L23
The issue is that due to the auto-calibration the noise makes the value jump from 0 all the way to 255.
Yes, i know that reverse engineering is hard, but i don't know how the existing driver was written, and what the developers can do :)
@StollD : "My idea would be run after the finger tracking, and look like this:" I think it's good, some kind of "averaging", which is very common is sw input processing. In addition, the parameters should be also good, i think the possibility to tune the stuff is always a good thing.
I used to have averaging in a very old version of my algorithm. All it did was cause the cursor to lag so far behind that it could be visible outside of your finger. This makes it difficult for one to paint pictures in drawing programs.
If you want averaging, then you also need prediction. I had already been thinking of prediction to smoothen the scrolling a bit more. I never got to implementing it as this prediction component would be at least half as complex as the qzed notebook.
What @StollD sketched up, with the touch panel sampling rate, i think will not lead to such massive lag.