NekoSMS
NekoSMS copied to clipboard
apsun
Add settings that can block all incoming SMS from unknown numbers and alphanumeric senders by default
Hi, first of all thank you for the great app.
Since I am its active user I decided to offer an improvement. The whole thing is about blocking SMS spam from advertising companies and scammers more efficiently. Currently app's default strategy can be called "enable everything that is not explicitly blocked". I personally think that SMS messages today have too little use cases, and using this strategy will require user to put a significant effort in setting up blacklist rules and some spam will still get through.
I do not use SMS to send texts, because I communicate with all people in messengers, like most folks nowadays probably. Perhaps once in a year I can receive an SMS from a friend or a family if they have missed the payment for the internet on a SIM card and have to write a normal SMS, but it will certainly be people from my contact list. There are exactly zero cases when I need to expect an SMS from an unknown phone number. I really need to receive a normal SMS from an alphanumeric sender sometimes, when I need to get a 2FA or a confirmation code, but all these senders are known to me and I can easily whitelist them. So to summarize it all, in my case 95% of all times I get an SMS means I got a spam that distracts me from my business and that is frustrating.
In my country SMS spam is big problem and spammers use two main strategies:
- Send spam from mailing services that can set an alphanumeric SenderID instead of a regular phone number. Such messages cannot be entirely blocked on both iOS and Android, since both operating systems consider such messages important and spammers know that. Even if you explicitly block the sender ID, the next time it sends you a spam message you will just avoid the sound notification, but the message itself will still be delivered to a SMS mailbox and you will have to delete it manually, and read it in process anyway (that was the reason I started to use NekoSMS at the first place). The typical spammer in this case is a legal business that annoys its customers with offers, sales, etc.
- Send spam from a valid phone number, but the number itself is a one-time number from a huge pool. Typical spammers in this case are scammers of all sorts, unscrupulous advertisers, illegal casinos etc. They bombard selected numbers with SMS but change phone number after every message, this makes it impossible to block them, since blacklist on Android can only accept one number at a time, no masks, regular expressions etc.
The solution for this problem I think is to give user an opportunity to use the strategy "block everything that is not explicitly enabled".
I added two checkbox options in the settings menu of the app. First will block all incoming SMS from alphanumeric SenderIDs by default unless there is a whitelist rule that explicitly allows this SenderID to send user an SMS. This allows you to ignore most of random advertising from companies that got your number somewhere and decided to impose their services to random people. And it also solves the problem of granular permissions of SMS contents. For example I want to see 2FA codes that my bank sends me in SMS, so I gonna create a whitelist rule with bank's SenderID and word "code" in message body and I will get those codes. But I know that the bank will also occasionally push me the advertisement of their services and that is surely something I don't want to see, so with this approach I don't even have to do anything to get rid from this kind of spam - very convenient. Second checkbox will block all incoming SMS from phone numbers unless there are explicit whitelist rules or the checkbox "whitelist contacts" is on. This perfectly solves the problem with scammers. As I mentioned I never expect SMS messages from strangers and people that I know will still be able to send me a normal SMS if they will decide to do so for some reason.
Both checkboxes are optional and disabled by default, so user experience remains the same, but people will get an opportunity to use new approach in blocking SMS spam on their devices. I think it will improve user experience in most cases and it makes the app cover 90% of everything a typical user expects from such application straight out of the box. People who want to use normal approach with manual blocking can just ignore new options and continue to use the app as before.
I tested both options, including cases when sender is in the contact list or in a whitelist rule. Everything works great.
Here is how it looks:
Hope this change will make its way to the master branch and thank you again!
Wow, thanks for the awesome PR! Glad to hear NekoSMS is useful to you :-)
Regarding the functionality, I'm curious if you could get the same results with regex filter rules. For example, blocking all non-numeric senders could be matched by adding a whitelist regex rule to match anything you consider "numeric", and then a .* regex blacklist filter to match everything else. And for blocking all non-whitelisted senders, just the .* blacklist filter should be sufficient when combined with the existing "whitelist contacts" setting, I think.
I've never actually seen a non-numeric SMS sender field, so that might complicate things?
This can be done by creating blacklist/whitelist rules as you mentioned, but it requires some technical knowledge and understanding of regular expressions for example. A simple toggle in the settings looks more user-friendly.
I live in Russia and non-numeric senders are a real pain here. Every store and every business use any opportunity to get your phone number and then push unstoppable SMS spam to it (since both Android and iOS cannot completely block it as I mentioned before) or sell your phone number to third party spammers. On my phone SMS mailbox currently has messages from "VTB" - that's a bank, "Orange-Fit" - some fitness club I think, no idea who is it, "YOUTRAVELME" - a notification about an incoming message from a website where I had registered, "KUPIBILETRU" - website where I bought tickets online, and a few dozens more. That's the spam I would like to get rid of, except the bank.
Ah, I see. I personally think the app should offer just the building blocks, and not duplicate this functionality with settings if it can be replicated with filter rules (in the spirit of "there's only one right way to do it"). Blocking non-numeric senders seems like a fairly specific usecase, so I'm not sure this makes sense as a setting that all users would see.
That said, "block everything except what is explicitly whitelisted" is a pretty common usecase, and it might not be intuitive that a regex of .* is the right way to do that (especially for people unfamiliar with regex). I think that would be something worth adding as a setting, since it is applicable to a wide range of users. Thoughts?
As an engineer I agree that "offer the building blocks" in general is much better approach than offering both general customization opportunities and some selected specific use cases.
The idea I implemented here is the same "block everything except what is explicitly whitelisted", but with one important difference. It was important for me to split "all senders" in two categories - "all phone-number senders" and "all non-phone-number senders". This is important for RU/CIS area and the entire pull request is inspired by my personal experience. As I mentioned above we have two categories of spammers. And what's most important - this two categories usually never get crossed. It means that a person in 95% cases will get spam only from one of them, and will not get spam from another. That's why it is important to separate them. I'll try to explain it.
First category could be called "legal", it includes enterprises, businesses, stores with SMS notification and annoying loyalty programs. I call them "legal" because in Russia using alphanumeric sender IDs as well as advertising mailings by SMS are legal, but the law restrictions require those companies to register their sender ID in government communication ministry, and when some mailing operator receives an order from a certain company to send SMS messages to a list of phone numbers it have to check that this company is officially registered in communication ministry and it must authenticate the company, ensure that the order came from a real company representative, not some random person who want to impersonate it. This law also requires companies to stop sending any advertising if client officially asks for it. Actually it is a very good approach and it makes sense, because users can address their complaints and abuse letters to government ministry instead of wasting time trying to outplay company's shady lawyers in the court. Government ministry inspects the case and can apply fines to unscrupulous companies and fines for breaking this law are significant.
Unfortunately there is no legal separation between notifications that are absolutely necessary to use company's product and the one that basically a spam. Usually it means that if the company has officially registered itself and got the right to send you SMS notifications, it will send you both important stuff like 2FA codes and also "information about company's products and services" which is a spam. The ratio, as you can guess, is frustrating. In average you will get 10 SMS with a spam to 1 sms with 2FA code. And unscrupulous companies just deny their services completely if you ask them to stop the spam. They stop it (because fines are high as I mentioned), but they also stop sending all the notifications, like 2FA codes etc, which makes it impossible to use their services at all. This is popular strategy for a number of companies, and of course it is right to fight this in a legal field - create new laws that will restrict such things, like people do in EU. But that is a long and difficult way, and an average user just wants to stop the spam here and now. The unscrupulous companies know that there is no effective technical way to block SMS spam on modern mobile operating systems and they use it. A lot of companies add "handling personal data by the third parties" lines in their service contracts, and this "third parties" often happen to be advertising SMS spammers that get a clear permission to send you SMS notifications this way. And those companies can also use "all-or-nothing" approach when a customer sign a contract with them, so you can whether use their services and inevitably be spammed, or you will not be spammed but you also cannot use their services either. Fortunately, technically savvy folks can use NekoSMS and stop the spam technically with "block everything except what is explicitly whitelisted" approach and whitelisting the minimum required stuff. And that's just enabling one toggle in the setting added in this pull request.
Second category is "illegal" spammers, because they use valid phone numbers but those are one-time-numbers to spam as I mentioned in previous message. Usually it is shady "business" that cannot register themselves officially and get themselves an alphanumeric sender ID. The law clearly forbids them to send SMS notifications. This category includes illegal gambling, casinos, and most importantly loansharks. Loansharks is a big problem is RU/CIS. Those are private companies that collect debts. In Russia you can collect debts not only by asking the government/court/police, but also with these companies. I don't understand why do they still exist. This entire area is extremely grey from the point of the law and most people in it are often indistinguishable from gangsters or thugs. They are restricted by the law in many ways, but they use every opportunity to terrify people if that technically doesn't break current laws. Sometimes they or their speaking bots can call you 24/7, and bombard you with SMS. They use pools of one-time-numbers so it makes it impossible to block them. I never borrow money myself but I once suffered such call/SMS bombarding when some guy borrowed money from a loansharks and left my phone number instead of his own. It took me a lot of time to get contact the company and get excluded from those mailing lists. You can easily find yourself in such spam list also if you buy a SIM-card with a number that previously belonged to someone who borrowed money from the loansharks or left his number in a few gambling places etc.
I get pushed a lot by the spam from the first category of spammers. So it means that I gonna always keep the first toggle turned on, and whitelist selected sender IDs. But sometimes I need to receive a confirmation code or a 2FA code, I expect it during a whole certain day, I don't know the exact time and I don't know the exact sender ID to whitelist it. Then it would be great to quickly turn off the first toggle for a day, receive a required message and then turn it on again. I don't get any spam from the second category, but if I will, I want to have a possibility to quickly turn on second toggle and block it all until I settle everything's up with, for example, debt collectors, and then turn it off when I don't get SMS-bombarded from one-time-numbers again. So it is very important to have a possibility to block only one particular category of spam SMS and I don't block normal phone numbers, or vice-versa. And it is also important to be able to quickly turn it on/off separately.
The reason I offered to add it to the general settings is because otherwise for this specific case user would have to manually add two complex regular expressions to blacklist - one regex that would check that the sender is a valid formatted phone number and another regex that checks that sender is anything but a valid phone number. Currently, app does not provide a possibility to quickly enable/disable selected filtering rules, and that's OK, because with initial "enable everything that is not explicitly blocked" approach we usually make a blacklist rule once and never turn it off anymore.
I agree that this entire pull request might be too specific for RU/CIS area or even too specific for my personal experience but before offering this pull request I have checked the internet and found out that users from Latin America and some other Eastern Europe countries also face similar problems. So I thought that this change might be useful for them as well. Also both toggles are disabled by default and I have added a description to not confuse users and make sure they turn it on only if they understand what it is made for. Of course accepting or denying this change is completely your decision, and I cannot insist it. If that does not fit the vision of the project then that's completely OK, I'll just keep a fork version for myself, but I feel that this can be useful for other people too.
Sorry for the late reply! I've been busy with other responsibilities recently. I thought about this some more and do think it makes sense to treat "non-numeric" senders specially, assuming this is not something that normal end users have access to. So regarding the intent of this PR, I take back what I said earlier - this does make sense to me now :-)
With that out of the way, the other hard part that remains is defining what is "non-numeric". I'm reluctant to add features that are not universally applicable to all users - the regex you use ^[+]*\\d*[(\\s-]{0,2}[0-9]{1,4}[)]{0,1}[-\\s./0-9]*$ seems quite complex. In case you didn't know, all numbers are normalized to E.164 format (e.g. +1 (234) 567-8900 -> +12345678900) before being matched against filter rules. I don't think matching against dashes/spaces/parentheses actually does anything - could the check be simplified to just ^\\+?[0-9]+? That would be a lot simpler and can be used regardless of country/carrier/etc.
Hello again. Wow, great! Thank you! To be honest I hadn't look closely into SmsFilterPattern constructor and haven't noticed that numbers are normalized to a standart form, so made a regex that would cover most of possible cases. Of course it makes sense to make it as easy as possible.
I suppose I will simplify regex to "^\+?[0-9]+" and run a few checks on different devices with different versions of android, just in case, make sure that SenderIDs of phone numbers are correctly normalized and regex works and be back to you with results.
Thank you again!
Hi, sorry for the long delay. I had been busy all previous week, and this one too. The change we have discussed is still in my plans, I haven't forgotten about it. Please don't close this pull request :) I will do it as soon as I get some free time to make the fix and carefully retest all the cases.
I do wonder - given that the rule for "numeric" senders is not particularly difficult (which was part of your motivation for adding this in the first place), would it be better to instead add the ability to toggle on/off individual filter rules? That would remove the need to special case numeric senders. Thoughts?