element-meta icon indicating copy to clipboard operation
element-meta copied to clipboard

Published 20 hours ago • verified publisher icon element-hq

Invite spam harassment counter measures

Open konomikitten opened this issue 1 year ago • 22 comments

Your use case

What would you like to do?

  • I'd like to be able to report invites. At the moment the options for invites are: "Reject" "Reject & Ignore" we need a third option of "Reject, Ignore and Report".
  • The option to turn off invites completely.

Why would you like to do it?

There has recently been a massive harassment campaign using invite spam. Bigots who perpetrated these attacks used invite spam to achieve it. Because invites allow them to both send their full username and host name they can put threats, abuse and slurs in them. Also avatars are sent as well enabling them to put offensive images to the person they are invite spamming.

Also because there is no report option for invites users receiving threats and abuse have to go through the tedious process of using [email protected]. A lot of users probably don't even know this exists and therefor have no one to reach out to to report this abuse.

How would you like to achieve it?

The client should be updated to allow ignoring all invite requests and have a third option of Reject, Ignore and Report for invites.

Have you considered any alternatives?

No response

Additional context

No response

konomikitten avatar Jul 25 '24 03:07 konomikitten

This is very much needed, and can affect certain communities constantly... (making the "occasional" designition very inaccurate)

emi-rose avatar Jul 25 '24 03:07 emi-rose

I also found the Occasional tag to unjustified, for a lot of us this happens frequently.

konomikitten avatar Jul 25 '24 03:07 konomikitten

A lot of the users on our matrix server have recently been complaining about invite spam and asking if they can turn off invites as a result. This would be a good feature to be able to recommend to them.

supakaity avatar Jul 25 '24 04:07 supakaity

Unfortunately I'm still receiving harassing invites while using the matrix.org home server. At this point until The Matrix Foundation put more protections in place or even give an indication of wanting to do so anyone reading this should consider element and the matrix protocol open to abuse/spam and not worth using.

If you are targeted there is nothing you can do to protect yourself. It's not worth your time to use.

konomikitten avatar Jul 25 '24 10:07 konomikitten

I want to note that the Matrix specification doesn't support reporting spam invites to the homeserver, for the cases where invite spam isn't coming from an overall spam homeserver.

The Matrix specification also doesn't support ignoring invites.

This Week in Matrix 2019-08-30 brought up invite spam and quoted 3 counter-methods from @turt2live, but only method 2 there (use of synapse-simple-antispam or Mjolnir) really scales, unless you want to automate the Synapse admin API from method 1 I suppose (but that's still specific to Synapse).

Invite metadata was known to be a spam vector as far back as 2015.

Other related issues I found:

bb010g avatar Jul 25 '24 22:07 bb010g

Thanks for the ping @bb010g and detailed link tree! The spec does have https://github.com/matrix-org/matrix-spec-proposals/pull/4151 now for reporting rooms to the homeserver, and matrix.org specifically monitors that report stream now too. Other (Synapse) server admins will see them appear under the room_reports table - an admin API like for event reports is planned, but not expected in the near future (sorry, lots to do on T&S generally).

Element iOS and I believe the other Element mobile clients support this API specifically for reporting rooms from the room directory, but it should be trivial to expand to other areas of the app. If T&S gets some time, we'll be contributing that change to Element Web/Desktop ourselves. In the meantime, PRs to add the feature are greatly appreciated.

The missing piece is definitely the ability to ignore rooms/invites entirely, as rejecting them can lead to further abuse. There's a few MSCs, as you've noted, with the addition of https://github.com/matrix-org/matrix-spec-proposals/pull/2270 - we're currently undecided on which direction to go, but the general approach of being able to ignore rooms is leading the process.

Another thought is to limit invites to "contacts only", assuming we can also specify what "contacts" are. This requires a lot more work to get implemented, but would help cut down on the spam at the source rather than being reactive. I also don't have a timeline for this project, but it's certainly top of mind right now.

We're a small T&S team at the moment and looking to expand once funding makes itself available. Folks can help us solve the invite issues by opening PRs for reporting rooms, contributing thoughts to the ignoring invites MSCs, and theorizing about how a contacts-only invite approach might work (and what other levels may be included in there). This is obviously quite a big ask, but we appreciate any movement folks are able to contribute towards solving these issues. We'll get to it ourselves in due course - we 'just' need to finish some of our other projects first.

turt2live avatar Jul 26 '24 00:07 turt2live

Thank you for opening this issue. This is a valuable discussion and the issues are very real. Since this touches on Trust & Safety in a broader sense, beyond Element's clients, I thought I should weigh in as Managing Director of the Matrix.org Foundation.

T&S is a definite sore spot right now, and that's why T&S forms the largest part of our small staff and is one of only two areas where the Foundation is actively funding development.

We are addressing T&S at the protocol level, and with tooling for homeserver admins starting with Matrix.org, and coordinating with client, server, and other developers in the Matrix ecosystem. Further, our roadmap for 2024 includes hiring another moderator and convening a ecosystem-wide Working Group on T&S to help us move further, faster, together to ensure the Matrix open federation is a pleasant place to be.

Of course, I don't expect anyone, much less people who have to cope with regular harassment, to take me at my word. Judge us on results.

Again, thank you. Keep speaking up and, where and when you can, chip in. This is definitely a communal journey!

bluesomewhere avatar Jul 26 '24 03:07 bluesomewhere

At this point until The Matrix Foundation put more protections in place or even give an indication of wanting to do so anyone reading this should consider element and the matrix protocol open to abuse/spam and not worth using.

@konomikitten have you considered that one can block matrix.org? If you're using Element as client you can also use it's option in labs to use policy lists as block list, and wildcard ignore @*:matrix.org, which will reject any invites from matrix.org users.

Edit: didn't mean to imply any bad blood towards the matrix.org staff, I have to admit I've had a positive time working with them related to these kinds of attacks.

TheArcaneBrony avatar Jul 26 '24 15:07 TheArcaneBrony

If you're using Element as client you can also use it's option in labs to use policy lists as block list, and wildcard ignore @*:matrix.org, which will reject any invites from matrix.org users

Does this only ignore invites or will I be ignoring my friends and contacts who also reside on the matrix.org server?

konomikitten avatar Jul 26 '24 15:07 konomikitten

They work as regular /ignore's, so i'd guess so? alternatively you could use the glob pattern on mxids, eg @*badword*:* would block all mxids containing badword on any server

TheArcaneBrony avatar Jul 26 '24 20:07 TheArcaneBrony

Hey @turt2live, some of the thoughts I've been having about this issue:

Even just having the ability to (silently?) drop, reject or blur/mask invites at the client level where the contact isn't in a room that you're in would be good as a first stage.

Going further, a feature like identifying that a "User is in {a / X} room/s that you're in" or "User is a friend of {a / X} friend/s", etc as a hint to "Why am I getting this invite?" would be good, and leave the invite blurred until the invited user decides if they want to reveal the details. Especially mixed in with the ability to specify conditions for dropping/masking invites. e.g:

When user is unknown to you (cold contact):
( • ) Silently ignore invite
(   ) Automatically reject invite
(   ) Blur invite details
(   ) Display invite normally
(   ) Automatically accept invite

When user is in a room you're in:
(   ) Silently ignore invite
(   ) Automatically reject invite
( • ) Blur invite details
(   ) Display invite normally
(   ) Automatically accept invite

When user is a contact of an existing contact:
(   ) Silently ignore invite
(   ) Automatically reject invite
(   ) Blur invite details
(   ) Display invite normally
( • ) Automatically accept invite

etc...

It doesn't remove the need for a protocol level reporting of bad actors, however a simple feature like this at the client level would allow users to have more control over the experience and not leave them feeling as vulnerable and powerless as they do at the moment.

Additionally, from the bad actor's side, knowing that they can be/probably are being ignored can reduce some of the "thrill" they receive at exploiting an unbalanced power dynamic, by empowering their targets, thus disincentivizing the "reward" they get for their bad behavior.

supakaity avatar Jul 29 '24 05:07 supakaity

I just want to add to this issue that I logged in with the matrix client Fractal recently and all the abusive rooms I was spammed with are listed under Historical after using Forget Room in Element.

So having rooms persist also aids in the abuse mentioned in this issue.

Again it's super disappointing that as far as I know no progress has been made on protecting matrix users from this type of abuse.

konomikitten avatar Oct 05 '24 23:10 konomikitten

Has there been any progress on this? I have an account on matrix.org that is getting spammed with invites including illegal material in the room avatar (CSAM, specifically, which in a lot of countries is illegal to store on a device even in cache as far as I know). I've sent in a few reports, but I can't see any way to prevent the invites, or even to block the users sending them without joining a room with them, which is a huge issue as I do not want CSAM on my device from entering one of these rooms, or to be targeted by them because I've initiated DMs.

At the very least, being able to block users from the people directory is a must, but reporting invitations to the homeserver (and to one's own homeserver, in fact, if they contain images that are illegal to store) and preventing further invitations to the room being sent, or an option to turn off receiving invitations entirely (or at least from people one doesn't share a room with) would be very welcome.

deepspaceaxolotl avatar Apr 27 '25 12:04 deepspaceaxolotl

@konomikitten Policy server is a thing now: https://matrix.org/blog/2025/04/introducing-policy-servers/

ShadowRZ avatar May 21 '25 07:05 ShadowRZ

The NixOS community is currently heavily targeted with invite spam (invite mentioning CSAM but I haven't joined the rooms obv), to the point that we've (temporarily?) made our main chat/support room invite-only.

raboof avatar May 23 '25 07:05 raboof

I'm working on releasing a FOSS project, and I wanted to use Matrix as a way to communicate with the community. I'm also already part of a few FOSS Matrix rooms (Cinny, GNOME, Fedora). I opened up Element for the first time in a year while in a public setting. And unfortunately, my invites tab had abusive invites in it. Thankfully, no room preview images were loaded--I only saw the room names. But this experience was negative, scary, and jarring. And seeing that this issue has been known and unsolved for a while, I cannot consider Matrix as a platform right now for my community.

@supakaity 's suggestions seem like an excellent stop-gap solution for this issue until it can be addressed at the protocol level.

I've not contributed to client development/protocol development for a few years now, so take my suggestion with a grain of salt. But I'd love for a way to record my preference to ignore/not receive all invites from untrusted users, store that preference with my homeserver/in a policy room, and have my homeserver automatically filter out unwanted invites. MSC3847 and MSC4155 seem to closely match that.

I understand that this an issue for Element, but it seems like a lot of conversation about the issue (irrespective of whether it should be solved client, server, or protocol-side) is happening here, so it feels relevant to post in this issue.

EDIT: I do greatly appreciate the work the Matrix team does, and I would love to use the platform in the future for my projects. Just wanted to share this because it is a blocking issue right now for me.

jamesjulich avatar Aug 01 '25 00:08 jamesjulich

Sorry for the year-long disappearance on my part - we on the Foundation's T&S team have been hard at work dealing with invite spam and other forms of abusive content. There's obviously still work to do, and we're actually in the middle of deploying more tooling behind the scenes as I write this, but we're committed to doing that work and taking on whatever new challenges are thrown at us.

We're in a much better position today than we were a year ago due to the developments not only within our team, but the broader Matrix T&S community of developers, safety experts, partners, adjacent teams (Synapse, Element, etc), and moderators. This doesn't mean we think we've solved invites - we know we haven't. It means we have the tooling necessary to prevent the majority of abusive invites from making it to end users and to clean up what we find makes it through (and then promptly patch that hole in the net).

Looking just at our team, there's quite a bit we've done over the last 12 months. I've tried to cover as much as I can of our work below, but almost certainly missed something, sorry. As mentioned later, we're working to improve how often and much we communicate publicly about our work, which should make posts like this both shorter and more frequent.

July - October 2024

As a bit of background, we have a T&S firewall installed on matrix.org which allows us to rapidly respond to spam incidents and protect at least our users from the harm. It's not perfect for a decentralized environment because protecting one server's users from harm doesn't mean other users aren't becoming victim to the harm. It does allow us to minimize the abuse potential though, and analyze the spam itself to build more sophisticated tooling in the fallout.

We were putting this firewall to work and concurrently discussing quick-win solutions to invite spam. We were also in the middle of rolling out authenticated media, and were giving that higher priority due to the higher impact at the time (our team is quite small, as I'll mention later, which means while we want to take on every possible project, we simply can't).

Meanwhile, we were deploying measures for in-room NSFW detection and attending the Matrix Conference (we were presenting there - this took some bandwidth away from our team). We were also wrapping up our implementation and internal deployment of account suspension, finally giving us a less destructive mechanism to stop various forms of spam. This was paired with mass redaction APIs in Synapse to help us quickly remove abusive content from rooms.

Finally, the Foundation's Governing Board was just starting to come together and was building a focus for T&S work across the ecosystem.

November 2024 - January 2025

As a small team, we were strongly impacted by (well deserved) holidays, but were still able to get through a large amount of work. We brought on a frontline team to take on the ticket queue (previously this was done by team leadership), immediately giving us more capacity to take on more projects. As part of this onboarding, we continued development of our internal investigation tooling that we'll open source when it's ready (or replace with other open source tooling - more on that later).

We were wrapping up our authenticated media rollout, preparing for FOSDEM, and starting our efforts to support the auth team in their MAS rollout (turns out T&S tooling cares a lot about how authentication works - sorry auth team 😅). Our CSAM mitigation work was also largely considered complete enough to put it mostly on hold to allow more capacity for other projects - we haven't stopped working on it, even today.

During this time we were continually making adjustments to our tooling generally, both publicly and privately. Things like sensitivity of certain content filters and tweaks in process to more heavily lean on account suspension where we can.

February - April 2025

Because of my dual role as T&S team member and SCT member, our capacity was pretty badly impacted this time by the MAS rollout. We were still iterating on our processes and approaches for spam mitigation though, including writing down our collective (internal) wishlists for tooling we want, and sending that through a few rounds of prioritization. This lead to a number of high priority quick win projects on our part:

  • Implementing and merging MSC4260: Reporting Users
  • Push notification suppression on spammy events
  • Give users a button to turn off invites
  • Hide/blur avatars on invites
  • Dedicated buttons to report invites
  • Iterate on how we do room shutdowns
  • Adjust our rate limit settings
  • Tweak the media quarantine mechanism to reduce chances of repeat media

Several of these were already in progress, but the others we began implementation on right away. For a few projects we temporarily shuffled people around to give T&S more capacity too. Concurrently, we posted about the work we're doing (and the funding needs of our team), and announced that we're switching to a curated room directory.

We also spent a bit of time thinking about our "reporting v2" project, but quickly put it on the shelf in favour of the above projects.

That was all in February. By March, we were tracking issues with redactions in rooms (and working to fix it), as well as supporting the Element team in their public content display policy, image handling (also here), invite reports, and starting support for MSC4155 filtering (exposed as a simple on/off toggle with more complexity underneath).

We then spent the majority of April prototyping and designing a concept of policy servers, including deploying the prototype to production in the public rooms we moderate.

Behind the scenes, we started work on a fairly blunt tool to automate the detection and processing of abusive invites. This tool currently catches a lot of the invites we (used to) see, and is being iterated on to date to catch even more.

Finally, our MAS rollout finished, giving us that bandwidth to pour into our suite of spam mitigations and support teams in their efforts to support us in our mission.

May - July 2025 (today)

In this quarter we've spent the vast majority of our time iterating on what we started the previous quarter: ensuring end users have the buttons they need to protect themselves, setting up more sophisticated policies in our policy server, deploying synapse-http-antispam to really put our new-found subscription to the Community Moderation Effort (CME) ban list to work, and improving the reliability of redactions for rooms.

Though, once again because I'm also an SCT member, our capacity in recent weeks has diminished a bit due to the room version 12 rollout (which still has some T&S features, and causes a couple of challenges for those same tools).

and more

I'm certain there's more that should be in this post, but I'm about 2.5 hours into writing this and looking at a field of tabs to source information and links for the above. At some point we were experimenting with the exact messaging we used to communicate that a room was shut down, and we started going back and removing invites that were previously sent (where we could - there's some client limitations here that also got fixed, though not always retroactively).

This post also doesn't do a great job covering the T&S community's efforts: setting up working groups within the Governing Board, strides in feature development on Draupnir and meowlnir, implementations and experimentation with policy servers, countless hours of free support to impacted communities, and a whole lot more.

Moving forward, we're expecting to be more communicative about what we're working on, which includes commenting on issues like this. We're also about to shift our focus slightly into internal tool maintenance, but also exploration of tools like Coop and Osprey from ROOST (oh, we also became partners with ROOST at some point in the last 6-8 months). We're already in the early experimentation phases of trialing HMA in our stack too, and expect that will be a strong subtheme to our work over the next 3 months.

So: if we take more than 6 months to reply here, please feel free to ping me/us as a reminder of our commitment. We should also be posting about our work with ROOST and HMA on the matrix.org blog, for those who are interested in those tools.

turt2live avatar Aug 01 '25 03:08 turt2live

@turt2live That is very helpful, and I appreciate the response. I am very grateful for the work you all do, and I appreciate that you took the time to address these concerns. I do apologize for not being more upfront about this in my original comment.

I spent some time researching this tonight. I saw that element-web and element synapse both include a partial implementation of MSC4155. If I wanted to contribute some development time towards this issue, what projects should I spend it on? Element clients (parity with element-web for mobile or desktop)?

jamesjulich avatar Aug 01 '25 03:08 jamesjulich

From the Element side, it looks like the Element X clients could do with support in particular. The now-legacy mobile apps aren't likely to get much support, unfortunately.

Otherwise I'd strongly recommend contributing to the client of your choice. Making the feature available to any number of users is more important than learning a new client's language stack (if that's a risk for you).

The feature is already enabled on matrix.org (despite unstable feature policy saying it shouldn't be), so at the very least those users can start using it. I've personally been using it on my own homeserver since about March with incredible success :)

I believe our team is overdue an opinion on MSC4155 in particular, so I'll put that a bit higher on the list of things we need to do.

turt2live avatar Aug 01 '25 04:08 turt2live

Hey 👋 , just wanted to weigh in as one of the developers poking at MSC4155.

I spent some time researching this tonight. I saw that element-web and element synapse both include a partial implementation of MSC4155. If I wanted to contribute some development time towards this issue, what projects should I spend it on? Element clients (parity with element-web for mobile or desktop)?

One of the things that MSC really needs is a user friendly client-side implementation of it's full set of features. We struggled to find a nice way to present the options of allow/block/ignore for invites in Element in a nice clean manner (which is not to say we're not still trying, but more community implementations will certainly help), so anyone who is able to demonstrate that will help the MSC progress from my point of view. It's also roughly still in my brain if you did want to chat about it :)

Half-Shot avatar Aug 01 '25 18:08 Half-Shot

There should be server side support for users disabling username-based invites from accounts not sharing any rooms with them or as a whole. There should be a way to generate and share invite codes enabling people to invite you to rooms despite this restriction. This would enable people to partially or fully disable invites via username while still being able to share an invite code for people to invite them to a direct message or other room. The invite codes should show a usage history and should be possible to revoke. This way, you could post one on your public profile for people to invite you on Matrix but could see if it's getting abused. Invites should show how they were done, whether it was by username or invite code. For username, shared rooms should be shown. For invites via username, the option to disable it for non-shared rooms when receiving an invite that applies to or to disable it a whole should be shown on the invite page. This would allow people to fully opt-out of invite abuse and use dedicated invite codes instead where they control managing abuse through where those are shared and monitoring/revoking them. The invite codes should also have an expiry by default with configuration for that.

Matrix needs to take abuse very seriously and make dealing with it one of the top priorities if it's going to succeed. The majority of the GrapheneOS community migrated from Matrix to Discord after we provided it an option despite all of the inertia from Matrix being the only official option for years. State resets and other technical issues with the protocol/server/client were part of this but the primary reason people left was due to the abuse experienced on Matrix which has been much worse for GrapheneOS than nearly any other communities. We need users to have kind of tools I've described here in order to continue using Matrix or any other similarly decentralized chat platform. Abuse handling should be primarily based around powerful tools for users followed by room admins and then server admins. Trying to deal with pervasive decentralized abuse via centralized server admin tools with nearly all open registration servers barely having any active moderation relative to usage will not work well. The problems need to be solved in a decentralized way by designing things around abuse. Abuse needs to be considered for everything just like security. Not doing this has been one of the major failures of Matrix. It's entirely possible to support invites in a way that users can protect themselves from abuse by locking down or disabling username-based invites. Please consider addressing this in a much more fundamental way by reconsidering how invites work as a whole. This can be done for other things too such as media.

thestinger avatar Aug 12 '25 06:08 thestinger

@thestinger

Trying to deal with pervasive decentralized abuse via centralized server admin tools with nearly all open registration servers barely having any active moderation relative to usage will not work well.

Please explain in total detail about:

  • How it's "deal with pervasive decentralized abuse via centralized server admin tools"?
  • How it's "nearly all open registration servers barely having any active moderation relative to usage"?

ShadowRZ avatar Sep 07 '25 02:09 ShadowRZ