element-meta
element-meta copied to clipboard
element-hq
Add support for a GIF keyboard / ability to easily send GIFs
(So I lied. But this one is really my last comment for the night. I promise. Honestly.) Would this be covered by sticker support, if the stickers could be GIFs?
I think this is covered by the sticker api at least. Probably can close this one.
...except it isn't, as the sticker API isn't a GIF keyboard. I can't search for "arrested development" and get a selection of memes to pick from.
As i get friends onto riot mostly the first thing is "how do i send and search gifs"
Is there an opposition to looking at adding something like giphy directly into the UI?
Having this same conversation with someone right now. Their words not mine:
Also, it won't be mainstream until it has gifs gifs are everything
🤔
My boss comes in my office once a week and wants to have an update on the "gif-thing-status in the new chat software"
This really feels like something that should be in Element, I've put a bounty on it:
https://www.bountysource.com/issues/28576328-we-need-a-gif-keyboard-toooooo-11-1
This really feels like something that should be in Element, I've put a bounty on it:
https://www.bountysource.com/issues/28576328-we-need-a-gif-keyboard-toooooo-11-1
Added to this, no chat app can persist in this world of memes and custom emojis without the functionality to share them easily
The bounty is now at US$125, after an additional contribution by myself and itsthebatman
We've reached US$200 which was the goal set by @denim2x so hopefully we get some action soon!
Discord uses giphy, while i think this is a bit weird to have jammed into a privacy-oriented and generic messenger, i think at least giving the option of sending GIFs straight from there would be useful. (Discord has a /giphy command that pops up suggested GIFs (which you can click on), and picks the first one when pressing enter)
Jonathan de Jong @.***> writes:
Discord uses giphy, while i think this is a bit weird to have jammed into a privacy-oriented and generic messenger,
So does the Signal Android app, but IIRC they proxy the requests so it's not possible to correlate them to specific clients. I guess element could do that too (letting the user set the proxy server)...
I guess element could do that too (letting the user set the proxy server)...
I guess it could but feels like a bit of an overkill for GIFs...
Maybe allow a direct call to giphy servers for now, but work towards a "define your own proxy" setting, and maybe go towards a (reduced) set of domains you can proxy HTTP requests with from the synapse server.
Though that latter action is maybe a bit risky, seeing that it effectively can transform a matrix server into a HTTP proxy, which isn't the goal and can be a security vulnerability.
Also, if I'm not mistaken, gifs that I recieve via a bridge (e.g. from signal, discord, etc) are rendered as a link instead of playing in-place.
Can't I give more than 1 thumb up to this issue? If I create a github pseudo-users botnet and fill it up of thumbs up would this get prioritized?
I feel like I'm dumb without gifs...
@Yajo I added to the bounty to try and get someone to work on it, but seems like it's been stalled out pending review by @matrix-org/design and @matrix-org/product (see: https://github.com/matrix-org/matrix-react-sdk/pull/5814)
I just want to give an extra data-point to this; Meta (Facebook) was ready to acquire Giphy, but was recently blocked by antitrust regulators, so I don't think relying on giphy at all for GIFs is a good idea, I recommend looking into either other open-source GIF hosting and indexing websites, or to take a similar approach to this as with stickers and emojis; put them in rooms.
Would not be better to use Tenor GIF instead of Giphy ? In my duplicate post, i said Google Chat is using GIPHY but this is wrong, it use Tenor GIF as keyboard gif.
@n4rkip0d im not sure swapping Facebook for Google in this situation will make much of a difference.
That said, I think that, with this other site, a degree of choice could be made on the user side, in which the user can explicitly opt-in to these websites for the GIF selection UX.
However, I'd also like to see element provide a native solution to this, where users can collect or share their GIFs amongst eachother, adding tags or categories.
I personally do not think there should be a default gif search included in Matrix (or Element for that matter) that ties you to a specific GIF API or service. Instead, I think this is a good candidate for a widget (or widgets) so that each user can decide whether they want to opt in.
I have been recently migrating myself and my friends over to Matrix as the new spot for our group chats. While not all of them are technical, most can appreciate the principle of self-hosting an e2e chat server and having more ownership of their data. What they DID NOT appreciate is the lack of a proper GIF keyboard.
As a result, I have been working on the following gif keyboard for Matrix and testing it amongst my friends: torinreilly/matrix-gif-keyboard.
It works by overriding the sticker-picker widget and sending the animated gifs as stickers. You can host the server yourself and it can pull gifs from Tenor, GIPHY, or from a custom list of sites using DuckDuckGo (not sure if this is worth it as discussed here).
Here's the catch. The way stickers are implemented in the Element clients means the sticker URL has to be an MXC URI. In other Matrix clients like FluffyChat, stickers can have remote URLs (unfortunately FluffyChat does not support widgets though :/). So prior to sending the GIF it must be uploaded to a Matrix server. I am accomplishing by providing the GIF keyboard app server a Matrix access token to perform the uploads. This is fast enough to be fairly imperceptible by the end user, but it also exposes an endpoint that can be used to upload just about any image to your Matrix content repo! Not good! The only saving grace is the rate limiting that is supported by the content repository /upload endpoint.
Outside of requiring a login to the widget, I can't think of a good way to ensure this endpoint isn't abused. Currently, there only a few values that can be passed to the widget including username and the matrix server url, but nothing that can be used to ensure the user is authenticated. Passing things like the username and matrix server url could be a bit of security through obscurity, but it is easily faked so not worth implementing IMO. Looking for suggestions here.
If we could get the Element clients to support stickers with remote URLs it would remove the need for and upload in the first place, while saving us disk space and removing the auth headaches. It is also possible that we could get this to work using other room events, but I ran into a lot of trouble trying to send most type of events from a widget (m.room.message including images, etc). If anyone has experience/advice along those lines I would love to pick your brain.
In the meantime, please check it out if you are interested! I personally would not implement this in any kind of production for the time being. If you would like to try it out and don't want to host it yourself, I could let you use one of my hosted GIF keyboard widgets but only if you promise to not abuse my server too much!
Here are the key issues I'd love some feedback on: A better way to authenticate uploads to the content repository Does not work as a custom widget/integration Investigate if maintaining the DuckDuckGo API backend is worth it
Here is an example of how it looks on Element on iOS:
(Note, I'm not a maintainer, or even contributor for that matter.)
@torinreilly While I can appreciate your comment and approach to this, I think it's disregarding user-experience, no?
There are users who don't even understand what a file system is, or think their documents were deleted because it's no longer in their "Recent" in Microsoft Office anymore.
A lot of my friends are non-technical, they don't want to tinker with things or "learn the software". If it's not immediately obvious, or not intuitive, then it might as well not be a feature at all to them. With GIF keyboards kind of becoming a "standard" feature, I don't think it's ideal to make extra steps to access them.
I think it'd be reasonable to implement a GIF server natively, it's up to the user if they choose to use it or not. An option to turn off the UI for it would be great, or even better, a way to provide custom sources. I don't think the default experience should be through a widget though, which most people new to Element won't even understand immediately.
That being said, your project and widget look awesome. I especially love that you've considered solutions beyond Giphy and Tenor. A nice source to also consider could be Creative Commons Search?
@SethFalco Hi, Seth. Thanks for the thoughtful response. I think you are right on a lot of points here and a flexible native solution COULD be ideal provided it allowed enough choice to the end user. The thing I find nice about widgets is you are not beholden to the design decisions of anyone! If someone doesn't like how my widget is implemented or which services it targets they could use a different one. Or even fork it themselves and make it available to whoever they like.
Just to clarify, my intent is not for everyone to set up their own GIF keyboard server. In my case, I have a single app server that uses a bot account's token to perform the uploads to the content repository, but the message sent is actually on the requesting users behalf. Provided someone has setup a server, each user only needs to complete the one time setup of adding the widget in place of the sticker-picker described here: https://github.com/torinreilly/matrix-gif-keyboard#3-enable-widget.There is also the ability to add it through the UI as a "Custom Integration", which only requires the URL the app is hosted at (still working out some issues with this route for my app, it displays fine but messages are not sent).
I agree that extra step is inconvenient and might be confusing to some users, but I think most would not balk at it provided there was a sufficient guide.
I will definitely look at adding more search options. The DuckDuckGo API does allow for filtering results by license, so I was planning to implement that BUT I think Creative Commons Search would probably provide better results in the case of more open licenses! Thank you!
I still can't believe we don't have a solution for this issue. Must I continue to manually download and upload gif files to truly express myself in element? Will someone please think of the memes!
I've just made the move from Discord/Telegram for our org over to Matrix/Element and while doing customisations we just realised using gifs 'natively' is not available.
This has created a lot of frustration for our users as one would expect this day and age and are looking for a solution.
The Giphy bot solution is unintuitive as users simply don't know what to search for to get an exact match of what they wish to express. Moreover, they often don't remember the command to trigger the search.
Naturally, we would want to see a gif icon next to the emoji and attachment icons in the UI which triggers a selector and search field.
I like the @torinreilly solution as it deals with the encrypted room issue that the giphy bot does not. So just on first glance I would be inclined to have this baked into the UX as mentioned.
This is a critical feature that cant be missing in todays world of chat/messaging apps. I will be looking at the code to see how I can contribute.
@torinreilly were there any security issues or like thoughts you had in implementing this solution?
@arkitoure There are definitely some security issues in my current solution. The way widgets are currently handled, there is no way to verify that the user of a widget is logged in to a specific Matrix server. In reality, there are only a handful of properties that are accessible to widgets (room name, username, etc) but any one of those could easily be faked in an http request so they really only provide security through obscurity. Any additional params you wish to include must be passed as part of the URL used when setting up the widget.
In my current implementation, the widget app server has the ability to upload files directly to the Matrix server for which it is configured. This essentially gives anyone with access to the widget server URL cart-blanche to upload files to the Matrix server (not good!).
This may be solved in a couple of ways:
First, I am hoping that the often discussed support for using a Matrix server as an OpenID source comes to fruition. If this happens we could easily verify the user via OpenID.
The second idea I had was to create a token or tokens that could be supplied as part of the widget URL during setup (ie https://widgeturl.com/?token=hdwry823hwekhr). This token could then be checked against a list of valid tokens by widget app server. This is better, but still relies on trusting users to not widely distribute tokens (might work OK for small communities or groups of friends) but for any large scale implementation the overhead of monitoring and revoking tokens used for abuse might not be worth it. (Additionally, I do not love the idea of passing the token as a URL param, but this is currently the only viable way to pass data to widgets as far as I know.)
The third option is to remove automatic upload to the server and require the user to copy and paste the GIF from the keyboard. I find this option to be less than intuitive and does not always work as expected, especially on mobile. It does however remove almost all of the risk of abuse by third parties.
The latter two solutions are less than ideal, but I could certainly add support for both of these with relative ease if there was demand.
There are early designs for this up at https://www.figma.com/file/ifIvZA4XIP1JTyndHFr5Ub/Community-PRs?node-id=21%3A15691 however the technical undertaking of the proposed changes is a bit on the difficult side.
Specifically, it relies on stickers being an API surface we can call due to them being a widget - this isn't impossible, and is in fact planned under my widgets work umbrella, but not something we have today. The designs also appear to rely on power levels for this stuff, which is a bit difficult to enforce at the moment, but not impossible with the current structure and future capabilities like extensible events.
For a code review perspective: I don't think we can reasonably accept PRs on this until product and design have taken a much closer look, possibly with an angle of custom stickers/emoji too, as otherwise the code is likely to bitrot or change dramatically.
Why is this a server api at all? Just make a client-only keyboard that pastes a gif into the chat. I want to like matrix but y'all have a damn-near infinite feature matrix that makes progress nigh impossible.
For simple support, sure, we could do it entirely client side. For future options of potentially paid sticker packs and similar though, there needs to be a server :)
For simple support, sure, we could do it entirely client side.
Then do that? Don't let perfect be the enemy of good.
For future options of potentially paid sticker packs and similar though, there needs to be a server :)
Dear god, stop.