Custom emoji

[jwheare]

Following on from discussion here: https://github.com/ircv3/ircv3-specifications/pull/289#issuecomment-279668885

Services like Slack and Twitch have custom emoji that go beyond the unicode standard. It would be good if IRC messages and reactions could support this too.

For reactions we could retain the plain text +react tag to serve as a fallback or tooltip/alt text value, but send a separate URL in a +react-image tag that clients can optionally display if they want to.

@+reply=123;+react=:lol:;+react-image=https://example.com/custom-emoji/lol.png :[email protected] TAGMSG #channel

---

For custom emoji in messages, we could send text replacement offsets, similar to twitch's custom "emotes" tag (https://dev.twitch.tv/docs/irc/tags/#privmsg-tags)

Using an +emoji client tag with JSON, it might look something like:

@+emoji=[{"url":"https://example.com/custom-emoji/lol.png","idx":[[0,5]]}] :[email protected] PRIVMSG #channel ::lol:

Plain text clients would just show a message :lol:, supporting clients would replace the text starting at index 0 with length 5 with the defined image.

---

I have working implementations of both of these in an unreleased project (suitably prefixed with draft/. Posting this here as a placeholder and for comments before I write up proper specs.

[SadieCat]

Allowing arbitrary urls seems like a privacy risk to me.

[jwheare]

This is no more of a risk for clients that already embed arbitrary image URLs.

[SadieCat]

This is no more of a risk for clients that already embed arbitrary image URLs.

If you don't embed image urls you don't miss anything.

[jwheare]

Thats why there's a text fallback. Clients that don't want to embed could even make the text clickable so users can open the urls if they want.

[jwheare]

I would probably want to enforce https URLs though.

[lopcode]

It would be really nice to have this functionality in an actual spec. 👍

I don't think it's up to the spec to allow and disallow where images are stored, other than setting the expectation of a URL. I can definitely foresee clients allowing disabling / blacklisting / whitelisting though.

Are "non-normative" sections places for recommendations? HTTPS could be a strong recommendation, as well as having something mentioning privacy risks for client implementations.

[lopcode]

Another thought: is it up to the spec to recommand image formats? I suspect PNG / webp are the most appropriate, but again that's likely a client decision (whether to support it or not).

[Zarthus]

Twitch sets custom emoji at server-level, it's less of a privacy infringement there because they always come from one source you trust (as it's the network you connect to)

Are plans for a network-supported whitelist of custom emotes part of your plan, or is this just limited to "let anyone specify anything at all, be it ascii goatse or just a regular picture of a cat"?

[jwheare]

Plan is to support anything, client side only to be as broadly usable as possible. Something like a slack bridge could perform automated conversion of their custom emoji per server, but to the client it would be indistinguishable from a client sent tag.

[lopcode]

Moderation is best left to client and server implementations, as long as we're not painting people in to a corner in the spec somehow IMO.

[Zarthus]

Per IRC, I've suggested a sort of Telegram-like Sticker-set where clients can maintain their own sets.

This would allow for:

• client configurable whitelisting (display all stickers always, click on a sticker you haven't added to display it, display only stickers you have added) (effectively fixing privacy concerns and abuse)
• caching
• easily sharable files that can sync across clients, provided this specification defines a minimal normalized format or a future extension does
• simple right click -> add sticker set to my library possibility

It would also sync nicely with existing implementations of the react specification by allowing for multiple tabs of custom sticker sets, like in this screenshot (Telegram mobile client for Android)

Yes, some of these suggestions are client-side implementation limited, and outside of scope, but I believe the benefits by defining a normalized format would all help create this sort of ecosystem, and it's one that I believe solves a lot of problems I have with the existing base idea.

[DanielOaks]

I can see this idea being useful. However, allowing any URL and practically forcing automatic download+display of them for graphical clients makes me feel iffy about this from a privacy standpoint.

I'd feel more comfortable with something along the lines of the 'set' idea above, making it easier for clients to whitelist a number of known-good sets/packs for automagic download+display while letting the others be optionally clickable.

The set idea still feels a little clunky but better, and I think we could get to a decent middle-ground on privacy vs usability.

[jwheare]

I think we need to be clear about our expectations for images. There are multiple features that involve image URLs and it'd be nice if we don't have to rehash this discussion each time.

Some image-oriented features I have in mind so far are: user avatar metadata, custom emoji/reactions/stickers, message attachments, embedded image URLs, channel logo metadata.

Whether or not a client fetches these images automatically or lets you opt in/out, or lets you use an anonymising proxy, or allows you to white/blacklist them by host, etc etc, is really probably always going to be a decision for that particular client to make. No one is being forced to do anything.

If every one of these features has a spec that requires clients to use some sort of verified network of trust system before ever showing an image I don't really see adoption getting off the ground. We should definitely mention the privacy implications and suggest some mitigation considerations, but I'm really opposed to hampering these features with requirements that put up road blocks for users who are aware of the implications, or trust the environment just getting a nice default experience.

Defining a way to share custom emoji sets between clients would be useful, for allowing users to input them, but I don't think it should be required for clients displaying them. A possible client feature could be only displaying explicitly trusted URLs, sure, but not a mandated one.

[dpyro]

Why not just allow each channel to include a link to an emoji stickersheet image?

[nomis]

Custom emoji could be handled by the server having a URL prefix and then a defined lookup URL that returns all the available image names for the network and for each channel.

e.g. Server prefix is https://emoji.example/

The global emoji list would be available at https://emoji.example/index.xml and per-channel lists are available at https://emoji.example/%23channel.xml. Clients should make full use of HTTP caching headers to avoid placing too much demand on the web server providing this data.

It would then be up to IRC networks to define how these images are registered. Client privacy is maintained because each client is already known to the IRC server. This can be made to work for user and channel avatars too.

For inline text replacement I'd suggest using real emoji or the Unicode private use area.

e.g. @+emoji=\uE000=lol,\uE001=cry,👻=spooky PRIVMSG #channel :example text \uE000! \uE001 quite 👻

(The emoji list would specify 👻 as the fallback character. Clients that cannot render images would be able to render the fallback emoji or \uFFFD.)

[jwheare]

The fallback should always just be text, not invisibles or arbitrary emoji that don't relate to the custom image. So, index positioning is required to avoid replacing the wrong stuff in the wrong place.

[GIJack]

Loading arbitrary URLs is just looking to be exploited and abused. Also adding custom images is also looking to be abused as well.

[TingPing]

Yes this would be a nightmare, clients would always use them insecurely, web apps only have a chance because they blame somebody when something inevitably gets exploited.

[trobotham]

Custom emoji could be handled by the server having a URL prefix and then a defined lookup URL that returns all the available image names for the network and for each channel.

e.g. Server prefix is https://emoji.example/

The global emoji list would be available at https://emoji.example/index.xml and per-channel lists are available at https://emoji.example/%23channel.xml. Clients should make full use of HTTP caching headers to avoid placing too much demand on the web server providing this data.

It would then be up to IRC networks to define how these images are registered. Client privacy is maintained because each client is already known to the IRC server. This can be made to work for user and channel avatars too.

For inline text replacement I'd suggest using real emoji or the Unicode private use area.

e.g. @+emoji=\uE000=lol,\uE001=cry,👻=spooky PRIVMSG #channel :example text \uE000! \uE001 quite 👻

(The emoji list would specify 👻 as the fallback character. Clients that cannot render images would be able to render the fallback emoji or \uFFFD.)

I would recommend using .well-known for the metadata

[Summertime]

horrible idea, but, if external servers are outright no good, then TAGMSGs to facilitate client-to-client emoji sharing?

upside, no privacy risk, no IP leaks, no e2ee leaks (assuming e2ee is used for the client tags too), no gigantic emoji

downside: no even-sane-sized emoji, would require some state-of-the-art encoder/decoders to get those sizes

extra big downside: a message like <person> Hi! :D could inflate to >4KB/person traffic, ~~but all IRC servers are dwindling so luckily it'll never be a problem ha ha~~

---

Alternative, for server-to-client, TAGMSGs would still work, with the same limitations, just with a more centralized repository , emojiserv la same as nickserv or chanserv

[GIJack]

Arbitrary images and URLs are a security risk that transmitting UNICODE and letting the clients render images stored on their computers, hopefully done by trusted system packages is not.

IRCv3 should mostly be adding security mechanisms, authorization and authentication. But keep it simple, is what allowed it to scale to sizes not seen in modern systems(100k concurrent users in the same namespace)