wordpress-activitypub icon indicating copy to clipboard operation
wordpress-activitypub copied to clipboard
verified publisher icon Automattic

Emoji: First pass at support in Interactions

Open obenland opened this issue 1 year ago • 15 comments

Fixes #970.

See https://mastodon.social/@obenland/113788023027390776 See https://obietester.blog/2025/01/06/99/#comment-31

Proposed changes:

  • Adds callback function that replaces custom emoji strings with their image representations when available.
  • Adds unit test to cover the new function.
  • Runs author name and comment content through the new callback on insert and update.

Other information:

  • [x] Have you written new tests for your changes, if applicable?

Testing instructions:

  • Go to Mastodon, for example, and reply to a federated test post with a custom emoji. :yikes:, :AngeryCat:, etc.
  • Go to the comment list in wp-admin and make sure the emoji render as images.

obenland avatar Jan 07 '25 17:01 obenland

I think before allowing images in comments, we have to think about/implement a proper blocking and moderation tooling (even more if it is about side-loading images): https://www.theverge.com/2023/7/24/23806093/mastodon-csam-study-decentralized-network

pfefferle avatar Jan 10 '25 10:01 pfefferle

I just pushed an update that moves the replacement into a filter that runs after comment_content and the author have been sanitized. That way, only custom emoji images will be added to the content.

obenland avatar Jan 10 '25 19:01 obenland

Heads up: This PR adds img tags to the list of allowed HTML tags for Interactions!

Would sideloading the image onto the site with media_sideload_image be helpful here?

  • It would ensure the images remain available in the future.
  • It would avoid image hotlinking, which can be frowned upon.
  • It would add some validation to the image data before to simply display it on the site.

Of course, we'd need to be okay adding potentially a lot of new media to a site once that's enabled. I'm not sure that's okay.

jeherve avatar Jan 13 '25 11:01 jeherve

It would avoid image hotlinking, which can be frowned upon.

With custom emoji being shared tags, is that not kind of expected?

obenland avatar Jan 13 '25 17:01 obenland

With custom emoji being shared tags, is that not kind of expected?

I'm honestly not sure what the common practice is with other Fediverse software. From what I can tell, on Mastodon and on GoToSocial the images from remote instances seem to be cached on the local server:

GoToSocial:

image

Mastodon

image

jeherve avatar Jan 14 '25 14:01 jeherve

Introduction

To successfully introduce custom emoji functionality in WordPress, it is necessary to first standardize the way standard emojis are rendered across the WordPress ecosystem. Additionally, by analyzing the implementations of other Fediverse instances like Mastodon and Misskey, we can identify their limitations and strengths, allowing for a more strategic and robust approach to custom emoji support in WordPress.

1. The Need for Standardization in WordPress Emoji Rendering

Currently, the WordPress ecosystem features at least four different methods for rendering standard emojis, which creates fragmentation that must be addressed before introducing custom emojis.

  • Problem: Fragmented Rendering Methods

    • Mastodon/Misskey: Render emojis using SVG assets hosted on their own servers.
    • WordPress (Default): Uses the operating system’s system font.
    • WordPress (Fallback): If the system font does not support certain emojis (e.g., flag emojis on Windows), the rendering method varies by environment.
      • WordPress.com Blogs: Use Twemoji SVGs from s0.wp.com.
      • Self-hosted WordPress: Use core emoji SVGs from s.w.org.
      • WordPress Reader (for self-hosted posts): Emojis are displayed as PNG images proxied through i0.wp.com.

  • Improvement Direction

    • Before introducing a custom emoji system, WordPress.com, self-hosted WordPress, and the WordPress Reader should all render standard emojis in a consistent manner. This requires a unified approach to emoji rendering across the ecosystem.

2. Analysis of Other Instances and WordPress Improvement Directions

Limitations found in other platforms provide actionable improvement goals for WordPress, while their strengths offer models to emulate.

Mastodon’s Limitations: Inconsistent CDN Paths

  • Problem: Mastodon uses different CDN paths for custom emojis displayed in user profiles versus posts. Even when viewing emojis from external instances, the cached paths vary, leading to inconsistency.
  • WordPress Improvement Direction:
    • Learn from Mastodon’s example: WordPress should provide custom emojis through a consistent path or proxy system, regardless of where they are displayed (profiles, posts, comments).

Misskey’s Strengths and Limitations

  • Strengths to Emulate:
    • Consistent Proxy Usage: Misskey uses proxies like proxy.misskeyusercontent.jp to serve custom emojis, ensuring consistent paths across all contexts (profiles, notes, etc.).
    • Rich Metadata: Misskey supports extensive metadata for custom emojis (name, tags, category, NSFW status, local-only flag, license, etc.), enabling advanced management.
  • Limitations to Address:
    • Reaction Aggregation: Even if the same custom emoji is used for reactions on different instances, Misskey (and Mastodon) cannot aggregate reactions across instances due to different original paths.
    • WordPress Improvement Direction:
      • Move beyond simply comparing image paths. Design a mechanism to identify and aggregate reactions for identical custom emojis across instances, possibly by using original source URLs or unique identifiers for each emoji[8].

Key Considerations for Custom Emoji Implementation in WordPress

1️⃣ Unified Emoji Rendering

WordPress currently relies on system fonts for emoji display, but for unsupported emojis (like flags on Windows), it falls back to Twemoji SVGs from s.w.org or s0.wp.com. The fact that WordPress.com and self-hosted sites use different paths leads to compatibility issues, especially with the WordPress Reader.

Solution:

  • Set up a dedicated CDN path for custom emojis (e.g., custom-emoji.my-site.com/emoji/).
  • Integrate Twemoji and custom emojis into a unified custom renderer (e.g., using Twemoji.parse() in JavaScript).
  • Offer an option to disable system font fallback and enforce SVG/PNG rendering.

2️⃣ Custom Emoji Registration and Management System

Mastodon and Misskey allow admins to add custom emojis via their admin panels, managed as JSON databases. WordPress should implement similar functionality via a plugin.

Essential Features:

  • Name (:emoji_name:)
  • Tags and Categories
  • NSFW Flag
  • CDN Path
  • License Information
  • Local/Global Option (whether to include emojis in Fediverse transmission)

3️⃣ Handling Custom Emojis in ActivityPub Integration

ActivityPub protocol can include emoji metadata. Mastodon and Misskey use the customEmoji field to declare each emoji’s path.

Problem:

  • Emoji reactions from different instances are not aggregated because the same emoji name can have different paths on each instance.

Improvement:

  • Introduce a custom emoji federation registry (e.g., fediemoji.org).
  • Identify emojis by unique ID or hash value, with CDN path as optional.

4️⃣ Reader and External Embed Support

For custom emojis to display correctly in the WordPress Reader or via oEmbed, CDN CORS must be enabled and the public path must be fixed.

Mastodon / Misskey Limitations and Improvement Possibilities

Issue Current Approach Improvement Direction
Emoji reaction aggregation fails Same emoji name, different instance path Common registry; identify by hash or GUID
Note vs. profile CDN path mismatch Mastodon: different cache paths Unify CDN cache paths or normalize paths
Lack of metadata (Mastodon) Only name is registered Add metadata like Misskey (tags, license, NSFW, etc.)
Embed/proxy path confusion Misskey: proxy.misskeyusercontent.jp Proxy maintains original path; verify with CDN hash

WordPress Implementation Strategy (Proposal)

📦 Custom Emoji Plugin Design

  • Twemoji-based custom parser
  • Emoji metadata JSON management
  • Register custom-emoji CDN path
  • Integrate with ActivityPub plugin to inject customEmoji field
  • Optional server-side reaction integration

🌐 Custom Emoji Federation Registry API Design

  • Provide name, URL, hash, license, and category
  • Allow instances to sync this list to their local DB
  • Include original registry ID in ActivityPub transmissions

Conclusion

Custom emoji implementation is more than just image uploads.

  • Emoji identification (name, hash)
  • Unified CDN path
  • Standardization for Fediverse transmission

These must be addressed in parallel for seamless federation. Mastodon and Misskey have encountered issues due to the lack of a unified approach. WordPress could take a leading role by launching a custom emoji registry API to solve these problems proactively[8].

Summary Table: Emoji Rendering and Custom Emoji Management

Platform Emoji Rendering Method Custom Emoji Management Key Issues/Strengths
WordPress System font, Twemoji SVG, PNG proxy Limited, no standard plugin Fragmented, Reader compatibility
Mastodon Server-hosted SVG, PNG Admin panel, JSON DB Inconsistent CDN paths
Misskey Proxy, server-hosted SVG/PNG Admin panel, rich metadata Consistent proxy, rich metadata

By standardizing emoji rendering, introducing a robust custom emoji management system, and addressing federation challenges, WordPress can provide a seamless and expressive custom emoji experience for users, admins, and the wider Fediverse.

refer 제가 간략하게 알아본 바에 따르면, 마스토돈과 미스키의 경우 각 인스턴스에서 이모티콘을 렌더링합니다. 🥳 :partying_face: https://mastodon.social/emoji/1f973.svg https://uri.life/emoji/1f973.svg https://misskey.io/twemoji/1f973.svg 워드프레스의 경우 시스템 폰트를 사용합니다.

문제는 시스템폰트가 없을 때 워드프레스가 이모지를 렌더링하는 방식입니다. 윈도우는 국기 이모지를 지원하지 않습니다. 🇰🇷 https://mastodon.social/emoji/1f1f0-1f1f7.svg https://misskey.io/twemoji/1f1f0-1f1f7.svg

워드프레스닷컴의 블로그의 경우 https://s0.wp.com/wp-content/mu-plugins/wpcom-smileys/twemoji/2/svg/1f1f0-1f1f7.svg 워드프레스 리더에서 확인하면 🇰🇷 셀프호스팅 워드프레스의 경우 https://s.w.org/images/core/emoji/15.1.0/svg/1f1f0-1f1f7.svg 워드프레스 리더에서 확인하면 https://i0.wp.com/s.w.org/images/core/emoji/15.1.0/72x72/1f1f0-1f1f7.png?ssl=1

그러니까 워드프레스 생태계에서 이모지 렌더링 방식이 4가지나 되므로, 커스텀이모지까지 고려하게 되면 상당히 복잡해집니다. 어느정도 교통정리를 해아합니다.

이제 커스텀이모지로 들어가봅시다. mastodon.social의 :mastodon: 이모지 https://files.mastodon.social/custom_emojis/images/001/099/845/original/629dc18288be6387.png

uri.life에서 mastodon.social의 :mastodon: 이모지 확인 https://uri.life/@[email protected]/114651985535580673 https://life.uricdn.one/cache/custom_emojis/images/000/000/952/original/4e6b8b95bf17eee8.png

https://uri.life/@thaumiel999 (마스토돈 플랫폼)에서 :mastodon: 게시 https://life.uricdn.one/custom_emojis/images/000/001/376/original/d9a3907f06edd0cc.png mastodon.social에서 확인 https://mastodon.social/@[email protected]/114651968583821676 https://files.mastodon.social/cache/custom_emojis/images/000/010/391/original/ee5885e3bd7ffad8.png https://uri.life/@[email protected] 프로필의 :mastodon: 아이콘 확인 https://life.uricdn.one/cache/custom_emojis/images/000/000/952/original/4e6b8b95bf17eee8.png

마스토돈에서는 프로필과 노트의 커스텀 이모지 cdn이 다르다는 것을 확인할 수 있음. 개선필요

미스키에서 mastodon.social의 :mastodon: 이모지 확인 노트/프로필 https://proxy.misskeyusercontent.jp/image/files.mastodon.social%2Fcustom_emojis%2Fimages%2F001%2F099%2F845%2Foriginal%2F629dc18288be6387.png?emoji=1 인스턴스 파비콘 미리보기 https://proxy.misskeyusercontent.jp/preview/mastodon.social%2Fpacks%2Fassets%2Ffavicon-48x48-DMnduFKh.png?fallback=1&preview=1

misskey.io의 :misskey: 커스텀 이모지 프로필과 노트에서 일관된 경로 https://proxy.misskeyusercontent.jp/image/media.misskeyusercontent.jp%2Femoji%2Fmisskey.png?emoji=1 마스토돈에서 미스키 이모지 확인 프로필과 노트 모두에서 일관된 경로 https://files.mastodon.social/cache/custom_emojis/images/000/273/113/original/b29393e466430baf.png

https://daepi.so/proxy/image.webp?url=https%3A%2F%2Fmedia.misskeyusercontent.com%2Femoji%2Fmisskey.png&emoji=1

미스키는 이모지 정보에 이름, 태그, 카테고리, 열람주의, 로컬에만, 라이선스 등 옵션이 있음. https://misskey.io/notes/a8sajiih9ywy0851 https://proxy.misskeyusercontent.jp/image/cdn.daepi.so%2Fimages%2F35e15e31-241d-4ef8-8ae4-87173daffef9?emoji=1 https://proxy.misskeyusercontent.jp/image/media.misskeyusercontent.jp%2Femoji%2Fmisskey.png?emoji=1 특이한점으로는 같은 커스텀이모지 :misskey: 인데 인스턴스가 다르면 이모지 리액션이 합산이 안됨. 오리지널 경로가 아닌 다른 인스턴스 이모지 리액션이 안보임. 개선 필요.

Jiwoon-Kim avatar Jun 09 '25 07:06 Jiwoon-Kim

@Jiwoon-Kim I appreciate your input and feedback, but comments and issues of this length are not helpful. They generally lack a specific ask or suggestion and are incredibly hard to read. Going forward, please keep comments/issues concise and actionable, like I asked for previously.

obenland avatar Jun 09 '25 19:06 obenland

@obenland Thank you for your feedback, and I really appreciate the attention.

As someone coming from a non-developer background, I tend to approach these topics from a more experiential and design-oriented perspective. That often leads me to start from broader conceptual ideas before narrowing down to specifics—especially when things are interconnected or have potential design implications that aren't immediately obvious.

I understand that this approach can result in comments that feel messy or overwhelming, and I apologize if it made the discussion harder to follow. I added the reference materials at the end to reduce the need for re-investigation, but I realize now that might have come across as overly verbose.

Also, since this pull request already exists, I thought discussing the context and UX considerations here would help avoid scattering the conversation into too many isolated issues. But I see now that clarity and conciseness in individual issues is important, so I’ll aim to keep future contributions more structured and actionable.

If you ever have time, I’d be grateful for any advice on how to break down larger conceptual suggestions into well-scoped issues. I’m eager to learn and collaborate more effectively with the team.

Thanks again!

Also, just to explain a bit about my workflow — I usually start with an idea in Korean, then use GPT to help flesh it out and translate it into English. Sometimes I’ll use Gemini or Perplexity to refine the wording too. So I’m not actually writing or editing in English directly most of the time.

If this were a Korean-language project, breaking things down into smaller, well-scoped issues like you suggested would honestly be much easier for me. But when working in English, it takes quite a lot more energy for me to carefully read through, translate, and restructure everything. That’s why I sometimes rely on machine translation to double-check and just post the draft as it is.

Hope you can understand that part of my process.

Anyway — I actually ran your feedback and my original comment through Gemini to auto-organize it a bit, and I’ll drop that summary in a follow-up comment below.

Jiwoon-Kim avatar Jun 10 '25 09:06 Jiwoon-Kim

I do understand, maybe part of your workflow could be to instruct your LLM to use natural language and distill its response to a sentence or two?

obenland avatar Jun 10 '25 13:06 obenland

@obenland Thank you for your understanding—I really appreciate it.

I agree that breaking down larger ideas into focused issues is a good approach, and I’ll try to normalize my prompts for LLMs to help with this in the future. However, it can be challenging to distill complex or interconnected ideas into just a sentence or two, especially when I’m trying to preserve the overall context and implications.

Here’s a bit more about my current workflow:
When I have an idea, I usually start by brainstorming in Korean and use GPT to help structure and clarify my thoughts. GPT is great at reorganizing fragmented concepts into coherent language and suggesting implementation directions. If the initial explanation is lacking, I add more context to make sure the AI understands the full picture. This process often involves several iterations and a fair amount of trial and error.

Once I’m satisfied with the structure, I use PowerToys’ Advanced Paste feature to organize everything into a markdown text file. I then use AI translation to convert it into English. Even just getting to this stage takes a lot of time and mental effort before I’m ready to post an issue on GitHub.

There’s a reason I sometimes prefer to bundle ideas together:
Gemini, for example, works best with very specific, focused requests. It excels at quickly categorizing and summarizing concise, scannable content where the desired outcome is clear. However, if I split every idea into separate issues, the overall context and the “big picture” of my suggestion can get lost. Additionally, there are limits to how many requests I can make per day, so I try to use them carefully.

I’ll definitely take your advice and experiment with using LLMs to summarize and distill my suggestions more effectively. Thanks again for your patience and feedback—I’m eager to keep improving my workflow and collaborating better with the team!

Below is the process and set of prompts I plan to use moving forward. I’m sharing them here as they may be useful for others as well.

1. Breaking Down a Long Proposal into Focused GitHub Issues

Prompt:

Act as a senior software project maintainer. Your task is to review the following long-form proposal and break it down into a list of smaller, focused, and actionable GitHub issues.

For each proposed issue, provide a concise title and a one-sentence summary describing its core request. Ensure that each issue is independent enough to be discussed and implemented separately.

Here is the proposal: [Paste the entire long-form proposal here, either in Korean or as a first draft organized by AI.]

2. Crafting a Concise and Actionable GitHub Issue

Prompt:

Using the context provided below, generate a concise and actionable GitHub issue in English. Structure the output in a standard markdown format with the following sections:

  • **### Is your feature request related to a problem? Please describe(Problem statement)
  • **### Describe the solutionou'd like (Desired solution)
  • **### Describe alternatives you've consideredConsidered alternatives)

The tone should be professional, clear, and suitable for an open-source project.

Here is the context for the issue titled "[Title generated in Step 1]": [Paste only the content relevant to this issue here.]

3. Summarizing an Issue or Comment for Quick Triage

Prompt:

Distill the following GitHub issue/comment into its most essential, scannable points. The output should be a short bulleted list or 1-2 concise sentences that clearly state the core problem and the proposed action. The goal is to make it easy for a maintainer to triage the issue quickly.

Here is the text to distill: [Paste the issue content or long comment generated in Step 2 here.]

Feel free to adapt or use these prompts as needed!

Jiwoon-Kim avatar Jun 10 '25 14:06 Jiwoon-Kim

https://developer.wordpress.org/news/2025/08/registering-custom-social-icons-in-wordpress-6-9/

  • https://www.w3.org/wiki/ActivityPub/Primer/Hashtags
  • https://socialhub.activitypub.rocks/t/fep-9098-custom-emojis/5408/6
  • https://codeberg.org/fediverse/fep/src/branch/main/fep/9098/fep-9098.md
  • https://misskey-hub.net/ns/

Jiwoon-Kim avatar Aug 24 '25 09:08 Jiwoon-Kim

  • https://misskey.io/about#emojis
  • https://misskey.io/twemoji/1f973.svg
  • https://mastodon.social/emoji/1f1f0-1f1f7.svg
  • https://misskey-hub.net/en/docs/for-users/resources/glossary/#custom-emoji
  • https://misskey-hub.net/en/docs/for-developers/api/endpoints/
  • https://misskey.io/api-doc
  • https://misskey.io/api-doc#operation/admin___emoji___import-zip
  • https://misskey-hub.net/en/docs/for-developers/api/permission/

Jiwoon-Kim avatar Sep 02 '25 13:09 Jiwoon-Kim

  • https://codeberg.org/fediverse/fep/src/branch/main/fep/c0e0/fep-c0e0.md
  • https://docs.akkoma.dev/stable/development/ap_extensions/#emoji-reactions

Jiwoon-Kim avatar Sep 30 '25 08:09 Jiwoon-Kim

Todo:

  • Store emoji files for comment authors on comment save.
  • Update replacement with using attachment

obenland avatar Oct 02 '25 20:10 obenland

It’s possible to display custom emojis from remote instances by extending the Smiley feature. See: https://wordpress.org/documentation/article/what-are-smilies/#where-are-my-smiley-images-kept

https://github.com/Automattic/wordpress-activitypub/discussions/2371

Jiwoon-Kim avatar Oct 25 '25 12:10 Jiwoon-Kim