Can I add Skia support to it?

[wcandillon]

I'm the maintainer of React Native Skia. Like three-fiber, this library uses it's own React reconciler for 2d graphics. The library works on Web as well with react (without the need to have react native web as a dependency). I would love to extend triplex to support the Skia reconciler. Let me know if that's something doable and if you could give me a sense of the amount of work it represents?

[itsdouges]

Hey mate thanks for raising this — can you give me more information on what it looks like to write a component with RN Skia? Examples would be great! Are there any differences between running it on Web vs. RN?

At a high level if it's its own reconciler, has its own distinct JSX host elements, and can run in a web environment then getting it running inside Triplex is possible.

Here's an example of how React Three Fiber / React DOM work today:

• the scene component is loaded here https://github.com/trytriplex/triplex/blob/main/packages/renderer/src/features/scene-loader/index.tsx#L45
• there is a code transformer that adds metadata to all components based on found JSX / hooks (video: https://www.youtube.com/watch?v=nkq-gNMSDLs)
• depending on that metadata the root scene is different https://github.com/trytriplex/triplex/blob/main/packages/renderer/src/features/scene-loader/index.tsx#L105C14-L105C19

All JSX elements can have their props adjusted by input controls, some JSX elements can have their props adjusted by visual controls.

Is this a side project for you or is this something Shopify is interested in pursuing?

[wcandillon]

I can confirm that RN Skia fits the requirements:

• Works on Web (we use CanvasKit, a webassembly file that can be sometimes tricky to load but something insurmountable)
• Has its own reconciller with host elements exactly like React Three Fiber
• No runtime differences to run on Web

Here an example of a Simple Skia demo with Remotion: https://github.com/wcandillon/react-native-skia/blob/main/apps/remotion/src/Playground/Breathe.tsx (we want to use this with Remotion which makes things simpler, no animation library needed and so on, it's all "clean" properties).

Thank you for asking about the use-case, this would be mainly/exclusively to be used for Remotion: https://www.remotion.dev/ I produce videos using React Native Skia and Remotion. This is a brief example a Skia & Remotion video: https://www.dropbox.com/scl/fi/c77c4ri73kbosrili5yrv/test.mp4?rlkey=jdc2t0ublhn0027zfe8v22ymr&dl=0 Here, if I could have chosed the parameters of my scene using a Visual editor (finding the correct camera position and so on), I could have saved a lot of time.

Let me know if you think that makes sense and how the path for it could look like.

[itsdouges]

Sweet!

Truth be told you could set it up now and see how it feels and if we need to do something more ingrained we can. As Remotion is always a React DOM root you could, assuming everything would be React Skia for a given project:

1. Create a config https://triplex.dev/docs/api-reference/config-options
2. Add Remotion + RN Skia providers using the global provider component https://triplex.dev/docs/building-your-scene/provider
3. Open a component, see what happens

If you have a project that could open different components from different reconcilers, with the above set up you can create .triplex configs where needed to change the global provider picked up for said component, so imagine a folder structure:

• /remotion
  • /.triplex/config.json (sets up provider for remotion)
  • component.tsx
• /skia
  • /.triplex/config.json (sets up provider for skia)
  • component.tsx
• /components
  • /.triplex/config.json (sets up provider for regular react-dom components)
  • component.tsx

Etc etc. Does that make sense?

[itsdouges]

@wcandillon hey mate did this help?

[wcandillon]

How would you recommend that I approach it? Start with getting triplex to work with a regular react code base? Then have some sort of development setup to continue? I am not seeing a path yet.

[itsdouges]

Do you have any example repository with Remotion + Skia? I'd be happy to wire things up to demonstrate :-)

[wcandillon]

yes this would be the main example: https://github.com/Shopify/react-native-skia/tree/main/apps/remotion It's part of the the monorepo (however for remotion just running yarn would be needed) but let me know if you would need to a standalone remotion project.

[itsdouges]

I reckon let's start with a standalone remotion + skia project and we can go from there

[wcandillon]

here it is https://github.com/wcandillon/remotion-skia-example

[itsdouges]

Is there specific bundler configuration needed to get skia working?

E.g. one error I have:

SyntaxError: The requested module '/node_modules/canvaskit-wasm/bin/full/canvaskit.js?v=862d4b20' does not provide an export named 'default'

🤔

[wcandillon]

yes: https://github.com/wcandillon/remotion-skia-example/blob/main/remotion.config.ts#L19 But also you can use this configuration that will not (should not) require any specific bundle configuration:

import { LoadSkiaWeb } from "@shopify/react-native-skia/lib/module/web";
import { version } from 'canvaskit-wasm/package.json';

LoadSkiaWeb({
  locateFile: (file) => `https://cdn.jsdelivr.net/npm/canvaskit-wasm@${version}/bin/full/${file}`
}).then(async () => {
  const App = (await import("./src/App")).default;
  AppRegistry.registerComponent("Example", () => App);
});

The heart of the matter here is to have the .wasm file available

[itsdouges]

Oh great, will take it further tonight! Thanks mate.

[wcandillon]

Thanks a lot, here's an example of a video entirely done in Remotion/Skia: https://www.youtube.com/watch?v=wUsFNlas620

[itsdouges]

Cool video! 😁 saw simondev in there! Hehe

[itsdouges]

Have you gotten react native skia running with Vite? Triplex uses it. I saw canvaskit example which helped get a bit further: https://github.com/simoncollins/skia-canvaskit-vite

Unfortunately now I'm getting stuck with react native module being hit and everything blowing up. If we can solve this without dipping into bundler config that would be amazing as Triplex doesn't currently expose any.

[wcandillon]

@itsdouges I see. We're entering tricky territory here. A big thank you for making this preliminary work (I hadn't thought of that at all). I think it would be interesting to run remotion and Skia without the React Native dependency. But does that mean that remotion needs to run with vite as well? Sorry I'm a bit lost here.

[itsdouges]

It really depends on what you're looking for. Are you looking to run the entire remotion pipeline inside Triplex? If so, yes that probably won't work without some substantial effort on either side depending on what remotion does. I'm not familiar with remotion so would need some insight. Do folks generally run remotion using the remotion CLI? Has anyone ever ran it inside Storybook? That's basically the same thing as running it in Triplex.

If you're looking to run some isolated components inside Triplex it's easier. The main thing is just getting it to run first! Have you ever loaded up a react skia project with Vite?

[itsdouges]

I have a pull request to open up custom vite configs that will unblock some other folks as well. We can use it here to stub out the react native dependencies.

Will ping this issue when it's released!

[itsdouges]

https://triplex.dev/docs/api-reference/config-options/unsafe-vite-config would be able to unblock this, probably.

[itsdouges]

I'm going to close this issue now. With custom vite configs you can change a lot of the internals where needed to run skia, if you'd be interested in exploring this further please jump on Discord and I can help you out!

https://discord.gg/nBzRBUEs4b