Working with matrices to accumulate transformations

[raffaeler]

Question

I have to draw a custom map supporting translations, rotations and zooming. The transforms of every gesture is always additive in respect of the previous gestures. From a math perspective this simply means multiplying the new transform and proceed to the next.

The transform style in React Native apparently does not support matrices (and this is very surprising to me). How can I work with matrices in react-native-svg in order to accumulate the transformations?

Thank you!

[msand]

Perhaps this can help? https://twitter.com/wcandillon/status/1244988861635268608 https://www.npmjs.com/package/zoomable-svg

[msand]

And yeah, you can use matrices with transforms in react-native-svg, and matrix multiplication follows the normal linear algebra, so just compute what ever product of matrices you want, and give it as a transform to e.g. a G element.

[raffaeler]

I already watched the video last week and also tried to use your zoomable-svg but the problem is that I need to accumulate all the transformations: scaling, rotation and translation. The video shows that the transform is reset at the end of the gesture: definitely too easy because it never accumulates matrices.

How can I specify a matrix?

• transformMatrix has been deprecated according to the docs
• the transform: {[ matrix: [...] is apparently not working

[msand]

Same way as in any other svg https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/transform#Matrix

  <rect x="10" y="10" width="30" height="20" fill="red"
        transform="matrix(3 1 -1 3 30 40)" />

I don't see the problem, you know how to multiply two matrices no? https://en.wikipedia.org/wiki/Matrix_multiplication#Definition

[raffaeler]

Of course I know. I did not find the transform="matrix(...)" syntax in the docs. Is this supported for react-native-svg only or is it standard for all the react-native based libraries?

I would prefer to apply the transforms to the element whose child is the SVG. Is this a potential performance hit?

[msand]

This only works on react-native-svg, as it comes from the svg spec. Would have to do profiling to check performance. Quite likely, natively animated/computed logic would be more performant than a plain js version.

[msand]

and style={{transform: [{matrix: something}]}} should certainly work in both react-native and react-native-svg

[raffaeler]

Thanks. I am probably going to use animation only to smooth the movements, but I only need to apply the transforms to change the map view.

and style={{transform: [{matrix: something}]}} should certainly work in both react-native and react-native-svg

I was not successful in using it on a simple View element filled with a color. Will retry ...

Thank you!

[msand]

There's also this happy chap i tried to help once, might be useful for you as well: https://github.com/react-native-community/react-native-svg/issues/1064

[msand]

But, when I think about it, seems you don't need to accumulate any state at all, you just need to keep track of where the pointers were when the number of active gestures change, if it changes from 0 or 2 to one pointer, keep track of where it was when that happened, and on new gesture events, the distance of the pointer to that point is your translate transform. If it changes from 0 or 1 to 2, store the position of the two pointers, on new events, the change in distance between the two new positions and two initial ones, is your scale transform, the change in angle between them is your rotation, and the distance between the midpoints of the two pairs is your translation. Gestures / states in between should not have any effect / non-linear accumulation affecting the outcome, transform only depends on current pointer data and when the number of active pointers changed.

[raffaeler]

hmm, let's make an example:

• you pinch the map, and it zooms at a specific point (the center of the pinch)
• than you rotate it, with a different pinch point
• now you pan it If you now want to zoom or rotate, you need to accumulate those transforms over the previous ones. The transforms style (without the matrix) just apply a fixed sequence of translation, rotation, scale. Either you keep the matrix and then try to extract the translation, rotation, scale from that matrix, or you will never be able to keep the gestures correctly. Since I don't skew, the matrix should be reversable, but never did it as keeping the matrix state is far simpler. Do you agree or am I losing something? I always worked this way in other technologies and it worked.

[msand]

Ah, yes, when the gesture ends you should accumulate the transform for sure, was just thinking about individual gestures on top of the current state. So the initial transform is the identity, lets call it A, and then you multiply a transform that has the translate, scale, rotate and, offset needed for the just finished gesture, B = TSRO (translate, scale, rotate, offset), and make C = BA the new accumulated transform

[raffaeler]

Exactly. The video you linked is about Instagram where the photo goes back to the original position after the gesture... that's too easy and I already can do it. When you go to google maps instead, you accumulate the transformations, so a state is needed (starting, as you said in your last message from the identity matrix) by multiplying every transition (I typically order scale, then rotation and lastly translation). BTW every time you pinch, you either have a primitive centering the rotation/zoom, otherwise you have to manually translate + rotate/zoom + translate back.

FYI I tried using reanimate and discovered (sadly after two days) they do not support matrices, so I can't work with their library.

[msand]

Alternatively you can have four different transforms, one for each primitive, and accumulate those, will need to consider interactions between translations and scale-rotate in the gestures a bit more carefully then.

[raffaeler]

Since I don't need to skew, they should be reversable. In this case I could maintain a matrix and then "extract" the separate t,r,s,o parameters. Never did that but in theory it should be possible.

[msand]

But yeah, to clarify the native aspect, if you add animations, it'll probably just feel more disconnected from the gesture, you want to minimize the number of cycles from the gesture event being registered, to the final rendered output being visible on the screen. So you probably want to use react-native-gesture-handler, as that allows the processing to stay completely native, rather than doing a context switch to javascript, run event handler, change state, run react lifecycle, commit changes back to native and only then start rendering, rather than just computing the matrix transform and invalidating one View or Svg / G element... If it doesn't have the matrix support you seek, I recommend forking it and implementing the support yourself and making a pull request there. I can probably help on the way in case there's any questions on the native side.

[msand]

This one use-case would probably deserve its own tailor-made performance / use-case optimized package, something like react-native-pan-zoom-rotate / react-native-zoomable perhaps, pull requests to zoomable-svg for a native mode would be welcome as well.

[msand]

And yeah, if you have the transform as one matrix, you don't need to split it up, just start with the initial A and B matrix as identity, when the number of active pointers change, store the positions, for each event update B (or update the decomposed matrices / primitive transformations and multiply them together to get B), and when the number of active pointers change, accumulate B into A, and set B to identity again

[msand]

So the structure would be something like this:

<Svg style={{transform: [{matrix: B}, {matrix: A}]}}>
  <Text>some content goes here</Text>
</Svg>

Or, equivalently

<Svg>
  <G style={{transform: [{matrix: B}]}}>
    <G style={{transform: [{matrix: A}]}}>
      <Text>some content goes here</Text>
    </G>
  </G>
</Svg>

[msand]

I assume you're familiar with these, but just in case you want a refresher (they're also in the css / svg specs), here's the equivalent matrices to the primitive transforms: https://github.com/react-native-community/react-native-svg/blob/ffa2e69c17ce02b21f393a5b57cdbef1c039fe3d/src/lib/extract/transform.peg#L1-L105

And the main state changes that would need to be implemented in native logic instead: https://github.com/msand/zoomable-svg/blob/fe724c2652595bb6176731be96fde1151e30f21a/index.js#L420-L511

[msand]

Also, the calculation for the rotation is missing there, so something like this:

const initialAngle = Math.atan2(initial_y1 - initial_y2, initial_x1 - initial_x2);
const rotate = Math.atan2(y1 - y2, x1 - x2) - initialAngle;

[raffaeler]

But yeah, to clarify the native aspect, if you add animations, it'll probably just feel more disconnected from the gesture, you want to minimize the number of cycles from the gesture event being registered

My wish is to use animation only to restore a position after the user makes a search. Probably it doesn't make sense (in my scenario) to use animation while the user is actively making a gesture.

I understood how the react-native-gesture-handler library works and it is a great idea (creating the AST for the desired transformations and generate the native code that uses the refs under the hood), but I don't know if I will have time to implement the fundamental matrix support.

[raffaeler]

perhaps, pull requests to zoomable-svg for a native mode would be welcome as well.

As soon as I come to a solution, I will be more than glad to either publish it or making a pull-request

[msand]

Also, maybe these two can be useful for learning more about react-native transforms: https://snack.expo.io/@msand/new-instagram-stories https://snack.expo.io/@msand/rotate-cube

The instagram stories one has three alternative implementations, Stories2 requires a fork of react-native-reanimated I made https://github.com/software-mansion/react-native-reanimated/pull/538

[msand]

Also, Stories1 in new-instagram-stories has a source code parameter / constant called "alt" with a bit different transformation

[raffaeler]

Talking about the transform using matrix over standard react-native elements and the Svg:

• style={{transformMatrix: [1, 0, 50, 0, 1, 50]}} This doesn't work
• style={{ transform: [ { matrix: [1, 0, 50, 0, 1, 50] } ] }} on a react-native element throws with "Error updating property 'transform' of a view managed by: RCTView"
• style={{ transform: [ { matrix: [1, 0, 50, 0, 1, 50] } ] }} on Svg throws with "Invariant Violation: Matrix transform must have a length of 9 (2d) or 16 (3d)..."
• style={{ transform: [ { matrix: [1, 0, 50, 0, 1, 40, 0, 0, 1] } ] }} on a reac-native element or also Svg throws with "Error updating property 'transform' of a view managed by: RCTView"

I didn't find even a single example showing how to use the matrix on react-native ... astonishing.

Of course I am able to use the SVG notation on inner elements of the Svg. The following works:

<Rect x="0" y="0" width="100" height="100" fill="red"  transform="matrix(1 0 0 1 50 50)" />

What is the syntax for matrix? Do you know any example of that?

P.S. I am still reading/working on the other posts you wrote.

Thank you

[msand]

The error comes from here: https://github.com/facebook/react-native/blob/0b9ea60b4fee8cacc36e7160e31b91fc114dbc0d/Libraries/StyleSheet/processTransform.js#L172-L182

There's some useful helpers in that file as well: https://github.com/facebook/react-native/blob/0b9ea60b4fee8cacc36e7160e31b91fc114dbc0d/Libraries/StyleSheet/processTransform.js#L19-L114

So, to use the normal react-native transform style property (with a list of transforms containing matrices), you need to give the matrices as arrays with either 9 (2d) or 16 (3d) numbers, i.e.

  style={{
    transform: [
      { translateX: tx },
      { translateY: ty },
      { scale: s },
      { rotate: r },
      { translateX: ox },
      { translateY: oy },
      { matrix: [ // 9 numbers doesn't seem to work
          1, 0, 0,
          0, 1, 0,
          0, 0, 1
        ]
      },
      { matrix: [ // seems to work
          1, 0, 0, 0,
          0, 1, 0, 0,
          0, 0, 1, 0,
          0, 0, 0, 1
        ]
      }
    ]
  }}

. For the svg standard syntax, you need to give a transform attribute as string instead (n.b. not a style property, but directly on the element instead, although we support it in the style props as well for simplicity, it's not required by the spec), i.e. transform="matrix(a b c d e f)" i.e. 6 numbers inside the parenthesis, or any sequence of svg transform primitives as a string.

Another supported syntax in react-native-svg elements is: giving an array of 6 numbers i.e. transform={[a, c, e, b, d, f]} as the transform attribute / style property, the same as the output of the svg transform string parser referred to in an earlier comment, instead of an array of react-native transform objects.

[raffaeler]

The only transform/matrix I didn't test was the 16 elements ... and of course it worked. But (as I posted before) the one with 9 elements does not work and this is what cheated me. Thank you

[msand]

Oh, that's quite possible, not sure why, would have to set breakpoint in both the javascript, java and objective-c code of react-native and react-native-svg to double check. Might be that only the 16 works properly with the react-native syntax, to fit together with the other 3d transforms.

[msand]

With the tailor-made module, I'm thinking it wouldn't depend on anything but react-native (at most react-native-svg as well, i.e. no reanimated, no react-native-gesture-handler), and would be a single View, accepting only a single child, and its only concern would be to handle any pan-zoom-rotate gestures on that child in native code, calculate the needed matrix, and set the new transform either on itself or on the child, using the ViewManagers directly: https://github.com/facebook/react-native/blob/d0871d0a9a373e1d3ac35da46c85c0d0e793116d/React/Views/RCTViewManager.m#L169-L174

https://github.com/facebook/react-native/blob/f2d58483c2aec689d7065eb68766a5aec7c96e97/ReactAndroid/src/main/java/com/facebook/react/uimanager/BaseViewManager.java#L76-L84

https://github.com/react-native-community/react-native-svg/blob/ffa2e69c17ce02b21f393a5b57cdbef1c039fe3d/ios/ViewManagers/RNSVGNodeManager.m#L157-L164

https://github.com/react-native-community/react-native-svg/blob/ffa2e69c17ce02b21f393a5b57cdbef1c039fe3d/android/src/main/java/com/horcrux/svg/RenderableViewManager.java#L1211-L1225

And would be a simple wrapper:

import * as React from 'react';
import { View, Text } from 'react-native';
import PZR from 'react-native-pan-zoom-rotate';

export default () => (
  <PZR>
    <View>
      <Text>Gesture This</Text>
    </View>
  </PZR>
);