how can we dynamically change effects in compositionplayer

[pawaom]

how can we dynamically change effects in composition player

suppose we want to change rotation, scale etc on button click, or change different GLshader effects like black and white video, RGB adjustment etc on button click, etc , dynamically, We already have the ability to implement various effects based on duration, but this is a more specific related to preview the changes as we implement them,

[kartik1225]

I am also looking into this feature. Right now when new effect is applied that time I am basically reseting the composition instant which is not ideal.

[pawaom]

It was easier to use with GlSurfaceview with updating onDrawFrame also there were options to GLES20.glBindTexture and modifying the scale, rotation, flipping etc of GlSurfaceview I dont know how Compositionplayer works but it does use GL shaders, so wonder if this is useful.

[claincly]

You can dynamically update the effects in some cases now, but depends on what you want to achieve.

The easiest way is to retain a copy of the shader program (GlShaderProgram) whose parameter you want to change, and just set the parameter on the fly it should be picked up from the next frame.

If you want to change effects, like from contrast to color filter, you could set Composition again, but that's inefficient. My suggestion would be to insert both effects into the pipeline and dynamically tweak its parameters on the fly.

[pawaom]

I came across this

https://github.com/android/socialite/pull/139 which follows some of your suggestion, but as you said it is inefficient,

What I want is to allow users to set the different types of effect which they choose, and allow a preview of these effects, which can be picked from a Glsl file, like this https://github.com/android/socialite/pull/139/commits/d95777952dfb2647ebd7e9a01e798f1dbef81705#diff-bc9e743a978c8e280f601cdf25c31aa60e009ae30066c260eafbe787c1bc5fc0

There two problems with the approach you suggested

1. the effect will be visible only in the next frame, so we have to keep the video playing but people might want to see the effect on the existing frame itself
2. If insert multiple effects into the pipeline , it will be difficult to manage, imagine having 20 odd options for the users and loading all of them, it can be resource expensive specifically on lower end devices,

[claincly]

Your observation is accurate.

1. the effect will be visible only in the next frame, so we have to keep the video playing but people might want to see the effect on the existing frame itself

This is usually fine with playback if that's what you are mainly aiming for. If you want to apply the effects on the frame that is currently on the screen, you can use this method to do so

https://github.com/androidx/media/blob/af2d3748ec61308972111ae410d2030135bc7b98/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionPlayer.java#L444

The caveat is that this would use more memory and introduce extra texture copies even in normal playback.

2. If insert multiple effects into the pipeline , it will be difficult to manage, imagine having 20 odd options for the users and loading all of them, it can be resource expensive specifically on lower end devices

Yes this is true - but if you mainly have matrix-based effects, they are merged into one giant shader so it's cheaper on that end. We are currently working on a fix to address this issue, and after which you'll be able to just load the effects that you have changed

[pawaom]

Thanks for the quick reply.

We are currently working on a fix to address this issue, and after which you'll be able to just load the effects that you have changed

This will be great, can we know the timeline when this feature will be available, There is another problem for which I had raised an issue https://github.com/androidx/media/issues/2439

If these two are resolved most of the requirements of compositionplayer will be fulfilled

[droid-girl]

For the timeline, the feature is in progress, however we do not know when it will be fully ready. Please keep an eye on the release notes and upcoming commit messages. Are you only using stable releases or you can depend on the main branch?

[pawaom]

In production I am using , stable Release , for testing I try Main branch some times

[pawaom]

I tried the Effects Demo, it seems this function works with Exoplayer in this

[pawaom]

https://github.com/androidx/media/commit/176e67a0a38b7994ac49a3f45cb7929c52830c1a seems it has been solved