ggml ggml : new operations supported in `encodec.cpp`

Hi!

I just released encodec.cpp which features a handful of new operations, like transposed 1d convolution, argmax and activation functions like elu and tanh.

Is it something you'd like implemented in ggml?

Jun 24 '23 10:06 PABannier

Wow - great work!

argmax, elu and tanh seem like good additions. Wondering if transposed_conv_1d could be avoided - either become part of conv_1d (see #285) or do the transposition in user-code and then ggml_cont().

In any case, I will add reference to your project in the README as it is a very important step towards supporting Bark TTS. Would be great if you add a few basic steps to test out the implementation - I see you have a Quick start section that will probably be used exactly for that

Jun 25 '23 10:06 ggerganov

relevant: I'm working on the 1d/2d cleanup from the interface's (ops) perspective #313

Jun 27 '23 07:06 iboB

@iboB I've implemented a common interface for 1d conv with even and odd kernels in encodec.cpp. If it's of interest for you, you can find it here: https://github.com/PABannier/encodec.cpp/blob/main/ggml.c#L6550 .

Jun 27 '23 16:06 PABannier

@ggerganov I'm not sure I get how to implement conv_transpose_1d in user code using only conv_1d. In any case, it might be worth benchmarking your solution and the implementation of conv_transpose_1d as implemented in encodec.cpp since it is used during forward passes by encodec.

For tanh, elu and argmax, I just opened a PR.

Jun 27 '23 16:06 PABannier