Lyra support

[trengginas]

This PR will add support for lyra codec (https://github.com/google/lyra).

Requirement:

• Lyra codec library (version 1.3), download from here

Notes and limitations:

• The implementation has only been tested using 1.3. Other versions might not work and have compatibility issues.
• Currently there's no standards related to SDP specification for Lyra. The rtpmap SDP attribute will follow the specification defined in rfc8866. It will also specify the bitrate setting as an fmtp param. e.g.:

  a=rtpmap:96 lyra/16000
  a=fmtp:96 bitrate=3200
  

• Currently Lyra only supports Android, Linux, Mac and Windows (no iOS). On windows, the Lyra build only tested for x64 and Release mode only.

Building Lyra:

• Check the requirements for building the lyra codec on their github pages.
• Specific for v1.3, the working (tested) bazel version is 5.0.0 (here)

Build commands for Win, Mac, Linux

bazel build -c opt  :encoder_main
bazel build -c opt  :decoder_main

Build commands for Android

bazel build -c opt android_example:lyra_android_example --config=android_arm64 --copt=-DBENCHMARK

Setup include and lib folder

Bazel will download and build the required third party code used by Lyra. We can use this file libgen.py.zip to generate a single static library and header files required by pjsip. Run the script from the Lyra source folder.

python3 libgen.py

For windows, add --platform=win to the command.

Add Lyra support

For GNU targets (Mac, Linux, Android):

• Run configure or configure-android command and specify the Lyra source folder using --with-lyra option. ./configure --with-lyra=[lyra_src_folder]

  You should see this on the output if everything is in order. checking lyra usability... yes

• continue building the library

Configuring the codec:

• Lyra supports 3200, 6000, 9200 bitrate. You can specify the bitrate from pjmedia_codec_lyra_config::bit_rate/CodecLyraConfig::bit_rate. Note: the remote must have the same bitrate.
• Lyra required some additional (model) files (lyra_config.binarypb, lyragan.tflite, quantizer.tflite, soundstream_encoder.tflite). Fortunately, Lyra provided those files on their source folder (model-coeffs). You can copy those files to a new folder and specify it to pjmedia_codec_lyra_config::model_path/CodecLyraConfig::model_path.
• [For android app]. You can include the model files above as assets to your project and later copy them to the local folder and specify it as CodecLyraConfig::model_path. Check here as reference.

[sauwming]

The version selection is very tricky indeed, can only work for specific Lyra and Bazel version. I tried Lyra 1.3.2 with Bazel 5.3.2 and it didn't work, while 1.3.0 and 5.0.0 was a success.

The option --with-lyra is not properly declared, btw. aconfigure:11875: WARNING: unrecognized options: --with-lyra

The detection also failed:

aconfigure:10151: checking lyra usability
aconfigure:10185: g++ -o conftest -std=c++17 -I/Users/ming/teluu/tools/lyra-1.3.0/include/com_google_absl -I/Users/ming/teluu/tools/lyra-1.3.0/include/gulrak_filesystem -I/Users/ming/teluu/tools/lyra-1.3.0/include/com_google_glog/src -I/Users/ming/teluu/tools/lyra-1.3.0 -g -O2 -DGLOG_DEPRECATED=__attribute__((deprecated)) -DGLOG_EXPORT=__attribute__((visibility("default"))) -DGLOG_NO_EXPORT=__attribute__((visibility("default")))  -L/Users/ming/teluu/tools/lyra-1.3.0/lib    -lupnp -lixml conftest.cpp -llyra -lopus -lssl -lcrypto -lupnp -lixml -lm -lpthread  -framework CoreAudio -framework CoreServices -framework AudioUnit -framework AudioToolbox -framework Foundation -framework AppKit -framework AVFoundation -framework CoreGraphics -framework QuartzCore -framework CoreVideo -framework CoreMedia -framework Metal -framework MetalKit -framework VideoToolbox -L/opt/local/lib -lSDL2  -L/usr/local/lib -lavdevice -lavformat -lavcodec -lswscale -lavutil >&5
In file included from conftest.cpp:84:
/Users/ming/teluu/tools/lyra-1.3.0/lyra_decoder.h:25:10: fatal error: 'absl/types/span.h' file not found
#include "absl/types/span.h"
         ^~~~~~~~~~~~~~~~~~~

[nanangizz]

How about our SDP specs (e.g: codec name, supported parameters (mandatory & optional)), perhaps it is a good idea to describe it here too?

[trengginas]

The version selection is very tricky indeed, can only work for specific Lyra and Bazel version. I tried Lyra 1.3.2 with Bazel 5.3.2 and it didn't work, while 1.3.0 and 5.0.0 was a success.

The option --with-lyra is not properly declared, btw. aconfigure:11875: WARNING: unrecognized options: --with-lyra

The detection also failed:

aconfigure:10151: checking lyra usability
aconfigure:10185: g++ -o conftest -std=c++17 -I/Users/ming/teluu/tools/lyra-1.3.0/include/com_google_absl -I/Users/ming/teluu/tools/lyra-1.3.0/include/gulrak_filesystem -I/Users/ming/teluu/tools/lyra-1.3.0/include/com_google_glog/src -I/Users/ming/teluu/tools/lyra-1.3.0 -g -O2 -DGLOG_DEPRECATED=__attribute__((deprecated)) -DGLOG_EXPORT=__attribute__((visibility("default"))) -DGLOG_NO_EXPORT=__attribute__((visibility("default")))  -L/Users/ming/teluu/tools/lyra-1.3.0/lib    -lupnp -lixml conftest.cpp -llyra -lopus -lssl -lcrypto -lupnp -lixml -lm -lpthread  -framework CoreAudio -framework CoreServices -framework AudioUnit -framework AudioToolbox -framework Foundation -framework AppKit -framework AVFoundation -framework CoreGraphics -framework QuartzCore -framework CoreVideo -framework CoreMedia -framework Metal -framework MetalKit -framework VideoToolbox -L/opt/local/lib -lSDL2  -L/usr/local/lib -lavdevice -lavformat -lavcodec -lswscale -lavutil >&5
In file included from conftest.cpp:84:
/Users/ming/teluu/tools/lyra-1.3.0/lyra_decoder.h:25:10: fatal error: 'absl/types/span.h' file not found
#include "absl/types/span.h"
         ^~~~~~~~~~~~~~~~~~~

Just fixed the --with-lyra option declaration. For the error, have you execute the python script above? It is needed to setup the static library and copy the required header files to the include/lib folder.

[sauwming]

Just fixed the --with-lyra option declaration. For the error, have you execute the python script above? It is needed to setup the static library and copy the required header files to the include/lib folder.

Ah, I thought the python script is under the Build commands for Android section.

Executing the script, the lib generation in lib/liblyra.a succeeded, but the header copying failed: distutils.errors.DistutilsFileError: cannot copy tree '/private/var/tmp/_bazel_ming/54ee470338414c00ed158de552250b16/external/com_google_absl': not a directory

[nanangizz]

Btw, in case you manage to find (and test against) any other SIP phone/B2BUA/server supporting Lyra, I think it would be great to be mentioned here too.

[trengginas]

Just fixed the --with-lyra option declaration. For the error, have you execute the python script above? It is needed to setup the static library and copy the required header files to the include/lib folder.

Ah, I thought the python script is under the Build commands for Android section.

Executing the script, the lib generation in lib/liblyra.a succeeded, but the header copying failed: distutils.errors.DistutilsFileError: cannot copy tree '/private/var/tmp/_bazel_ming/54ee470338414c00ed158de552250b16/external/com_google_absl': not a directory

The bazel build process will download all the required third party on its output_base settings. You can check manually if the folder exists. And if it doesn't exists you can try rebuild the lyra library. I use this command to build the library:

bazel build -c opt :encoder_main
bazel build -c opt :decoder_main

To clean up the build output, you can use:

bazel clean --expunge

I tried rebuilding from a fresh folder of lyra source and the header files are there. Tested here building on windows, mac, linux and android. Not sure if bazel behaves differently on other machines.

[sauwming]

Okay, cleaning and rebuilding Lyra solved the issue.

• A few build warning:

../src/pjmedia-codec/lyra.cpp:252:33: warning: ISO C++11 does not allow conversion from string literal to 'char *' [-Wwritable-strings]
    const pj_str_t lyra_tag = { "lyra", 4 };
                                ^
../src/pjmedia-codec/lyra.cpp:335:47: warning: ISO C++11 does not allow conversion from string literal to 'char *' [-Wwritable-strings]
        codecs[*count].encoding_name = pj_str("lyra");
                                              ^
../src/pjmedia-codec/lyra.cpp:442:55: warning: field precision should have type 'int', but argument has type 'pj_ssize_t' (aka 'long') [-Werror,-Wformat]
    PJ_LOG(4, (THIS_FILE, "Codec opened, model_path=%.*s, chan_cnt=%d, "

• For the model files, there seem to be two options for our sample apps: a. Perform the copying in our Makefile (i.e. copy from with_lyra_path/model_coeffs to pjsip_apps/bin/model_coeffs). b. Alternatively, set PJMEDIA_CODEC_LYRA_DEFAULT_PATH to with_lyra_path/model_coeffs (no copying needed).

Also, perhaps provide more detailed instruction on how to perform the copying and specifying the path for Android?