Can I use a converted tensorflow-lite model with basic-pitch? Do I need to adapt the basic_pitch_predict-function in some way?

[lorenzkort]

[drubinstein]

Hi @lorenzkort . Yes it is possible. I have example code in #100 . I have not been able to merge it yet due to issues with GitHub actions's testing infrastructure when it comes to CoreML models.

[lorenzkort]

Awesome man!! Do you have a compiled linux CentOS binary for tf-lite? Or can you show me how to install it? If I run the setup.py file and install tflite-runtime I get a warning that tflite-runtime is not installed and I can't use pip install since your PR isn't on pypi yet. Here is the error:

>>> import basic_pitch
WARNING:root:Coremltools is not installed. If you plan to use a CoreML Saved Model, reinstall basic-pitch with `pip install 'basic-pitch[coreml]'`
WARNING:root:tflite-runtime is not installed. If you plan to use a TFLite Model, reinstall basic-pitch with `pip install 'basic-pitch tflite-runtime'` or `pip install 'basic-pitch[tf]'
WARNING:root:onnxruntime is not installed. If you plan to use an ONNX Model, reinstall basic-pitch with `pip install 'basic-pitch[onnx]'`
WARNING:root:Tensorflow is not installed. If you plan to use a TF Saved Model, reinstall basic-pitch with `pip install 'basic-pitch[tf]'`
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/var/task/basic-pitch/basic_pitch/__init__.py", line 95, in <module>
    ICASSP_2022_MODEL_PATH = build_icassp_2022_model_path(_default_model_type)
NameError: name '_default_model_type' is not defined

[drubinstein]

You can always pip install tflite-runtime or pip install tensorflow locally. That being said, if you're trying to use the branch , you can always install the directory with pip install /var/task/basic-pitch assuming that's where you placed you clone of basic-pitch.

[lorenzkort]

Thanks for your help! I've ran into an error which occurs when tflite.Interpreter(model_path=ICASSP_2022_MODEL_PATH) is called and looks like it's related to SWIG of your PR-version of basic-pitch. I'm installing tflite-runtime from source because AWS-linux (python3.8) doesn't have GLIBC_2.27 when installed with pip install tflite-runtime Can you help me out?

The Error:

Wrong number or type of arguments for overloaded function 'InterpreterWrapper_CreateWrapperCPPFromFile'.  
Possible C/C++ prototypes are:    
tflite::interpreter_wrapper::InterpreterWrapper::CreateWrapperCPPFromFile(char const *,std::vector< std::string > const &,std::string *)    tflite::interpreter_wrapper::InterpreterWrapper::tflite_interpreter_wrapper_InterpreterWrapper_CreateWrapperCPPFromFile__SWIG_1(char const *,PyObject *)

My Docker file:

FROM public.ecr.aws/lambda/python:3.8

RUN yum update -y && yum groupinstall -y "Development Tools"
RUN yum install -y gcc-c++  development 
RUN yum install -y python3-devel

# ENV
ENV PYTHON_SITE_PACKAGES=/var/lang/lib/python3.8/site-packages
ENV NUMBA_CACHE_DIR=/tmp/

# install Tensorflow Lite Runtime (compiled from source for AWS-linux)
RUN pip3 install numpy wheel
RUN yum install -y git
WORKDIR ${LAMBDA_TASK_ROOT}/tflite
RUN git clone --branch v2.2.0-rc0 https://github.com/tensorflow/tensorflow.git
RUN sh ./tensorflow/tensorflow/lite/tools/pip_package/build_pip_package.sh
RUN pip3 install tensorflow/tensorflow/lite/tools/pip_package/gen/tflite_pip/python3/dist/tflite_runtime-2.2.0rc0-cp38-cp38-linux_x86_64.whl

# Copy and install a PR-version of basic-pitch which supports Tensorflow Lite, for effective model serving:
# https://github.com/spotify/basic-pitch/pull/100
WORKDIR ${LAMBDA_TASK_ROOT}
COPY ./basic-pitch ${LAMBDA_TASK_ROOT}/basic-pitch
RUN pip3 install ./basic-pitch

# Copy over the lambda function
COPY lambda_function.py ${LAMBDA_TASK_ROOT}

# Set the CMD to your handler (could also be done as a parameter override outside of the Dockerfile)
CMD [ "lambda_function.handler" ]

My Python Script (ICASSP_2022_MODEL_PATH points to the tf-lite model):

import os
import tflite_runtime.interpreter as tflite
from basic_pitch.inference import predict as basic_pitch_predict
from basic_pitch import ICASSP_2022_MODEL_PATH

try:
    # Load the TFLite model
    tflite.Interpreter(model_path=ICASSP_2022_MODEL_PATH) 
    interpreter = tflite.Interpreter(model_path=ICASSP_2022_MODEL_PATH)
    interpreter.allocate_tensors()
except Exception as e:
    print( {"statusCode": "Error", "body": str(e)} )

[drubinstein]

You can pip install tflite-runtime from pypi. Looks like youre insalling tensorflow 2.2 when tensorflow is currently at 2.14 and I believe basic pitch requires >2.4.1.

[lorenzkort]

I installed it like you said, but I get the following error: "CreateWrapperFromFile(): incompatible function arguments. The following argument types are supported:\n 1. (arg0: str, arg1: int, arg2: list[str], arg3: bool) -> tflite::interpreter_wrapper::InterpreterWrapper\n 2. (arg0: str, arg1: int, arg2: list[str], arg3: list[Callable[[int], None]], arg4: bool) -> tflite::interpreter_wrapper::InterpreterWrapper\n\nInvoked with: PosixPath('/var/lang/lib/python3.8/site-packages/basic_pitch/saved_models/icassp_2022/nmp'), 1, [], [], False"}

[drubinstein]

This has been merged as a part of 0.3.1.