Perform Inference using Python instead of command line

[vashiegaran]

Hi, is there any example of python code that can run inference instead of using the command line? If so can you kindly share

[research4pan]

Thanks for your interest in LMFlow! You may check speculative_inference.py, where you can replace SpeculativeInferencer with Inferencer (along with their arguments) in examples/chatbot.py. Please feel free to contact us again if there is any problem during the usage. Thanks very much 😄