Make Lorax Easier for New Users

[AdithyanI]

Feature request

Description

I want to make it easier for new people to use Lorax, especially those coming from other tools. Right now, they have to set max-input-length and max-tokens when they start using Lorax, which can be hard for them.

Problem

New users or people switching from other tools find it tough to set up Lorax because of these settings:

• max-input-length: This has to be set when you start Lorax.
• max-tokens: This is needed for each API request.

They want to start using Lorax quickly without worrying about these settings.

What I'm Asking

Make it easier for people to move to Lorax.

After spinning up a server. They should be able to just change the URL they use and start working. Set default values for max-input-length and max-tokens so new users don't have to. There are different ways to handle this and have them sane defaults.

Making these changes will help more people use Lorax easily. I'm looking forward to what Travis thinks about this. And I will let the creators of this awesome software decide what is the best way forward.

Thanks for thinking about my idea.

Motivation

See above.

Your contribution

Happy to brainstorm and contribute to the discussion. And definitely willing to test it.

[bi1101]

if you're following the recent update, there's support for OpenAI conpatible endpoints, which should be pretty easy to transition from existing setup