[Feature request]: allow to have a parameter to set a value in max_tokens for some models (claude-3-5-sonnet)

[cristenger]

What do you need?

Maybe there is a way to change this in a proper way but I couldn't find it. In my case I had problems with making claude-3-5-sonnet work correctly, I was able to fix it by increasing the number of max_tokens allowed in utils.py

async def claudeStream(self, system, user):
       from anthropic import AsyncAnthropic.
       self.claudeApiKey = os.environ["CLAUDE_API_KEY"]
       Streamingclient = AsyncAnthropic(api_key=self.claudeApiKey)
       buffer = ""
       async with Streamingclient.messages.stream(
           max_tokens=40096,
           system=system,
           messages=[user],
           model=self.model, temperature=self.args.temp, top_p=self.args.top_p

[patonw]

A "proper" way could involve adding a CLI option around https://github.com/danielmiessler/fabric/blob/dd4b896f4dfa2923272743b7fa4f92a2db4aef86/installer/client/cli/fabric.py#L49-L56

and then referencing that as self.args.max_tokens from https://github.com/danielmiessler/fabric/blob/dd4b896f4dfa2923272743b7fa4f92a2db4aef86/installer/client/cli/utils.py#L103