enh(dspy): Move max_tokens explicitally into `GPT3` constructor

[harrysalmon]

Makes the reason for output truncation clearer as per #730

Also adds a comment to the predict file where the max tokens are modified before the call is made.

[okhat]

This is mainly during retries etc. It's not very straightforward to change.

[okhat]

Basically, the user's max_tokens is used at first, then max_tokens / 2 if we need more fields, but 75 is sort of like a min_max_tokens. Setting a large enough max_tokens could be all that's required here?

[harrysalmon]

Thanks for taking a look @okhat

Yeah the main thing for me is putting it into the init so it's clear to the user that they need to set it. What do you think of this part?

In terms of the retries, do you mean if max_tokens doesn't leave the LLM enough tokens to pass the full input tokens, it'll retry with max_tokens / 2 and then with 75?

[arnavsinghvi11]

@harrysalmon just following up on this. I think we can merge the change but leave the existing retry behavior. lmk if it's ready to merge.

[arnavsinghvi11]

Closing for now to avoid breaking cache errors.