[prefect-shell] Replace prefect with prefect-client in the dependencies and avoid prerelease version installs

[tyong920]

Describe the current behavior

1. Unnecessary dependency on prefect:

• prefect-shell currently depends on prefect>=3.0.0rc1, which brings in the full prefect package.
• For projects that only need to interact with the Prefect API and do not require task flow definitions or runtime, the prefect-client library would be a sufficient and lighter dependency.

2. Automatic installation of prerelease versions:

• The dependency specification prefect>=3.0.0rc1 allows the installation of prerelease versions, such as recent .dev versions.
• This behavior leads to frequent automatic updates to potentially unstable versions, which may introduce unexpected changes or issues.

Describe the proposed behavior

1. Use prefect-client instead of prefect:

• Replace the prefect dependency in prefect-shell with prefect-client to reduce dependency weight and avoid unnecessary imports.
• This would allow prefect-shell to remain focused on API interactions and avoid coupling to the task execution and orchestration features provided by prefect.

2. Avoid prerelease version installations:

• Adjust the dependency specification to prevent prerelease versions from being installed unless explicitly intended, such as by setting prefect>=3.0.0.
• This ensures stability by preventing .dev or rc versions from being automatically pulled in during dependency resolution.

Example Use

No response

Additional context

This change will make prefect-shell easier to integrate in projects where a lighter, API-only client is preferred and prevent potentially unstable releases from being automatically installed.

[malhotra5]

I could take a shot at this! I'm thinking of implementing Solution C but with the same schema as Openhands proper. It include:

LLM_NUM_RETRIES (Default of 8) LLM_RETRY_MIN_WAIT (Default of 15 seconds) LLM_RETRY_MAX_WAIT (Default of 120 seconds) LLM_RETRY_MULTIPLIER (Default of 2)

This would help with consistency in retry methods.

[enyst]

Please see also: https://github.com/All-Hands-AI/OpenHands/issues/5087

[enyst]

Just curious here, sorry if I'm missing something obvious, why are we using litellm.completion(), could we use our llm.completion() instead? It was intended to be compatible with litellm.completion() and it has a retry mechanism.

        response = litellm.completion(
            model=llm_config.model,
            messages=[{'role': 'user', 'content': prompt}],
            api_key=llm_config.api_key,
            base_url=llm_config.base_url,
        )

e.g.

        @self.retry_decorator(
            num_retries=self.config.num_retries,
            retry_exceptions=LLM_RETRY_EXCEPTIONS,
            retry_min_wait=self.config.retry_min_wait,
            retry_max_wait=self.config.retry_max_wait,
            retry_multiplier=self.config.retry_multiplier,
        )
        def wrapper(*args, **kwargs):
            """Wrapper for the litellm completion function. Logs the input and output of the completion function."""
...
@property
    def completion(self):
        """Decorator for the litellm completion function.

        Check the complete documentation at https://litellm.vercel.app/docs/completion
        """
        return self._completion

[malhotra5]

Ah yeah I noticed this too @enyst! I've implemented your suggestions in #5187; it was approved but I didn't have push access to merge it at the time 😅

I'll try to get it in soon

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[enyst]

I think this has been addressed by reusing the llm.completion method, which obeys the user configuration (like min_retry).

The rest of the problem here is tracked in other issues, for example on implementing an automated routing mechanism or other features that would improve the behavior with rate limits. (example)

I'll close this, but please feel free to reopen if you see fit.