feat: add async prompt fetching API with shared caching and tests

[jitokim]

[!IMPORTANT] Introduces async prompt fetching API with shared caching and tests in client.py and test_prompt.py.

• Async API:
  • Adds aget_prompt() in client.py for async prompt fetching with caching.
  • Implements _fetch_prompt_and_update_cache_async() for async server fetches with retries.
• Fallback Handling:
  • Refactors fallback prompt creation into _create_fallback_prompt_client().
• Tests:
  • Adds test_async_get_fresh_prompt, test_async_get_prompt_uses_cache_without_fetch, and test_async_get_prompt_returns_fallback_on_failure in test_prompt.py to validate async fetching and caching behavior.

^{This description was created by for e0b162630f2ee71f6e4fe544dcf252db208500ce. You can customize this summary. It will automatically update as commits are pushed.}

Disclaimer: Experimental PR review

Greptile Overview

Updated On: 2025-10-31 00:29:58 UTC

Greptile Summary

Adds async prompt fetching API (aget_prompt) that mirrors the synchronous get_prompt method with identical caching semantics.

Key Changes:

• Implemented aget_prompt() async method with same signature and behavior as sync version
• Added _fetch_prompt_and_update_cache_async() to handle async API calls with retry logic
• Refactored fallback prompt creation into shared _create_fallback_prompt_client() helper to eliminate code duplication
• Both sync and async methods share the same PromptCache instance for consistency
• Async background refresh correctly uses run_async_safely() to handle event loop conflicts
• Added comprehensive test coverage for async functionality including cache hits/misses and fallback scenarios

Implementation Quality:

• Clean code deduplication with the new helper method
• Proper async/await patterns throughout
• Correct use of AsyncMock in tests
• Imports follow project conventions (top-level placement)
• Maintains backward compatibility

Confidence Score: 5/5

• This PR is safe to merge with high confidence
• The implementation is well-structured with proper async patterns, comprehensive test coverage, and no breaking changes. Code refactoring reduces duplication while maintaining identical semantics between sync and async variants. All tests follow best practices with appropriate mocking.
• No files require special attention

Important Files Changed

File Analysis

Filename	Score	Overview
langfuse/_client/client.py	5/5	Added aget_prompt async method and refactored fallback creation into _create_fallback_prompt_client helper. Implementation correctly shares cache with sync method and properly handles async refresh logic using run_async_safely.
tests/test_prompt.py	5/5	Added comprehensive async tests for aget_prompt covering cache hits, cache misses, and fallback scenarios. Tests properly use AsyncMock and asyncio.run. Imports correctly placed at module top.

Sequence Diagram

sequenceDiagram
    participant User
    participant Langfuse
    participant PromptCache
    participant AsyncAPI
    participant TaskManager

    User->>Langfuse: aget_prompt(name)
    Langfuse->>PromptCache: get(cache_key)
    
    alt Cache Miss
        PromptCache-->>Langfuse: None
        Langfuse->>AsyncAPI: prompts.get(name)
        AsyncAPI-->>Langfuse: prompt_response
        Langfuse->>PromptCache: set(cache_key, prompt)
        Langfuse-->>User: PromptClient
    else Cache Hit (Valid)
        PromptCache-->>Langfuse: cached_prompt
        Langfuse-->>User: cached_prompt.value
    else Cache Hit (Expired)
        PromptCache-->>Langfuse: expired_prompt
        Langfuse->>TaskManager: add_refresh_prompt_task()
        Note over TaskManager: Background refresh in thread
        TaskManager->>AsyncAPI: prompts.get(name)
        AsyncAPI-->>TaskManager: updated_prompt
        TaskManager->>PromptCache: set(cache_key, updated_prompt)
        Langfuse-->>User: expired_prompt.value (stale)
    end