⚡️ Speed up function `__getattr__` by 37% in PR #3982 (`stas/remove-embedded-mode-from-skyvern`)

[codeflash-ai[bot]]

⚡️ This pull request contains optimizations for PR #3982

If you approve this dependent PR, these changes will be merged into the original PR branch stas/remove-embedded-mode-from-skyvern.

This PR will be automatically closed if the original PR is merged.

---

📄 37% (0.37x) speedup for __getattr__ in skyvern/library/__init__.py

⏱️ Runtime : 2.35 milliseconds → 1.71 milliseconds (best of 119 runs)

📝 Explanation and details

The optimization changes the import strategy from from module import Class to import module as alias followed by attribute access (alias.Class). This seemingly minor change delivers a 37% speedup by leveraging Python's import caching mechanism more efficiently.

Key optimization:

• Original: from skyvern.library.skyvern import Skyvern - performs selective import extraction
• Optimized: import skyvern.library.skyvern as _skyvern + Skyvern = _skyvern.Skyvern - imports module then extracts class

Why this is faster: The import module as alias pattern is more efficient than from module import symbol because:

1. Reduced namespace resolution overhead - Python doesn't need to resolve and extract specific symbols during import
2. Better import cache utilization - module-level imports hit Python's import cache more efficiently
3. Simpler symbol table operations - direct module import followed by attribute access is faster than selective symbol extraction

Performance characteristics from tests:

• First-time access: 36-47% faster (cold cache scenarios)
• Repeated access: Up to 83% faster (hot cache scenarios where globals lookup dominates)
• Mixed workloads: Maintains consistent speedup across valid/invalid attribute patterns
• Error cases: Negligible impact on AttributeError paths

This optimization is particularly effective for lazy loading patterns like __getattr__ implementations, where the import overhead can be significant relative to the simple attribute lookup operation. The consistent speedup across all test cases indicates the optimization doesn't introduce performance regressions in any usage pattern.

✅ Correctness verification report:

Test	Status
⏪ Replay Tests	🔘 None Found
⚙️ Existing Unit Tests	🔘 None Found
🔎 Concolic Coverage Tests	🔘 None Found
🌀 Generated Regression Tests	✅ 2179 Passed
📊 Tests Coverage	100.0%

🌀 Generated Regression Tests and Runtime

from typing import Any

# imports
import pytest
from skyvern.library.__init__ import __getattr__

# unit tests

# --- Basic Test Cases ---

def test_getattr_skyvernclient_returns_class():
    """Test that __getattr__ returns the Skyvern class for 'SkyvernClient'."""
    codeflash_output = __getattr__("SkyvernClient"); result = codeflash_output # 3.22μs -> 2.21μs (45.3% faster)

def test_getattr_skyvernsdk_returns_class():
    """Test that __getattr__ returns the SkyvernSdk class for 'SkyvernSdk'."""
    codeflash_output = __getattr__("SkyvernSdk"); result = codeflash_output # 2.92μs -> 2.09μs (39.3% faster)

def test_getattr_sets_global_for_skyvernclient():
    """Test that __getattr__ sets the global variable for 'SkyvernClient'."""
    # Remove from globals if present
    globals().pop("SkyvernClient", None)
    codeflash_output = __getattr__("SkyvernClient"); result = codeflash_output # 2.62μs -> 1.91μs (37.1% faster)

def test_getattr_sets_global_for_skyvernsdk():
    """Test that __getattr__ sets the global variable for 'SkyvernSdk'."""
    globals().pop("SkyvernSdk", None)
    codeflash_output = __getattr__("SkyvernSdk"); result = codeflash_output # 2.65μs -> 1.94μs (36.6% faster)

# --- Edge Test Cases ---

def test_getattr_invalid_name_raises_attribute_error():
    """Test that __getattr__ raises AttributeError for unknown attribute."""
    with pytest.raises(AttributeError) as excinfo:
        __getattr__("UnknownAttribute") # 1.78μs -> 1.76μs (1.08% faster)

def test_getattr_empty_string_raises_attribute_error():
    """Test that __getattr__ raises AttributeError for empty string."""
    with pytest.raises(AttributeError) as excinfo:
        __getattr__("") # 1.55μs -> 1.56μs (0.640% slower)

def test_getattr_case_sensitivity():
    """Test that __getattr__ is case sensitive."""
    with pytest.raises(AttributeError):
        __getattr__("skyvernclient") # 1.66μs -> 1.62μs (2.46% faster)
    with pytest.raises(AttributeError):
        __getattr__("SKYVERNSDK") # 1.01μs -> 1.03μs (1.94% slower)

def test_getattr_numeric_string_raises_attribute_error():
    """Test that __getattr__ raises AttributeError for numeric string."""
    with pytest.raises(AttributeError):
        __getattr__("12345") # 1.53μs -> 1.47μs (4.07% faster)

def test_getattr_special_characters_raises_attribute_error():
    """Test that __getattr__ raises AttributeError for special characters."""
    with pytest.raises(AttributeError):
        __getattr__("@SkyvernClient!") # 1.57μs -> 1.46μs (7.52% faster)



def test_getattr_many_invalid_names():
    """Test __getattr__ with many invalid attribute names to check performance and error handling."""
    for i in range(100):
        name = f"InvalidAttr{i}"
        with pytest.raises(AttributeError):
            __getattr__(name)

def test_getattr_many_valid_and_invalid_names():
    """Test __getattr__ with a mix of valid and invalid names."""
    valid_names = ["SkyvernClient", "SkyvernSdk"]
    invalid_names = [f"NotValid{i}" for i in range(50)]
    # Valid names should return classes
    for name in valid_names:
        codeflash_output = __getattr__(name); result = codeflash_output # 4.83μs -> 3.28μs (47.4% faster)
    # Invalid names should raise AttributeError
    for name in invalid_names:
        with pytest.raises(AttributeError):
            __getattr__(name)

def test_getattr_global_pollution_with_many_calls():
    """Test that repeated calls do not pollute globals with invalid names."""
    initial_globals = set(globals().keys())
    # Call with many invalid names
    for i in range(100):
        name = f"NotSkyvern{i}"
        try:
            __getattr__(name)
        except AttributeError:
            pass
    # Only valid globals should be added
    allowed = {"SkyvernClient", "SkyvernSdk"}
    new_globals = set(globals().keys()) - initial_globals

def test_getattr_repeated_valid_calls_return_same_class():
    """Test that repeated valid calls return the same class object."""
    codeflash_output = __getattr__("SkyvernClient"); cls1 = codeflash_output # 2.90μs -> 2.12μs (36.3% faster)
    codeflash_output = __getattr__("SkyvernClient"); cls2 = codeflash_output # 1.24μs -> 701ns (77.3% faster)
    codeflash_output = __getattr__("SkyvernSdk"); sdk1 = codeflash_output # 1.53μs -> 1.03μs (48.5% faster)
    codeflash_output = __getattr__("SkyvernSdk"); sdk2 = codeflash_output # 1.07μs -> 591ns (81.2% faster)

def test_getattr_stress_valid_and_invalid_mix():
    """Stress test with a mix of valid and invalid names."""
    valid_names = ["SkyvernClient", "SkyvernSdk"]
    invalid_names = [f"Invalid{i}" for i in range(500)]
    results = []
    for i in range(100):
        name = valid_names[i % 2]
        results.append(__getattr__(name)) # 108μs -> 59.7μs (82.4% faster)
    for name in invalid_names:
        with pytest.raises(AttributeError):
            __getattr__(name)
# codeflash_output is used to check that the output of the original code is the same as that of the optimized code.

import sys
import types
# function to test
from typing import Any

# imports
import pytest
from skyvern.library.__init__ import __getattr__

# -----------------------
# 1. Basic Test Cases
# -----------------------

def test_getattr_returns_skyvernclient_class():
    """Test __getattr__ returns the correct Skyvern class for 'SkyvernClient'."""
    codeflash_output = __getattr__("SkyvernClient"); result = codeflash_output # 3.03μs -> 2.06μs (46.6% faster)

def test_getattr_returns_skyvernsdk_class():
    """Test __getattr__ returns the correct SkyvernSdk class for 'SkyvernSdk'."""
    codeflash_output = __getattr__("SkyvernSdk"); result = codeflash_output # 2.77μs -> 1.97μs (40.6% faster)

def test_getattr_sets_globals_on_first_access():
    """Test that __getattr__ sets the correct global after first access."""
    codeflash_output = __getattr__("SkyvernClient"); result = codeflash_output # 2.69μs -> 1.87μs (43.8% faster)

def test_getattr_returns_same_class_on_second_access():
    """Test that __getattr__ returns the same class (from globals) on repeated access."""
    codeflash_output = __getattr__("SkyvernClient"); first = codeflash_output # 2.58μs -> 1.88μs (37.2% faster)
    # Remove sys.modules to force global fetch
    sys.modules.pop("skyvern.library.skyvern", None)
    codeflash_output = __getattr__("SkyvernClient"); second = codeflash_output # 356μs -> 341μs (4.37% faster)

# -----------------------
# 2. Edge Test Cases
# -----------------------

def test_getattr_raises_attributeerror_for_invalid_name():
    """Test that __getattr__ raises AttributeError for unknown attributes."""
    with pytest.raises(AttributeError) as excinfo:
        __getattr__("NotARealAttribute") # 1.94μs -> 1.88μs (3.24% faster)

def test_getattr_case_sensitivity():
    """Test that __getattr__ is case-sensitive."""
    with pytest.raises(AttributeError):
        __getattr__("skyvernclient") # 1.75μs -> 1.65μs (5.99% faster)
    with pytest.raises(AttributeError):
        __getattr__("SKYVERNCLIENT") # 1.03μs -> 1.03μs (0.000% faster)
    with pytest.raises(AttributeError):
        __getattr__("skyvernsdk") # 932ns -> 902ns (3.33% faster)
    with pytest.raises(AttributeError):
        __getattr__("SKYVERNSDK") # 751ns -> 801ns (6.24% slower)

def test_getattr_with_empty_string():
    """Test that __getattr__ raises AttributeError for empty string."""
    with pytest.raises(AttributeError):
        __getattr__("") # 1.61μs -> 1.54μs (4.54% faster)


def test_getattr_does_not_set_global_on_failure():
    """Test that __getattr__ does not set a global if AttributeError is raised."""
    before = set(globals().keys())
    with pytest.raises(AttributeError):
        __getattr__("NotARealAttribute") # 1.99μs -> 2.08μs (4.32% slower)
    after = set(globals().keys())

def test_getattr_global_is_not_overwritten():
    """Test that __getattr__ does not overwrite an existing global with a different object."""
    class Fake: pass
    globals()["SkyvernClient"] = Fake
    codeflash_output = __getattr__("SkyvernClient"); result = codeflash_output # 3.12μs -> 2.27μs (37.6% faster)

# -----------------------
# 3. Large Scale Test Cases
# -----------------------

def test_getattr_many_repeated_accesses():
    """Test __getattr__ can be called many times in succession without error."""
    for _ in range(500):
        codeflash_output = __getattr__("SkyvernClient"); result = codeflash_output # 527μs -> 286μs (83.9% faster)
        codeflash_output = __getattr__("SkyvernSdk"); result2 = codeflash_output # 517μs -> 286μs (80.8% faster)

def test_getattr_with_many_invalid_names():
    """Test __getattr__ raises AttributeError for many different invalid names."""
    for i in range(100):
        with pytest.raises(AttributeError):
            __getattr__(f"InvalidName_{i}")

def test_getattr_global_leakage_after_many_calls():
    """Test that only the correct globals are set after many calls."""
    # Clear any existing
    globals().pop("SkyvernClient", None)
    globals().pop("SkyvernSdk", None)
    for _ in range(100):
        __getattr__("SkyvernClient") # 108μs -> 59.6μs (81.5% faster)
        __getattr__("SkyvernSdk") # 105μs -> 59.4μs (77.3% faster)

To edit these changes git checkout codeflash/optimize-pr3982-2025-11-12T19.26.05 and push.

---

⚡️ This PR optimizes the __getattr__ function in the Skyvern library's initialization module, achieving a 37% performance improvement by changing the import strategy from from module import Class to import module as alias followed by attribute access. The optimization leverages Python's import caching mechanism more efficiently while maintaining identical functionality for lazy loading of SkyvernClient and SkyvernSdk classes.

🔍 Detailed Analysis

Key Changes

• Import Strategy: Changed from from skyvern.library.skyvern import Skyvern to import skyvern.library.skyvern as _skyvern followed by Skyvern = _skyvern.Skyvern
• Performance Optimization: Applied the same pattern to both SkyvernClient and SkyvernSdk imports in the __getattr__ function
• Code Structure: Maintained the existing lazy loading pattern and globals caching behavior while optimizing the underlying import mechanism

Technical Implementation

flowchart TD
    A[__getattr__ called] --> B{Check attribute name}
    B -->|SkyvernClient| C[Import skyvern.library.skyvern as _skyvern]
    B -->|SkyvernSdk| D[Import skyvern.library.skyvern_sdk as _skyvern_sdk]
    B -->|Other| E[Raise AttributeError]
    C --> F[Extract Skyvern = _skyvern.Skyvern]
    D --> G[Extract SkyvernSdk = _skyvern_sdk.SkyvernSdk]
    F --> H[Cache in globals and return]
    G --> H
    
    style C fill:#e1f5fe
    style D fill:#e1f5fe
    style F fill:#c8e6c9
    style G fill:#c8e6c9

Impact

• Performance Improvement: 37% speedup in function execution time (2.35ms → 1.71ms), with up to 83% improvement in hot cache scenarios
• Import Efficiency: Better utilization of Python's import cache through module-level imports rather than selective symbol extraction
• Backward Compatibility: Zero breaking changes - the API remains identical while the internal implementation is optimized
• Scalability: Consistent performance gains across different usage patterns including first-time access, repeated calls, and mixed workloads

Created with Palmier

---

[!IMPORTANT] Optimizes __getattr__ in skyvern/library/__init__.py for a 37% speedup by changing import strategy, verified with extensive testing.

• Optimization:
  • Optimizes __getattr__ in skyvern/library/__init__.py by changing import strategy from from module import Class to import module as alias followed by attribute access.
  • Results in a 37% speedup by improving import cache utilization and reducing namespace resolution overhead.
• Performance:
  • First-time access is 36-47% faster; repeated access is up to 83% faster.
  • Consistent speedup across valid/invalid attribute patterns; negligible impact on AttributeError paths.
• Testing:
  • Extensive regression tests confirm correctness and performance improvements with 100% test coverage.

^{This description was created by for bd941c36a1ce7908361cea6e1a3d5fb1a213168a. You can customize this summary. It will automatically update as commits are pushed.}

[coderabbitai[bot]]

[!IMPORTANT]

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codeflash-ai[bot]]

This PR has been automatically closed because the original PR #3982 by stanislaw89 was closed.