asyncio-gevent icon indicating copy to clipboard operation
asyncio-gevent copied to clipboard

feat: add autorestore_context option and make it True by default

Open spumer opened this issue 8 months ago • 4 comments

The main reason for this behavior is we make usage of async_to_sync and sync_to_async is transparent. Programmers can use it without any unexpected side effects.


So, i can wait for your PR https://github.com/gfmio/asyncio-gevent/pull/19 and adapt my changes.

We use your lib on production since 2023 and i wrote article about that: https://habr.com/ru/companies/tochka/articles/798577/

We run sync handlers in fastapi through asyncio-gevent, and this save us! Thank you for this project!

Summary by Sourcery

Add context preservation functionality to async and sync function conversions in asyncio-gevent

New Features:

  • Introduce an autorestore_context option that preserves context variables when converting between sync and async functions
  • Add support for passing and restoring context variables across greenlet and future conversions

Enhancements:

  • Modify async_to_sync, sync_to_async, and greenlet_to_future functions to support context variable preservation
  • Improve context handling to make async and sync function conversions more transparent

Tests:

  • Add test cases to verify context preservation for both sync_to_async and async_to_sync function conversions

spumer avatar Apr 06 '25 12:04 spumer

Seems you are using me but didn't get OPENAI_API_KEY seted in Variables/Secrets for this repo. you could follow readme for more information

cr-gpt[bot] avatar Apr 06 '25 12:04 cr-gpt[bot]

Reviewer's Guide by Sourcery

This pull request introduces an autorestore_context option to greenlet_to_future, async_to_sync, and sync_to_async functions. This option, enabled by default, ensures that context variables are properly passed and restored when switching between asyncio and gevent, providing more transparent usage of async_to_sync and sync_to_async.

Sequence diagram for sync_to_async with autorestore_context

sequenceDiagram
    participant SyncFn as Synchronous Function
    participant geventGreenlet as gevent.Greenlet
    participant greenlet_to_future as greenlet_to_future
    participant asyncioEventLoop as asyncio Event Loop
    participant Future as asyncio.Future

    SyncFn->>geventGreenlet: gevent.Greenlet(fn, *args, **kwargs)
    activate geventGreenlet
    geventGreenlet->>greenlet_to_future: greenlet_to_future(greenlet, autorestore_context=True)
    activate greenlet_to_future
    greenlet_to_future->>asyncioEventLoop: loop.run_in_executor(fn)
    activate asyncioEventLoop
    asyncioEventLoop-->>greenlet_to_future: Result from fn
    deactivate asyncioEventLoop
    greenlet_to_future->>Future: Resolves with result
    deactivate greenlet_to_future
    Future-->>geventGreenlet: Result
    geventGreenlet-->>SyncFn: Result
    deactivate geventGreenlet

    opt autorestore_context is True
        geventGreenlet->>SyncFn: Restore context variables
    end

Sequence diagram for async_to_sync with autorestore_context

sequenceDiagram
    participant AsyncCoroutine as Asynchronous Coroutine
    participant future_to_greenlet as future_to_greenlet
    participant geventGreenlet as gevent.Greenlet
    participant asyncioEventLoop as asyncio Event Loop
    participant Future as asyncio.Future

    AsyncCoroutine->>future_to_greenlet: future_to_greenlet(coroutine(), autorestore_context=True)
    activate future_to_greenlet
    future_to_greenlet->>asyncioEventLoop: loop.create_task(coroutine())
    activate asyncioEventLoop
    asyncioEventLoop-->>future_to_greenlet: Future
    deactivate asyncioEventLoop
    future_to_greenlet->>geventGreenlet: gevent.Greenlet(run_until_complete, future)
    activate geventGreenlet
    geventGreenlet->>Future: Await Future
    Future-->>geventGreenlet: Result
    deactivate geventGreenlet
    future_to_greenlet-->>AsyncCoroutine: Result
    deactivate future_to_greenlet

    opt autorestore_context is True
        geventGreenlet->>AsyncCoroutine: Restore context variables
    end

File-Level Changes

Change Details Files
Introduces an autorestore_context option to greenlet_to_future, async_to_sync, and sync_to_async functions, enabled by default, to automatically pass and restore context variables when switching between asyncio and gevent.
  • Added autorestore_context parameter to greenlet_to_future, async_to_sync, and sync_to_async functions.
  • Modified greenlet_to_future to copy the context and restore it after execution.
  • Modified async_to_sync to copy the context and restore it after execution.
  • Modified sync_to_async to copy the context and restore it after execution.
  • Added tests to verify context passing and restoration for async_to_sync and sync_to_async.
tests/test_asyncio_on_gevent.py
asyncio_gevent/greenlet_to_future.py
asyncio_gevent/future_to_greenlet.py
asyncio_gevent/async_to_sync.py
asyncio_gevent/sync_to_async.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an issue from a review comment by replying to it. You can also reply to a review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull request title to generate a title at any time. You can also comment @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in the pull request body to generate a PR summary at any time exactly where you want it. You can also comment @sourcery-ai summary on the pull request to (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the pull request to resolve all Sourcery comments. Useful if you've already addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull request to dismiss all existing Sourcery reviews. Especially useful if you want to start fresh with a new review - don't forget to comment @sourcery-ai review to trigger a new review!
  • Generate a plan of action for an issue: Comment @sourcery-ai plan on an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

  • Contact our support team for questions or feedback.
  • Visit our documentation for detailed guides and information.
  • Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai[bot] avatar Apr 06 '25 12:04 sourcery-ai[bot]

@gfmio can you review too?)

spumer avatar Apr 15 '25 17:04 spumer

Thank you for using asyncio-gevent, I'm glad you find it useful! Thank you also for your PR and sorry for the wait.

I've just had a quick look and I think ~~the PR looks good to me~~ (Edit: there is one deadlock issue that requires addressing). I agree that adding context restoration is important.

I tried to merge in main, but there are a few minor merge conflicts.

Can you please address those and do the deduplication the bot reviewers have highlighted?

Once that's taken care of, I'll merge in your PR and cut version 0.3.0.

Thanks!

gfmio avatar Jul 10 '25 15:07 gfmio