schedule
schedule copied to clipboard
dbader
AsyncScheduler: Ability to schedule сoroutines
Hey. Added the ability to create a scheduler for asynchronous tasks. The implementation of AsyncScheduler does not affect the existing functionality in any way and backward compatibility is fully supported. Even on python 2.7 the library also continue works. The new functionality has been covered with tests.
P.S. I plan to use this tool in my projects, so the asynchronous part will be developed and maintained by me.
Coverage remained the same at 100.0% when pulling 8e1c263ec130c513101b2cdaff8c832232d7d2ee on pirroman:async_scheduler into 705f3730e40fd1a4451dedcf28de510aa5c86212 on dbader:master.
bump again. This is a great feature. I pulled the branch and it works perfectly :)
I have related pull request https://github.com/dbader/schedule/pull/391 and i noticed yours now.
But you solution for running corutines is much better....mine was mostly concerned about handling os signals fast (even with long sleeps/waits between runs of run_pending)
i think we should join our efforts since you will eventualy have to write about pitfalls of asyncio and fact it will run concurenly all sechuded jobs... so control/data race condition may emerge
@07pepa, Hello! I'm so sorry for so long to respond. I am always in favor of cooperation. I suggest you make a PR to my fork. I promise to review it soon as possible.
@07pepa, Hello! I'm so sorry for so long to respond. I am always in favor of cooperation. I suggest you make a PR to my fork. I promise to review it soon as possible.
i will try to get it to it in next few weeks
Hi @pirroman, thanks a lot for your continued efforts on this pr! My apologies for this overdue response.
You are right, the world needs a simple library that can handle scheduling async jobs. Your implementation seems great. We just have to figure out how it best fits in the current codebase.
Assuming we drop python 2 support (which we should probably do before next release) we could just add the async methods to the main scheduler. In that case we wouldn't need to refactor and keep everything in a single scheduler class. Also, we could keep using the same testsuite for all python versions. I'm not entirely sure what the drawbacks of extra async functions in the main scheduler would be, ideas are welcome :smile:
Maybe something like?
schedule.every().day.do_async(job)
Some background on why I'm hesitant to refactor like this
This library is built to be a super simple solution for simple scheduling tasks. There are other great libraries out there that tackle the more advanced features like job persistence, multithreading and coroutines (like APSchedule). The power of schedule is simplicity and the goal is to keep it that way. Simplicity includes having a single source file and having few classes.
By adding the AsyncScheduler class, we are introducing a new concept to users: the scheduler type (the normal scheduler vs async scheduler). At this moment the extra scheduler is a small superset of features. But it opens the door for more types of schedulers... Next people will be building BackgroundScheduler and ThreadedScheduler, which is totally understandable! But I don't think this is the direction this repo/library should move towards.
On the contrary, having recipes and examples in the docs to show how to build more advanced systems is great because people can mix those into their own solutions.
into its own library?
Another idea is to have async-schedule as its own dedicated library. Your fork is already being used successfully and it can only get better from here. The fork could drop python 2.7 support, only support async and everything else you think a asyncio library needs. If you want you could keep the fork up-to-date with this library or import this library as a dependency and extend the Scheduler class, that is all up to you. Last thing left to do would be publishing it to pypi, update the docs and the world has a great new package. If you choose this route I would be happy to add a link to that package in the docs of this project.
But honestly the idea of having yet another scheduling library doesn't feel right either.
I would love to hear what you think about all of this.
Hi @SijmenHuizenga!
Assuming we drop python 2 support (which we should probably do before next release) we could just add the async methods to the main scheduler. In that case we wouldn't need to refactor and keep everything in a single scheduler class. Also, we could keep using the same testsuite for all python versions. I'm not entirely sure what the drawbacks of extra async functions in the main scheduler would be, ideas are welcome 😄
I don't have a problem with dropping support for python2 and python3.5. Then it would be easy to merge the tests and library support would be greatly simplified. But it doesn't have to be done right now, I wouldn't want to do that work as part of this request. Besides, while this request is pending, I'm getting conflicts( You can make an issue and I will do it in the next request.
schedule.every().day.do_async(job)
There is no sense in explicitly dividing tasks into synchronous and asynchronous. Asynchronous scheduler can handle all types. So the problem is with the methods: run_pending and run_all. Is it worth moving these methods to the current Scheduler? It seems to me not, because the delimitation will become less strict. Now, having an asynchronous scheduler you have no possibility to call synchronous methods run_pending and run_all, after merging this possibility will appear. All in all, for users the library will turn into mush. That' s my opinion.
Simplicity includes having a single source file and having few classes.
I don't agree that simplicity comes from having a single file. I thought that in this case simplicity meant ease to use and a low threshold to get into the library. I may have misinterpreted the concept of simplicity of this library. But I agree to discuss the possibility of simplicity by separating classes into separate files and by designing a simple and clear architecture. In my opinion it is easier to navigate the code base of the library this way, but I also respect your opinion on this.
By adding the AsyncScheduler class, we are introducing a new concept to users: the scheduler type (the normal scheduler vs async scheduler). At this moment the extra scheduler is a small superset of features. But it opens the door for more types of schedulers... Next people will be building BackgroundScheduler and ThreadedScheduler, ...
Here I agree and disagree. This change is due to the extension of the python language - it became possible to declare asynchronous functions(coroutines), and the lack of support for asynchronous functions and gave a lot of limitations of this library in the modern language. Those examples you described are BackgroundScheduler and ThreadedScheduler. The tasks that should be handled by these concepts could be handled by AsyncScheduler. So I don't see any obvious reason for additional concepts to appear until another type of function appears in the language 😄.
into its own library?
I honestly don't like that idea very much either. There would be a need for constant synchronization, and besides I have made changes to the organization of the code base, which would make any synchronization an unpleasant process. I propose to consider this as an extreme case.
P.S. Thanks for the feedback. I hope that we will soon come to a common solution that satisfies everyone! I am ready to participate in further support of my solution. So if new issues are born in view of our discussion, I am ready to do it.
Thanks a lot for the detailed response! I will take some time to experiment and get my thoughts together. But first I will be working on the 1.0.0 release as detailed in #412. After that I will be back here :)
As an experiment I've implemented async support by adding run_pending_async and run_all_async to the main scheduler. See #438. I would love to hear what you think about supporting coroutines in such a way @pirroman
@SijmenHuizenga @ndbeals I apologize for this looong delay. I promise I'll get back to progress this weekend.
@SijmenHuizenga Honestly, I like my version more. It is stricter and more structured. That is, the explicit creation of an asynchronous scheduler in the code will explicitly talk about the ability to handle asynchronous tasks, while combining asynchronous and synchronous interfaces creates more risks of using synchronous interface on asynchronous tasks.
In the recent changes I went a little further and used inheritance only to share common functionality. Interface classes are no longer divided, thought that one interface returning different types is bad.
I'm also looking for a way to schedule async tasks. Would love to get an update on if / when this might be merged!
It'll be nice if a note was placed in the README indicating no support for this library is being provided any longer
The library had commits this year, but for some reason the developers don't answer on this PR
@dbader not sure if this PR is just flying under the radar on your github notifications, but this seriously needs merged
Hello @SijmenHuizenga @dbader @connorskees Any chance of getting this merged? Having async support would be very helpful for this library.
Thanks!