Thespian icon indicating copy to clipboard operation
Thespian copied to clipboard

use epoll on linux

Open pjz opened this issue 8 years ago • 3 comments

select.select() is significantly slower on linux than select.epoll(), so, especially in the case of network services, there's a huge performance gain in using epoll.

I see a few ways forward:

  1. declare it WontFix.
  2. abstract the filehandle-watching so it can be switched based on what OS is running
  3. find/use something like libevent() which does the abstraction for you
  4. make completely different SystemBases - one for select(), one for epoll()

pjz avatar Mar 25 '16 18:03 pjz

I agree that select() is generally the least-performant approach. It is the currently implemented form because it is also the most broadly portable (for both operating systems and python versions).

In reference to #3 in your list, Python has the "selectables" in the standard library, but this was only introduced in 3.4.

Right now, Thespian supports all versions of Python from 2.6 through 3.5 by taking the "common path" as much as possible. This is definitely a performance issue that should be addressed at some point, so I'm not going to take method 1 (WontFix), and I would prefer to provide internal adaptation and avoid a proliferation of SystemBases (i.e. your method 2 as preference to method 4). If there is too much variation or there is a need for externally-supplied configuration, it may be necessary to introduce a new SystemBase but in general I would prefer to internalize the complexity to keep the Actor usage simple.

Keeping this as an open enhancement request to flag this for future performance work.

kwquick avatar Mar 25 '16 19:03 kwquick

By "selectables" do you mean selectors ? If so, the selectors34 package provides a forward-compatible version for python 2.6, 2.7 and 3.3... perhaps that's the correct way forward? (Also, thanks for bringing that to my attention, I hadn't seen it before!)

pjz avatar Mar 28 '16 12:03 pjz

another option would be just to use asyncio?

ysangkok avatar Dec 22 '17 22:12 ysangkok