perlcassa
perlcassa copied to clipboard
mkjellman
Use of perl threads
heya, just a heads up that the use of threads in perl is pretty much avoided at all costs. Maybe take a look at IO::Async ( event loop based )? Or if you want, I'd totally lend a hand.
threads are only used in one place where i deal with failure logic, and even then it's only to determine when to place the node back in the pool.
I personally hate modules that have a million dependencies but if you think it will improve the project i'm more than happy to work with you to refactor it around IO::Async or whatever module you would prefer.
is there a specific performance issue or are you more talking about best practices here?
Just best practices, I'm trying to find a decent source for the reasoning behind it; best I could find is http://grokbase.com/t/perl/beginners/1126w495zq/perls-threads . The specific performance issue I know of with perl threads is that you need to keep a pool of threads, you can't just spawn and kill them at will because they will leak memory. I definitely understand avoiding excess dependencies, I'll poke around and see if I think of something.
On Wed, Jan 16, 2013 at 10:21 PM, mkjellman [email protected]:
threads are only used in one place where i deal with failure logic, and even then it's only to determine when to place the node back in the pool.
I personally hate modules that have a million dependencies but if you think it will improve the project i'm more than happy to work with you to refactor it around IO::Async or whatever module you would prefer.
is there a specific performance issue or are you more talking about best practices here?
— Reply to this email directly or view it on GitHubhttps://github.com/mkjellman/perlcassa/issues/10#issuecomment-12352772.
believe me, i'm definitely not the first to suggest threads in perl for any high performance situation. and i've had my fair share of bad experiences with threads in perl, but in more recent releases they have definitely stabilized. Performance is bad when you spawn a thread because under the hood it's a fork.
this came up in IRC as well in a question from rcoli. let's refactor this :)
do you have a specific implementation in mind?
Thats no coincidence, we contract out to Palomino db at my job (SocialFlow.com) and i mentioned it in #socialflow On Jan 17, 2013 12:15 AM, "mkjellman" [email protected] wrote:
this came up in IRC as well in a question from rcoli. let's refactor this :)
do you have a specific implementation in mind?
— Reply to this email directly or view it on GitHubhttps://github.com/mkjellman/perlcassa/issues/10#issuecomment-12354818.
(rcoli works for Palomino db) On Jan 17, 2013 12:18 AM, "Samuel Kaufman" [email protected] wrote:
Thats no coincidence, we contract out to Palomino db at my job (SocialFlow.com) and i mentioned it in #socialflow On Jan 17, 2013 12:15 AM, "mkjellman" [email protected] wrote:
this came up in IRC as well in a question from rcoli. let's refactor this :)
do you have a specific implementation in mind?
— Reply to this email directly or view it on GitHubhttps://github.com/mkjellman/perlcassa/issues/10#issuecomment-12354818.
well glad the system is working. priority is getting the client ready for 1.2 and CQL3 but this seems like something pretty easy to refactor.
my key requirement would be that it continues to play nice in mod_perl
wanted to update this thread. I'm almost done implementing the binary protocol. This is all based on anyevent which should allow us to remove the dependency of the downed node thread using perl threads. I'll update when i've done more testing of the binary protocol implementation