force thread/process/pool by kernel

[jonathan-beard]

In the RaftLib C++Now tutorial (full slide deck here: http://www.jonathanbeard.io/pdf/cppnow2016.pdf) I proposed this syntax for partitioning a VM space:

Seems intuitive at first, however when you have multiple ports on a single compute kernel then you begin to have issues (i.e. you need to specify the same thing on multiple stream graph edges). So I'm looking at something that works more like this:

raft::map m;

raft::kernel a,c;
/** all of b forced to a single process, call acts as a decorator **/
auto b_proc( raft::parallel::process( b ) );
m += a >> b_proc >> c;
m.exe();

The end goal is of course to be able to force a few kernels to a specific type of resource (i.e., isolated process, thread, etc.). This isn't normally necessary, however, it sometimes comes up when building applications that utilize special IO devices or those that have strange VM behavior.

Any thoughts?

[adishavit]

If you do not need to modify b_proc, would it be possible to write this:

raft::map m;

raft::kernel a,b,c;
/** all of b forced to a single process, call acts as a decorator **/
m += a >> raft::parallel::process( b ) >> c;
m.exe();

I'm kinda out of my depth here, but another possible syntax would be an iomanip style manipulator:

raft::map m;

raft::kernel a,b,c;
/** all of b forced to a single process, call acts as a decorator **/
m += a >> raft::parallel::process >> b >> c;
m.exe();

Both suggestions, obviously, assume you do not need access to the temporary object created by raft::parallel::process and that the downstream operator>> knows how to take and handle (e.g. copy or move) an rvalue (which will evaporate after the statement is done).

[jonathan-beard]

I actually like the iomanip style best, however if you have multiple incoming ports to "b" then it would seem you have to reiterate on ever stream...or perhaps one is sufficient...seems a bit awkward either way....when awkward is what I'm trying to avoid in the first place.

Sent from my tiny screen, powered by ARM awesomeness.

On Aug 29, 2016, at 02:32, Adi Shavit [email protected] wrote:

If you do not need to modify b_proc, would it be possible to write this:

raft::map m;

raft::kernel a,b,c; /** all of b forced to a single process, call acts as a decorator **/ m += a >> raft::parallel::process( b ) >> c; m.exe(); I'm kinda out of my depth here, but another possible syntax would be an iomanip style manipulator:

raft::map m;

raft::kernel a,b,c; /** all of b forced to a single process, call acts as a decorator **/ m += a >> raft::parallel::process >> b >> c; m.exe(); Both suggestions, obviously, assume you do not need access to the temporary object created by raft::parallel::process and that the downstream operator knows how to take and handle (e.g. copy or move from it) an rvalue (which will evaporate after the statement is done).

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[adishavit]

What would a multi-input b code look like? Can you give some examples of this case in various syntices (!?) ?

[jonathan-beard]

Single port in-line iomanip-like syntax:

raft::map m;
raft::kernel a,b,c;
/** all of b forced to a single process, call acts as a decorator **/
m += a >> raft::parallel::process >> b >> c;
m.exe();

Dual port in-line iomanip-like syntax:

raft::map m;
raft::kernel a,c,d;
/** fake dual port kernel for example purposes **/
raft::dualPortKernel b;
/** all of b forced to a single process, call acts as a decorator **/
m += a >> raft::parallel::process >> b[ "x0" ] >> c;
m += d >> raft::parallel::process >> b[ "x1" ];
m.exe();

Visually this would look something like this:

However, using the decorator syntax from above, we could shorten to this:

raft::map m;
raft::kernel a,c,d;
/** all of b forced to a single process, call acts as a decorator **/
auto b_proc( raft::parallel::process( b ) );
m += a >> b_proc[ "x0" ] >> c;
m += d >> b_proc[ "x1" ];
m.exe();

Again, I don't anticipate that this type of syntax would be needed that often. I'm just concerned/dismayed that the solution I like the most with the iomanip-like operators requires the user to specify raft::parallel::process twice (and remember to do so). I suppose we could throw an exception if that weren't the case, however that is still clunky.

I'm definitely open to anybody with better solutions..but it seems for a (hopefully) little used feature, that decorators as the last example shows, might be the best option in the end. That being said, you could also have a modifier on the decorator constructor to also force core affinity or hardware type.

[adishavit]

I agree that the second option does indeed seem more readable in the case of multiple ports. Is there something preventing both usages? The iomanipulator might be named slightly differently so there is no confusion. Just like std manipulator a that are named differently than the corresponding stream methods.