seamless icon indicating copy to clipboard operation
seamless copied to clipboard

ContextPool executor to execute multiple workflows in parallel

Open sjdv1982 opened this issue 1 year ago • 0 comments

Mixed-style Seamless is expected to be in demand, especially in case of compiled transformers, which must always be part of a workflow, in combination with imperative syntaxes.

For the 0.12 release, there is already a planned feature seamless-runner / seamless-parallel-runner in order to execute workflow graphs from command-line (preferably under bin/seamless).

In addition, it will be necessary to execute workflows imperatively in parallel, as an imperative alternative to stdlib.map . For now, call this a ContextPool in analogy to multiprocessing.Pool . Here is some experimental working code that needs generalization:


NCONTEXTS=3

for n in range(NCONTEXTS):
    print(n)
    ctx = Context()
    ctx.set_graph(build_rotamer_graph)
    tf = ctx.build_rotamers
    tf.debug.direct_print = True
    ctx.translate()
    contexts.append(ctx)

for conformer in range(len(conformers)):
    print(conformer)
    ctx = contexts[conformer % NCONTEXTS]
    ctx.compute()
    if conformer >= NCONTEXTS:
        tf = ctx.build_rotamers
        print(conformer, tf.result.checksum, tf.status)        
    ctx.random_rotations = random_rotations
    ctx.scalevec = tensors[conformer][1]
    ctx.hierarchy = pre_analyses[conformer]["hierarchy"]
    
for conformer in range(len(conformers)-NCONTEXTS, len(conformers)):
    print(conformer)
    ctx = contexts[conformer % NCONTEXTS]
    ctx.compute()
    tf = ctx.build_rotamers
    print(conformer, tf.result.checksum, tf.status)        

sjdv1982 avatar Nov 21 '23 13:11 sjdv1982