sematic icon indicating copy to clipboard operation
sematic copied to clipboard

Auto convert list-of-futures to future-list for non-inline cloud job roots

Open augray opened this issue 2 years ago • 1 comments

When do_something() is a non-inline Sematic func in the example below, the root func is marked as resolved, but there is an error attached to it and do_something() never gets called.

@sematic.func(inline=False)
def pipeline(a: float, b: float, c: float) -> List[str]:
    result = [do_something() for _ in range(0, 2)]
    return result

Error:

Traceback (most recent call last):
  File "/app/sematic/examples/bazel/bazel_image.binary.runfiles/sematic/sematic/resolvers/state_machine_resolver.py", line 298, in _update_future_with_value
    value = future.calculator.cast_output(value)
  File "/app/sematic/examples/bazel/bazel_image.binary.runfiles/sematic/sematic/calculator.py", line 166, in cast_output
    return self.cast_value(
  File "/app/sematic/examples/bazel/bazel_image.binary.runfiles/sematic/sematic/calculator.py", line 192, in cast_value
    raise TypeError(
TypeError: Invalid output type for 'sematic.examples.bazel.pipeline.pipeline'. Cannot cast [<sematic.future.Future object at 0x7f6f289ac580>, <sematic.future.Future object at 0x7f6f289ac910>] to typing.List[str]: Cannot cast [<sematic.future.Future object at 0x7f6f289ac580>, <sematic.future.Future object at 0x7f6f289ac910>] to typing.List[str]: Cannot cast <sematic.future.Future object at 0x7f6f289ac580> to <class 'str'>

Luckily there is a work-around until this is fixed:

@sematic.func(inline=False)
def collect_result(result: List[str]) -> List[str]:
    return result

@sematic.func(inline=False)
def pipeline(a: float, b: float, c: float) -> List[str]:
    result = [do_something() for _ in range(0, 2)]
    return collect_result(result)

this leverages the fact that we automatically handle list-of-future to future-list for func inputs

augray avatar Oct 14 '22 19:10 augray

👍🏼

augray avatar Nov 04 '22 18:11 augray