[DesignProposal]: Dag IR support DataCollection with schema

[zhujiming]

Background and Motivation

triton server support schema

Design

dag info add some information every node add input_info、output_info. it most likely the picture.

Pros and Cons

No response

Anything else? (Additional Context)

No response

[zhujiming]

change input_info as a list of tuples

[filip-halt]

1. Why would we need to include the function in the input info? It would require a separate dict to track all functions and their corresponding schema name. Since the schema names all have to be unique, we can drop the function from the tuple.

2. Should we assume if there is only one schema it must be the output schema (glob).

3. to_list() isn't dag-able right now as its result is a list and not a dc.

[filip-halt]

Furthermore, one issue found with current scheme indexing.

If we have a generator function, lets say a glob that outputs two columns, glob[(path1, path2)](), the corresponding index in the system will be (path1, path2), which looks exactly as if we did image_decode[path, image](), which would result in index (path, image). In this case, the engine will think that glob has an input of path1 and output of path2, instead of only having an output of path1 and path2. @reiase

[junjiejiangjjj]

We only need to support the DC which starts with dummy_input ends with as_function

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Close the stale issues and pull requests after 7 days of inactivity. Reopen the issue with /reopen.