ray-project
[ADAG] Add visualization of compiled graphs
Why are these changes needed?
https://github.com/ray-project/ray/issues/47945
My test case
import ray
import time
import random
from ray.dag.input_node import InputNode
from ray.dag.output_node import MultiOutputNode
ray.init(address = "10.218.163.33:6274")
import os
@ray.remote
class Actor:
def __init__(self, init_value, fail_after=None, sys_exit=False):
self.i = init_value
self.fail_after = fail_after
self.sys_exit = sys_exit
self.count = 0
def echo(self, x):
self.count += 1
return x
def sleep(self, x):
time.sleep(x)
return x
@ray.method(num_returns=2)
def return_two(self, x):
return x, x + 1
def read_input(self, x):
return x
a = Actor.remote(0)
b = Actor.remote(0)
single_fetch = True
with InputNode() as i:
o1, o2 = a.return_two.bind(i)
o3 = b.echo.bind(o1)
o4 = b.echo.bind(o2)
dag = MultiOutputNode([o3, o4])
compiled_dag = dag.experimental_compile()
for _ in range(3):
refs = compiled_dag.execute(1)
if single_fetch:
for i, ref in enumerate(refs):
res = ray.get(ref)
assert res == i + 1
else:
res = ray.get(refs)
assert res == [1, 2]
compiled_dag.visualize()
compiled_dag.teardown()
plot
Related issue number
Checks
- [√] I've signed off every commit(by using the -s flag, i.e.,
git commit -s) in this PR. - [√] I've run
scripts/format.shto lint the changes in this PR. - [ ] I've included any doc changes needed for https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I added a
method in Tune, I've added it in
doc/source/tune/api/under the corresponding.rstfile.
- [ ] I've added any new APIs to the API Reference. For example, if I added a
method in Tune, I've added it in
- [ ] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
- [x] Unit tests
- [x] Release tests
- [ ] This PR is not tested :(
@ruisearch42 is this different from the visualization function in your overlap PR?
@ruisearch42 is this different from the visualization function in your overlap PR?
This is to visualize the DAG itself, the overlap PR visualizes the schedule. Both will be useful.
Hi, @ruisearch42 , I just update it and solved you comments. The new plots is above with different color and shape for different node. The tests are also included.
I suspect that graphviz is not available on the test environment so we need to pip install graphviz and sudo apt-get install graphviz for visualization test. Do you know how we can achieve that? I need to add dependency to which file for pip and apt-get? @ruisearch42
I suspect that graphviz is not available on the test environment so we need to pip install graphviz and sudo apt-get install graphviz for visualization test.
@Bye-legumes maybe you can try to use runtime environment.
Btw, would you mind answering my question here: https://github.com/ray-project/ray/pull/47958#pullrequestreview-2361319963? Thanks!
I suspect that graphviz is not available on the test environment so we need to pip install graphviz and sudo apt-get install graphviz for visualization test.
@Bye-legumes maybe you can try to use runtime environment.
Btw, would you mind answering my question here: #47958 (review)? Thanks!
This is for the complied graph with different nodes that we have different node handling for different node for complied graph. Also the data transport method is also included as we have hint. This is the result of that plot.
How I can set runtime environment for this test case? Should install packages in the test.py with pip install graphviz and sudo apt-get install graphviz
@Bye-legumes Auto merge is enabled, but tests are failing, please fix
I see, thx you keep the eye on this! I think this tests failed is not related as other PR also failed on same test. Let me merge the master and check if it will recurs!