pydeps
pydeps copied to clipboard
Feature Request: Pydeps for pipelines
Pydeps is an excellent tool for visualizing dependencies. I wonder if it could be adapted to show the dynamic dependencies of a particular pipeline. For example, imagine a simple pipeline built with numpy and matplotlib that goes from raw data to visualization.
Here's an example:
import numpy as np
from numpy_processing import process0, process1, process2 # which all have their own dependencies
from matplotlib_visualization import prepare_vis, vis # also with their own dependencies
fname = 'some_file_name.npy'
data = np.load(fname)
output0 = process0(data)
output1 = process1(output0)
output2 = process2(output1)
vis_prms = prepare_vis(output2)
vis(output2, vis_prms)
We've defined a pipeline which uses these methods in the following order:
np.loadnumpy_processing.process0numpy_processing.process1numpy_processing.process2prepare_visvis
Wouldn't it be cool to see a pydeps graph where these methods are provided as input, are positioned on the left column of a graph moving downwards, and the dependencies are shown to the right pointing to each part of the pipeline?
If this is possible, please let me know. If there's interest and I could be of assistance in writing code to support it, let me know.