Feature Request: Pydeps for pipelines

[landoskape]

Pydeps is an excellent tool for visualizing dependencies. I wonder if it could be adapted to show the dynamic dependencies of a particular pipeline. For example, imagine a simple pipeline built with numpy and matplotlib that goes from raw data to visualization.

Here's an example:

import numpy as np
from numpy_processing import process0, process1, process2 # which all have their own dependencies
from matplotlib_visualization import prepare_vis, vis # also with their own dependencies

fname = 'some_file_name.npy'
data = np.load(fname)

output0 = process0(data)
output1 = process1(output0)
output2 = process2(output1)

vis_prms = prepare_vis(output2)
vis(output2, vis_prms)

We've defined a pipeline which uses these methods in the following order:

1. np.load
2. numpy_processing.process0
3. numpy_processing.process1
4. numpy_processing.process2
5. prepare_vis
6. vis

Wouldn't it be cool to see a pydeps graph where these methods are provided as input, are positioned on the left column of a graph moving downwards, and the dependencies are shown to the right pointing to each part of the pipeline?

If this is possible, please let me know. If there's interest and I could be of assistance in writing code to support it, let me know.

[thebjorn]

Hi Andrew, thank you for your kind words. Your idea does sound interesting (and even useful), but perhaps not a perfect fit for pydeps..? I've known to occasionally be wrong, though, and I'm always happy to merge PRs (I'm also happy to help if you get stuck...)

[landoskape]

Let me look around and have a go at it. I'll ping if I need some help. Thanks!