cljol icon indicating copy to clipboard operation
cljol copied to clipboard

cljol's view doesn't play well with large data structures

Open jumarko opened this issue 5 years ago • 5 comments

Trying this on any vector containing more than a few hundreds of elements is painful. Not only the rendering process takes very long but the resulting picture is unusable. I tried it with (cljol.dig9/view (vec (range 1000))).

It would be nice if the visualizations were limited to certain "length" / "depth" and parent nodes also contained a "summary" size for all their children together.

jumarko avatar Jun 27 '19 13:06 jumarko

Agreed that large dumps of data are too much for a person to deal with :)

A summary size for all objects reachable from a given object is a nice addition to what is there now, as long as you realize that those can "overlap", i.e. a common set of objects might be reachable both from objects A and B, although neither A nor B can get to each other by following chains of references.

A depth restriction on what is displayed - yes, also sounds useful.

I am not sure what you mean by a length limit? As in, maximum number of references out of one object that are included? Which subset would you pick if the limit was exceeded?

jafingerhut avatar Jun 28 '19 17:06 jafingerhut

I don't have a super-clear idea how this should work but I was thinking about something similar to print-length

jumarko avatar Jun 30 '19 17:06 jumarko

If you try the latest version it uses the ubergraph library to create an intermediate graph before drawing, so all of loom's and ubergraph's algorithms are available for examining it and creating other graphs.

I have implemented the "total size" of all reachable objects, as well as the number of reachable objects, as a label for every object in the graph. Also a function that gives some summary statistics about the graph in text form.

It does not yet implement the depth limit, but that will be a pretty small amount of work from its current state.

jafingerhut avatar Jul 02 '19 09:07 jafingerhut

That's cool, thanks!

jumarko avatar Jul 02 '19 09:07 jumarko

OK, code and README on master branch updated a few minutes ago, with examples showing how to create the full graph for one or more objects, then to trim it down in a couple of ways: remove all leaf nodes (which could be repeated), or to limit the node's distance in reference hops away from one of the starting objects.

Oh, and also a function that prints summary statistics about the graph, without showing it, so you can decide whether you want to show it, or first use one of the trimming functions.

jafingerhut avatar Jul 03 '19 19:07 jafingerhut