Research: Viz usage as a React component

[tynandebold]

Description

We'd like to know how many users are using Viz as a React component.

Context

Do we need to continue supporting this use case?

Outcome

An answer to the question above.

Checklist

• [x] Include labels so that we can categorise your feature request

[tynandebold]

On 5 July, 2022, the following question was posed to the appropriate users channels in Discord and Slack:

Hello, everyone! We're looking to understand your usage of Kedro-Viz. Specifically, do you use it as a React component? https://www.npmjs.com/package/@quantumblack/kedro-viz

👍 for yes, 👎 for no

After ~48 hours, we have:

• Nine no answers
• Four yes answers from
  • Nikos, Nab, @datajoely, Tobias

It's also thought that these two products might be using Viz as a React component:

• ~~https://tokern.io/data-lineage/~~ This product is all Python.
• ~~https://realityfold.com/ (@921kiyo)~~

Next step: follow up with each positive (yes) response, plus the two products linked above, and as them about how they're using Viz. Specifically:

• What features they're using?
• What features they're not using?
• Do they need any interactivity?
• Would you be interested in a version of Viz that only displays the Flowchart (a.k.a. pipeline visualization) and nothing more?
• ...

[921kiyo]

@tynandebold Thank you for tagging me! I am not using Viz as a React component, I just used the Viz pipeline image in my website (sorry for the confusion...)

[tynandebold]

No confusion! Thank you for clarifying :)

[datajoely]

Thanks for pushing this @tynandebold I have a couple of thoughts on this.

In my eyes, much of the value of Viz has been providing largely back-end facing engineers with the ability to communicate to state/function/purpose of their data engineering / science efforts to non-technical audiences. I think there is potential to strive further in this direction.

• I think the Tokern and Reality Fold implementations provide some evidence that there aren't many 'structured flowchart' visualization libraries and when people with front end skills need a framework to do this sort of layout they've (admirably) reverse engineered Kedro-Viz to work for their purposes.
• In terms of Kedro-core user base, I imagine many of the profiles match my own i.e. Python background, and are a little bit scared anytime they need to type npm .... This is where the React component would be start, but only half the battle as it's not immediately useful to Python engineers.

In my own experience this medium article was a fun but painful experience to prepare. I had to work closely with @studioswong to find a way of compiling Viz as a SPA so that I could embed a version of the flowchart and also disable the sidebar behaviour. In my eyes, it should be simple to just say Viz component + JSON = Visualization.

With this in mind, where do I see real potential for this direction? DOCS

I think there are many places where a user may want to share a sub-graph of their pipeline and annotate / describe the purpose embedded in their docs. The Python world uses Sphinx as the primary documentation generator and this provides a plugin entrypoint where we could a native way of adding a Kedro Viz widget within the docs.

There Mermaid plugin feels like a good analogue for what we would need to do this properly, although I think shorter routes to MVP will exist.

[tynandebold]

Great context @datajoely, thank you.

Some thoughts and questions:

1. When you say, "This is where the React component would be start" do you mean if we had Viz a React component?
2. For your medium article, why didn't you use the export functionality of Viz and then add those output images to your article? Did you want the embed to be interactive? Your article was all about "layered thinking" and thus you showed and referenced the layers in Viz. The image exports don't actually include those, though if they did, would you have used them?
3. We do actually have this: Viz component + JSON = Visualization. That's our React component. Perhaps that isn't ideal, as you need to create a small standalone application to get the output. Instead, and still using your formula, what if we output a standalone and single HTML file with the necessary CSS and JS included to display the Visualization? Would that be better and more usable? Could you have used that in Medium? Or even in a GitHub readme?
4. For use in Docs, I think the image output of Viz is feasible, though maybe not feature rich right now? We have a feature request from a user (#638) where they mention they'd like a way to programmatically export/save the visualization for exactly this reason ("automatic documentation updates"). We're thinking about how to do this now.

[studioswong]

Hi folks - nice discussion! joining in the fun with my 2 cents here.

@tynandebold to your second and third point, my understanding is that @datajoely wants to showcase the interactive kedro-viz pipeline using medium embeds via a glitch hosted link (see this article). I had a brief look at embedding and rendering an html file as an embed in medium and don't think Medium supports it natively - i.e doing so will still require you to upload and host the html file somewhere which is probably the same amount of effort as setting it up as a react component export.

My curiosity is - what is the reason to not support the react component usage? It might seem counter-intuitive to look into setting up a new function to export and output a standalone HTML file in kedro-viz rather than utilizing the npm export setup which we have already setup nicely within our CI pipeline. There are other QB products who are consuming kedro-viz via npm ( such as Brix) - does stopping the npm support means they won't get the new goodies from latest versions of kedro-viz?

In terms of the the user request from #638 , to me it seems like a different use case as it looks like the user just wants a fast way to export svg/png directly via the terminal rather than via the Kedro-Viz UI, which is an entire different use case from those who consumes kedro-viz via an npm component ( which most likely they utilize it to showcase the interactive flowchart)

[datajoely]

@tynandebold

1. Yes I mean, IF we had it, we could potentially do some cool things.
2. My thinking is that a when describing a complex pipeline the ability to interactively fold and interrogate is super useful and better than any static image.
3. For Medium you need to use one of their accredited embed providers - Ideally I would have used Codeopen to just pass the JSON to a component and it would work fine. In this case I needed to do a much more involved process to get Glitch working.
4. I think this is the same point as 2, we can do static images fine - interactivity is the exciting part.

[tynandebold]

@datajoely

1. We do have that though. Am I missing something or are we speaking of different things?
2. Agreed.
3. Good point. I've tried using the Viz React component in CodePen and it doesn't work.
4. Noted.

[tynandebold]

@studioswong thanks for chiming in!

My curiosity is - what is the reason to not support the react component usage?

The short answer is to reduce complexity of the app in tandem with reducing the workload when building some potential future features.

There are other QB products who are consuming kedro-viz via npm ( such as Brix) - does stopping the npm support means they won't get the new goodies from latest versions of kedro-viz?

No, we still want to support other teams using Viz while trying to determine if the way they're using it is most optimal. For the Brix case and their setup, there was no other way to use it, so they took what we offered and ran with that. Yet what if there was a more suitable alternative (e.g. a HTML-generated view of the pipeline, a Kedro-Viz-Lite NPM package only showing the flowchart, a more rich static image export, etc., etc.)? Would that have been preferable?

This are some of the things we're trying to answer.

[datajoely]

Re 1 - As a novice JS user my understanding of 3 (Codepen) thought this wasn't valid.

[tynandebold]

You can basically import anything from NPM into CodePen. Here's a small React app running in there. I've tried using Viz in here and unfortunately we get an error with one of our dependencies. If we didn't, it should work just fine.

[tynandebold]

Given the feedback from user interviews, we'll continue to support using Viz as a React component.

We should strive to better our user's experiences with things like modularity, being lighter weight, adding access to click handlers, and more.

[tynandebold]

Closing this issue as the conclusion was reached that we'll still support this use case.