kedro-viz icon indicating copy to clipboard operation
kedro-viz copied to clipboard

[Proposal] Remove experiment tracking from Kedro-Viz

Open tynandebold opened this issue 2 years ago • 2 comments
trafficstars

Description

Experiment tracking should be removed from Kedro-Viz.

Context

For a combination of reasons (competition from other products, an initial MVP release that wasn't viable, ineffective marketing, etc.), experiment tracking wasn't well adopted and usage has plateaued.

There's not much wrong with maintaining a feature that has lukewarm adoption and low complexity. Experiment tracking is a much higher-complexity feature that adds to and creates tech debt of its own while simultaneously introducing an entire new type of API.

Lastly, with the release of the Shareable URL feature, we're now able to host the Kedro-Viz flowchart page statically. We can't do this with experiment tracking, and have to filter out this feature when the app is hosted in this manner. Removing it would free up this maintenance and continue to open to door to a plethora of static hosting solutions.

Supporting evidence (note: all charts are from 1 Dec. 2021 - 31 Oct. 2023):

Unique users, total

image

Note here that in July 2022, we had a then-high number of 89 users, which hasn't been meaningfully surpassed since, nearly 1.5 years later. We also haven't seen any increase in adoption since we released collaborative experiment tracking on 26 May 2023.


Unique users, run selection

image

The high of 20 in June 2022 hasn't been surpassed.


Unique users, toggle compare runs

image

The high of 14 in March 2022 hasn't been surpassed.


Unique users, selection of Metrics tab (chart data dates back one year only)

image

Unique users, download run data (chart data dates back one year only)

image

Very little engagement here.


Possible Implementation

Remove all features related to experiment tracking and all code that powers those features.

Possible Alternatives

Remove experiment tracking from Kedro-Viz and create a separate Kedro plugin with the same feature set.

Checklist

  • [x] Include labels so that we can categorise your feature request

tynandebold avatar Nov 03 '23 17:11 tynandebold