Improvements for visual representation of tabular data errors

[roll]

Overview

As a goodtables.next (including of course an errors report structure) author and active user of goodtables-web I always had a feeling that we have a good potential for visual improvements of our table errors representation.

Here is an example - https://raw.githubusercontent.com/frictionlessdata/goodtables-py/master/data/invalid.csv

id,name,,name
1,english
1,english

2,german,1,2,3

This simple and small file has validation report like this:

Yes it of course great looking etc but as an {USER} I miss here a cognitive connection with my initial data. Of course after reading it from top-to-bottom errors are clear but it takes some time. And it's 5 rows table. But many people having tables with hundreds errors and thousands rows.

So what I see here that to create improved UX we could use:

• WYSIWYG ideas (reducing the gap between an actual data and report representation)
• highly interactive approach
  • show error text on-demand - usually table errors have some patterns - for example all values in 4 column have a bad type so there is no need to show error message about it on every row, enough to show it for one row on-demand and for others it will be clear because human brain is powerful thing) Also user has a good understanding of its data so always simple color hint is enough to understand an error
  • allow collapse columns etc - anything that could help people to really understand what's happening with their data

So I've played with some of ideas and prototyped it. For example interactive values view for the same data:

I would be happy to get a feedback on this topic.

@amercader @akariv @pwalsh @smth @danfowler @callmealien

[pwalsh]

@roll I understand what you are aiming for here, but it changes the scope of our current work considerably to consider this now.

We are developing with a ~fast moving iterative model, and a particular timeline. The original scoping for this component was designed to reuse the existing templates/ui that @smth designed last year, specifically to not go into a rabbit hole on them.

Ideally, the time for a discussion like this might have been when I first scoped the issue, but now, when we are trying to ship the final work, it feels like a big distraction to me.

Just a note: the UI/X of the current error templates is not perfect - of course. All the original goodtables and associated work was done on a much shorter timeline than the current set of iterations.

However, while the original templates are not perfect, they were tested with a number of non-technical users, and were proven to be effective in conveying information to fix issues in the source files. This was made manageable by stopping the error iteration on 20 errors (no point in showing a user more than that in a user interface).

So, specifically, unless @smth or @amercader can made a very compelling case for why we should rewrite the error interface now, I'm leaving this on the backlog.

[roll]

@pwalsh @amercader Also don't see we will be able to start this improvements now - so just need some input on https://github.com/frictionlessdata/goodtables.io/issues/150#issuecomment-283623814 (esp. handling multiple files) to finish the component with goodtables-web design

[pwalsh]

@roll the datapackagist implementation of these templates, linked to from my original issue for this work, has a treatment for multiple files.

[amercader]

+1 to Backlog. Basically shipping a version based in the existing templates would allow us to get even more feedback from many users and polish them later on

[amercader]

But of course great suggestions @roll!

[smth]

I'll just add that, as I recall, that error report design was intended to work much as you describe @roll . Looks like some ideas were lost somewhere along the way.