bids-website icon indicating copy to clipboard operation
bids-website copied to clipboard

Provide mermaid diagram of relation between BIDS schema, specification and validators

Open yarikoptic opened this issue 10 months ago • 19 comments

This is based of the mermaid diagrams in the blog post we have: https://bids-website.readthedocs.io/en/latest/blog/2024/11/13/bids-validator-2.html and the goal is to give a visual depiction of relations between the components mentioned on this page and elsewhere.

  • I think with this we can Close #625

Here is a screenshot of preview. I do agree that text is a bit small'ish, so we might want to prune it but it is quite usable and IMHO useful... May be there is a way to make it "full screen"?

version1: horizontal, harder to read, no zooming etc

image

version2: vertical

image

version3: removed bids-apps and stats-models:

image

TODOs

  • [ ] add into figure that schema.json is published to https://jsr.io/@bids/schema and that is what used by deno validator

yarikoptic avatar Feb 21 '25 18:02 yarikoptic

I would remove the legacy validator from that diagram. Will simplify the whole thing.

Remi-Gau avatar Feb 21 '25 18:02 Remi-Gau

The preview on read the docs shows a mermaid syntax error.

This does not mean there is a real syntax error but that's the kind of problem when I was trying to use some of the more advanced mermaid features with mkdocs.

Remi-Gau avatar Feb 21 '25 18:02 Remi-Gau

It's not rendering here: https://bids-website--626.org.readthedocs.build/en/626/standards/index.html

effigies avatar Feb 21 '25 18:02 effigies

  • it is likely the requirement to have newer mkdocs-material ... learning about us using pip-compile... let's see if helps

  • re legacy one: I think it might still be worth being there for a year or so since many could still "relate" to it and here it explicitly depicts it... so "not sure" about removal just yet.

yarikoptic avatar Feb 21 '25 18:02 yarikoptic

FWIW: does render now.

yarikoptic avatar Feb 21 '25 18:02 yarikoptic

Looks like too much stuff going on. I can't read it at 100% zoom.

image image

effigies avatar Feb 21 '25 18:02 effigies

@candleindark -- would you be so kind to see if you could tame the drawing so it is of better fidelity without zooming?

Meanwhile, I have added here use of panzoom plugin (https://playg0n.github.io/mkdocs-panzoom/) which should enable zoom'ing "Mermaid and D2 are included by default". Seems to work nicely.

yarikoptic avatar Feb 24 '25 13:02 yarikoptic

@candleindark -- would you be so kind to see if you could tame the drawing so it is of better fidelity without zooming?

I think the problem stems having too many items in the graph rather the arrangement of the items in the graph, but I will make an attempt. It would be nice if there is some mermaid support that allows zooming in of a particular region of the diagram, like the behavior of the prezi app.

I don't think this problem is particular to us though. Many linkml generated diagrams suffer the same problem in presentation (For example, https://concepts.datalad.org/s/temporal/unreleased/).

candleindark avatar Feb 24 '25 21:02 candleindark

I think if we can get any large mermaid diagram to display in full window as https://anvaka.github.io/panzoom/demo/index.html would be great.

The example is a live demo of https://github.com/anvaka/panzoom which is the base for https://playg0n.github.io/mkdocs-panzoom/

candleindark avatar Feb 25 '25 05:02 candleindark

@candleindark I already added zoom/dragging here -- check preview at https://bids-website--626.org.readthedocs.build/en/626/standards/index.html . Do you propose something instead/on top of it?

yarikoptic avatar Feb 25 '25 16:02 yarikoptic

@candleindark I already added zoom/dragging here -- check preview at https://bids-website--626.org.readthedocs.build/en/626/standards/index.html . Do you propose something instead/on top of it?

In a way, yes. I do think the one I mentioned is more better, but it is not a Mkdocs extension. I don't know if it's worth the time/effort to get it working.

candleindark avatar Feb 25 '25 17:02 candleindark

@candleindark I already added zoom/dragging here -- check preview at https://bids-website--626.org.readthedocs.build/en/626/standards/index.html . Do you propose something instead/on top of it?

In a way, yes. I do think the one I mentioned is more better, but it is not a Mkdocs extension. I don't know if it's worth the time/effort to get it working.

Nevermind, I just found out some a feature of the mkdocs extension you are using. I think it is good enough.

candleindark avatar Feb 25 '25 17:02 candleindark

@yarikoptic I was able to make the items in the diagram appear a bit larger and sent a PR to your branch.

candleindark avatar Feb 25 '25 21:02 candleindark

@effigies please check it out now : https://bids-website--626.org.readthedocs.build/en/626/standards/index.html . I also updated screenshot in original description

yarikoptic avatar Mar 01 '25 14:03 yarikoptic

  • re legacy one: I think it might still be worth being there for a year or so since many could still "relate" to it and here it explicitly depicts it... so "not sure" about removal just yet.

we will forget to remove it in a year

it useless complexifes the figure that is already WAY too complex in my opinion

we have no intention of keeping maintaining the old validator, so let's not emphasize it

the original blog post gradually explained things and complexified the figure incrementally: I think that the landing page for 'standards' should have a simpler version and we can put the more exhaustive version somewhere else

Remi-Gau avatar Mar 01 '25 17:03 Remi-Gau

Just a thought sparked by @Remi-Gau's comment: what if this also was a blog post that came with explanations and is tagged with an author and date?

I haven't reviewed, but I feel like the editorial bar is lower for a blog post than a landing page.

effigies avatar Mar 01 '25 17:03 effigies

Just a thought sparked by @Remi-Gau's comment: what if this also was a blog post that came with explanations and is tagged with an author and date?

I haven't reviewed, but I feel like the editorial bar is lower for a blog post than a landing page.

I like the idea and we can have the simplified version in the landing page and pointing to the blog post for anyone who wants the gory details

Remi-Gau avatar Mar 01 '25 17:03 Remi-Gau

I think it is a good idea -- indeed could be a more descriptive blog post describing the situation, pointing to the blog post with more details on validators of the past. Coolio, we will move into a blog post. Thanks @effigies and @Remi-Gau . I will just move it to draft for now then

yarikoptic avatar Mar 02 '25 04:03 yarikoptic

Idea for a blog is neat but I do not think it is to materialize any time soon. So I am making this PR a candidate for merge - "perfect is enemy of the good" and I had a second look at the diagram and still find it useful to orient visitor in all the intricacies of dependencies, with URL hyperlinks coming quite handy.

yarikoptic avatar May 09 '25 15:05 yarikoptic

I think that if we are to continue with this PR, we better strip away any historical aspects and concentrate on current schema life cycle/use. With that in mind I do not think it would actually be worth a blog post, but rather should be kept current reflection.

yarikoptic avatar Nov 17 '25 23:11 yarikoptic

@candleindark could you provide a quick update to the diagram by stripping off historical aspects?

yarikoptic avatar Nov 17 '25 23:11 yarikoptic

@candleindark could you provide a quick update to the diagram by stripping off historical aspects?

By historical aspect, do you mean taking away the component of legacy-validator @ v1.15.1? Is there anything else?

candleindark avatar Nov 18 '25 00:11 candleindark