Ideas for improving the new documentation

[MarcSkovMadsen]

The upcoming revised documentation is a great improvement. I've already learned a lot and believe I understand Param and the ideas behind it much better. It's great.

The below is a list of ideas that I believe could improve the documentation for both new and experienced users.

1. Make it even more visible WHY Param would be valuable to "me".

The documentation is text heavy. I think that is great as soon as my attention has been caught and I understand why I should take the time to read and learn. The text is great.

I need something to catch my attention though. And explain it to people with other learning preferences than text.

My suggestion would be to add a 3 min video to the Introduction that explains the why with a real world, practical example. It should be a walk through of a very interactive Notebook that can be downloaded by the user right next to the video. And add in a link to Binder where I can try it out as well.

It might also be valuable to add a 30 sec .gif video that just plays and immediately explains the Why to me and catches my attention.

2. Make Getting Started Example even more friendly for users.

The first example I see looks like this.

import param
        
class A(param.Parameterized):
    title = param.String(default="sum", doc="Title for the result")
    
class B(A):
    a = param.Integer(2, bounds=(0,10), doc="First addend")
    b = param.Integer(3, bounds=(0,10), doc="Second addend")
    
    def __call__(self):
        return self.title + ": " + str(self.a + self.b)

>> o1 = B(b=4, title="Sum")
>> o1.a = 5
>> o1()
'Sum: 9'

The main question for me is whether using __call__ in the example is a good idea. I know about __call__ but I never see any colleagues or blogs in data, engineering, trading or science use it.

So I believe the big questions for users is: What is __call__. Why do you use __call__. I did not know I needed __call__ in my code. What is the use case for __call__? Why is __call__ useful to me?

So unless there is a point (which a have not yet understood) in using __call__ I would rename the function to sum.

You are also using multiple inheritance which might also throw some users off. I just see my new colleagues coming out of engineering and finance education struggling with the concept of Classes.

3. Add Pydantic to the Comparison

Pydantic is taking the Python world with storm. Together with FastApi. I would like to know why it's valuable to me to build domain models with Param when

1. I can use Pydantic and not Param with FastAPI.
2. Pydantic works using type annotations which provides a much better development experience in an IDE and with linters than Param.
3. The huge popularity of Pydantic means more colleagues would probably know and understand why I use Pydantic
4. There are (more) tools out there for Pydantic like PyCharm, MyPy, Hypothesis plugins. And more are probably coming.
5. Lots of orgs are building on Pydantic and want to be associated with it. See https://github.com/Kludex/awesome-pydantic

I believe Param and Pydantic could be integrated and that would help a lot of Pydantic users easily create visualizations, tools, apps and dashboards from their models. And Param users create modern Rest Apis via Fast Api. For a Streamlit+Fast API example check out https://testdriven.io/blog/fastapi-streamlit/.

4. Add a real world example

Add a section with a real world example. My personal profile would simply need that. You can write just as many theoretical arguments about why Param is great (which it is) but without a real, practical example you don't catch my attention. I move on.

I work with traders so in my world the Black Scholes option pricing model is a great example.

I would visualize it using Panel.

There are probably more generally understandable examples out there but this one I use at work to explain why Param is powerful.

FYI. @jbednar

[jbednar]

1. Make it even more visible WHY Param would be valuable to "me". My suggestion would be to add a 3 min video to the Introduction that explains the why with a real world, practical example. It should be a walk through of a very interactive Notebook that can be downloaded by the user right next to the video. And add in a link to Binder where I can try it out as well.

Sounds like a great idea!

However, what do you mean by "interactive"? If you mean Panel, I think that may muddy the waters a bit; I think it's important for people to understand that Param is important in its own right, whether or not you ever make a GUI with Panel. So I think any first example should not use Panel.

2. Make Getting Started Example even more friendly for users. def __call__(self): The main question for me is whether using call in the example is a good idea. I know about call but I never see any colleagues or blogs in data, engineering, trading or science use it. So I believe the big questions for users is: What is call. Why do you use call. I did not know I needed call in my code. What is the use case for call? Why is call useful to me?

Hmm; good point; probably assuming people know too much about Python classes. Personally, I believe that if a Python class does only one thing, as this one does, it's simply good practice to name that thing __call__, instead of coming up with some other arbitrary method name for people to learn and keep track of. I.e., __call__ just means "do the thing this class does, whatever that is". If all other things are equal, I believe examples should demonstrate good practice by default, to maximize the exposure that people get to good patterns rather than bad patterns.

Still, it's also important not to derail people from their current objective by confusing them, particularly not in introductory examples; it's not our job to train people in Python language feature usage! So I'd be happy to accept a change to rename that to sum, to avoid putting off people without that background.

You are also using multiple inheritance which might also throw some users off.

This example is not using multiple inheritance. Each class inherits from only a single base class, i.e. B(A) (B inherits from A) where A(param.Parameterized) (A inherits from Parameterized).

But sure, you're asking whether it's important to show such an inheritance hierarchy here. I do think inheritance hierarchies like that are an extremely important aspect of Param, and I do want all Param users to be aware that Param is particularly well suited to such hierarchies, allowing you to avoid duplication along the inheritance chain (particularly for docstrings). Without Param, tall inheritance chains almost never bother to document individual parameters well, because that same bit of docs would have to be duplicated at every parent and child class in the chain, because Python inherits methods and attributes across the hierarchy, but not attribute docs. If we don't have such a chain in the examples, we can't show one of the main reasons to use Param, i.e. that it lets you organize your code well and without duplication. Without such inheritance, Python users have a disincentive to design proper inheritance hierarchies; Param fixes that!

That said, it doesn't necessarily need to be the first example! :-)

3. Add Pydantic to the Comparison. Pydantic is taking the Python world with storm. Together with FastApi. I would like to know why it's valuable to me to build domain models with Param when

Great idea; can you please contribute a section covering Pydantic to the Comparison? I don't use Pydantic myself, and from a relatively brief study of the docs I don't see what it would offer me that I don't already have with Param. It would be great to have a more useful perspective from someone who has used both!

I believe Param and Pydantic could be integrated and that would help a lot of Pydantic users easily create visualizations, tools, apps and dashboards from their models. And Param users create modern Rest Apis via Fast Api. For a Streamlit+Fast API example check out https://testdriven.io/blog/fastapi-streamlit/.

I can see how Pydantic could be supported by Panel. I.e., just as Panel already does for Param, Panel could accept a Pydantic BaseModel-derived class and read the metadata it needs to synthesize widgets, etc. from it. I'm not quite seeing how Param and Pydantic could interact, though; can you explain? Seems to me like it would be confusing to mix Param and Pydantic in a particular class, even if you prefer Pydantic's syntax for specifying types.

4. Add a real world example. Add a section with a real world example. My personal profile would simply need that. I work with traders so in my world the Black Scholes option pricing model is a great example.

Maybe we'd want two very different real-world examples? Black Scholes is fine as one of them, as long as it's a simple Param class that works well with Panel.

To-do items:

• [x] Make a 2-5 min video that explains the "why" with a real world, practical example. It should be a walk through of a very interactive Notebook that can be downloaded by the user right next to the video, plus a link to Binder for trying it out.
• [x] Link to the video from the introduction and probably home page.
• [ ] Make a 30-sec .gif video that just plays and immediately explains the Why to me and catches my attention.
• [ ] Link to the gif from the introduction and probably home page.
• [ ] Change __call__ to sum
• [ ] Consider whether to put a first example with only a single level of inheritance
• [ ] Add pydantic to Comparisons
• [ ] Consider adding Pydantic(PaneBase) to Panel, analogous to Param(PaneBase).

[MarcSkovMadsen]

Here is a video illustrating the concept.

https://user-images.githubusercontent.com/42288570/122000237-c5b66280-cdae-11eb-8eee-a3d33c1d0b53.mp4

[MarcSkovMadsen]

and here is a faster version

https://user-images.githubusercontent.com/42288570/122000307-e383c780-cdae-11eb-9751-1f152243b0e0.mp4

[MarcSkovMadsen]

I will make a PR with the notebook. But please consider giving me right to push to the repository like I have for Panel @jbednar .

[ea42gh]

I'd be happy to edit such a video!

The way I see it, this would involve

1. creating the notebook I advocate using folding for each of the sections: this would require each of the section/subsection titles be in their own markdown cells
2. multiple recordings of somebody explaining the notebook. Here, audio quality is the most important consideration.
3. I could then take these multiple takes, and edit them to create a video.

How would we get this organized?

[jbednar]

Cool! Yes, please do submit the notebook, e.g. as examples/intro.ipynb or examples/promo.ipynb and we can edit that until we're happy with it. Then we can record something with audio and go from there (and address the other to-do items here).

[MarcSkovMadsen]

I will also submit the Notebook ASAP and we can iterate on it. But maybe it would be an idea to write down and formulate the purpose and target audience of the material. That could guide us when discussing content, layout, format etc.

For inspiration on what @ea42gh can do with video check out this video that @ea42gh produced for panel-chemistry https://drive.protonmail.com/urls/0XQCDYYY84#Zd1ZRPWvIg2X

[MarcSkovMadsen]

Started a Promo Notebook PR here https://github.com/holoviz/param/pull/487. Feel free to comment or contribute.

[ea42gh]

I have some suggestions for the documentation I'd like to make since we want to produce videos.

E.g., using <br>$\quad$ to reduce line lengths, adding additional subsections to create more of a framework for the audio, changing font sizes.

What is a good way to cooperate on this? PR on top of PR seems a bit clumsy!

[philippjfr]

@ea42gh Happy to make you an external collaborator so you can push to @MarcSkovMadsen's PR.

[ea42gh]

@phillipjfr How does that work? Can you point me to some explanation I could read? Does this cover it? https://medium.com/google-cloud/pull-push-to-someone-elses-upstream-github-pr-6073ae5005e7

[MarcSkovMadsen]

I am perfectly confident you doing an iteration on the "draft version" I made.

For me the purpose is to make sure

1. Not so strong Pythonistas understand if Param can be useful for them and is worth their effort to learn.
2. Strong Pythonistas immediately understand what Param is about and understanding enough to actually start using it.

So anything that can make the notebook communicate better is key.

[philippjfr]

@ea42gh Now that I've added you (and once you accept), you will be able to clone the repo, check out Marc's branch (once he's pushed it to the param repo), and push directly to that.

[ea42gh]

@jbednar re audio/video, some thought should be given to consistent branding:

1. intro and outro visuals and possibly background music
2. color theme for annotations (bubbles, arrows, etc; design should yield good readability for text over background)
3. a consistent set of annotations and transitions

What are your thoughts?

[MarcSkovMadsen]

Opened new and revised PR here https://github.com/holoviz/param/pull/488

[ea42gh]

I pushed a new version of Promo.ipynb to the param-promo branch and forgot to clear the outputs first :(

The idea is to have all sections folded prior to recording a video, then open each section/subsection in turn and execute code, if any.

[ea42gh]

The panel_chemistry video @MarcSkovMadsen mentioned above has been replaced with a newer version: https://drive.protonmail.com/urls/E7GSH67AX8#AXEjIroNYA6i

[ea42gh]

To push this forward, I created a video from the Promo notebook. Everybody interested (@philippjfr @jbednar @MarcSkovMadsen ?) please comment!

https://drive.protonmail.com/urls/K66V127PX8#RacCEK6TjoXa

Some of my own notes:

• audio quality is extremely important: I did not get clean audio when I recorded scenes
• The timing is off in places. Easy to fix, but I did not spend time on a placeholder
• I did not use any highlighting, zooming, panning for the same reason... note that for the highlighting to work well, a light background is preferable!
• I noticed that the parameter documentation strings and compute_hypothenuse should be corrected and anglicized...

[jbednar]

Thanks! I'm on spotty internet until Wednesday, but will take a look then.

[ea42gh]

Hope that means a great vacation!!

[rtbs-dev]

Just finding this by searching "pydantic" on the param issues list! Glad I'm not the only one!

I can see how Pydantic could be supported by Panel. I.e., just as Panel already does for Param, Panel could accept a Pydantic BaseModel-derived class and read the metadata it needs to synthesize widgets, etc. from it. I'm not quite seeing how Param and Pydantic could interact, though; can you explain? Seems to me like it would be confusing to mix Param and Pydantic in a particular class, even if you prefer Pydantic's syntax for specifying types.

Definitely see this, though it comes down to what kind of "support" you guys want to be thinking about. It would 100% make me happy to have a Pydantic(PaneBase). Though this might require more maintenance on your end? Two separate interfaces? But I suppose the primary alternative is to make a (round-trip-able?) conversion between param and pydantic, so that pydantic models can be used to generate Param's incrementally, as the feature gets supported?

Either way I have a bunch of pydantic models that I'd be willing to test out with panel when the time comes. This would let procedural UI creation from e.g. JsonSchema's be way easier!

Also probably worth checking out Pandera, which may be even closer to the way pandas get's used in Holoviews, but is increasingly interoperable with Pydantic. Hypothesis property testing for my dashboards, anyone? :smile:

[jbednar]

Thanks, @tbsexton , for making the possible Pydantic link more concrete. If indeed what you'd like is something acting like Pydantic(PaneBase) for Panel, I agree that at that point it's an open question whether to achieve that by directly implementing Pydantic(PaneBase) analogously to Param(PaneBase) (in Panel), or whether either a one-way or two-way converter between pydantic.BaseModel and param.Parameterized would be easier or work better (implemented using Param mechanisms alone).

I.e., whether or not Param itself makes sense to use with Pydantic, it does make sense to consider whether we could use mechanisms at the Param level to make Pydantic be well supported at the Panel level. (Was there some reason these all had to start with "P"??) I think this is concrete enough for me to discuss with @philippjfr and see if there's an obvious way forward for working with Pydantic in Panel. But I do think he and I already have our plates quite full for the next month at least, so if anyone else wants to try making some sort of wrapper, that's probably the only way it would move forward in the immediate future.

[jbednar]

@ea42gh , thanks so much for the video! I made changes to the notebook in https://github.com/holoviz/param/pull/488 , but I'll list comments specific to the video here. Basically, it looks great to me! Some minor notes:

1. At 8 minutes, I think it's a bit long for such a video. I think we have about 5 minutes of the user's attention, at most. Maybe cut the explicit event watching?
2. The music helps make it seem more professional, but on balance I think I'd rather just launch right into it, with no intro or outro music. This is some hands-on stuff, and I think people will want to jump right in!
3. In one bit you say "We have to initialize our class, of course", but I'd be careful there; it's true that the class has to be initialized, but it's only rarely required for a user to supply an explicit constructor; the default constructor that processes keyword arguments is very often the only constructor needed. Obviously we can't get into all that here, and I don't see a way to avoid having the explicit constructor in this case, so maybe just don't say "of course". :-)

Other than that, if people are happy with the notebook, I'm happy for you to record a slightly updated version and we can use that! Thanks so much.

[ea42gh]

@jbednar do you really want me to voice the video? I did not contribute to param, and I do have an accent!

I have always had trouble with how to start: @MarcSkovMadsen's "Hi there!" seems to work?

I think future videos should have a consistent look and feel, so please feel very free commenting and requesting changes. And yes, it is trivial to cut out words and sentences, and replace the visuals....

Unless I hear from you, I'll record a new version for further comments.

[jbednar]

I don't mind the accent, and this is a contribution to Param! :-)

"Hi there!" is certainly fine. I think future videos can have a variety of authors and speakers, to show the diversity and widespread applicability of Param!

[ea42gh]

@jbednar @MarcSkovMadsen I put up two new versions of the promo video:

1. the original file: https://drive.protonmail.com/urls/SAVKC3CWP0.#2foouSqCo1qY
2. the file with highlighting zoom and pan. https://drive.protonmail.com/urls/F3SK7AEPG4#CJ5JZ6fNo8Ds

please give me comments, change requests, etc..

[MarcSkovMadsen]

Hi @ea42gh . I took a look at video 2. I think its close to perfect. The two comments I have are

1. Personally I miss the intro music. I think it sets a nice inviting and relaxing scene for me to listen and learn. And just gives the video that extra professionalism other videos do not have. Without the music the video gets much more serious over fun.
2. I hear one or more birds in the background by the end of the video. Maybe they should be removed?

[ea42gh]

I Hi @MarcSkovMadsen

• Re 1: you and @jbednar are on opposite sides for this one! Personally, I like an intro just short enough to read a title for a series of videos: a unified look and feel. (Hint, hint, shall we do more videos?)
• Re 2: ouch, dubbing in audio is difficult: without a professional sound studio, the audio recordings never sound the same. I'll go see if I have some extra material to dub in! Just when are these bird sounds? I just listened to the video 3 times, I guess I get distracted by my voice...

[MarcSkovMadsen]

The bird is there in file https://drive.protonmail.com/urls/F3SK7AEPG4#CJ5JZ6fNo8Ds around 4:14.

I don't think it's a big problem.

[ea42gh]

@MarcSkovMadsen I had audio I could use: this version does not have a bird around 4:14 https://drive.protonmail.com/urls/SV5465BPMC#Ye4coiEhfEI7