Adding Sphnix Based Documentation

[amjadraza]

🚀 The feature

In order to readthrough the code and method implemented, I have to go through all code. Sphinx based Documentation can be added and hosted on at readthedocs server.

Motivation, pitch

Easy to read through the code and its feature. It will also help to see the transparency in code and how sensitive information is blocked sending to LLM model

Alternatives

No response

Additional context

No response

[amjadraza]

Happy to contribute if it is approved.

[yzaparto]

Hey, @amjadraza sounds like a great idea. It would be cool to see the pull request. @gventuri thoughts?

[gventuri]

@amjadraza sounds great, this is definitely becoming a priority!

Go for it 🚀

[amjadraza]

@gventuri @yzaparto thanks for the feedback. Let us get started

First we have to choose the template to use for the documentation. Below are two options, I recommend due widely used

1. Documentation using sphinx_rtd_theme : Very popular, Libraries like Databricks , Conda and Manay more
2. Documentation using pydata_sphinx_theme : Popular among DS libraries such as Pandas and Numpy

Due to the nature of work and relevance, I would suggest to use pydata_sphinx_theme. You guys can see the sample documentation on below link to further vote.

https://www.sphinx-doc.org/en/master/examples.html#documentation-using-pydata-sphinx-theme

Thanks

[TiansuYu]

I have experience with setting up sphinx and readthedoc before as well. Please ping me if you need more hands on it.

[HenriqueAJNB]

Hi, let me suggest another framework. Sphynx is an over-powered documentation framework based on reStructuredText, a more complex syntax than markdown.

If the idea is to implement the readthedocs template, using mkdocs it's a more straightforward way to achieve that, with full support to markdown without any extra extensions.

I'd like to hear other opinions, so we can discuss about it.

[gventuri]

Markdown could be easier to maintain and more straightforward. Overall I don't have a strong opinion on this. The more straightforward, the better.

Let's wait for further opinions!

[amjadraza]

Sphynx also supports mkdocs and Markdowns. @HenriqueAJNB please share any example that you are referring too. It is always better to have simple and easy-to-handle frameworks.

Thanks

[TiansuYu]

I am also lean more toward @HenriqueAJNB's suggestion: sphinx can bring a lot of overhead. The markdown support of sphinx is not a "baked-in" feature in my opinion. Markdown files has to be parsed before made availible to display. Depends on what markdown syntax flavor you are using, this can be very error-prone. Also note that Google style docstring does not always work on sphinx API doc generation. In terms of external links, and other advanced features, you still have to write it in sphinx syntax instead of markdown.

[amjadraza]

So far, we have received feedback to use the mkdocs framework due to its simplicity and all that.

Let us go ahead then @HenriqueAJNB @TiansuYu I might need your help if stuck somewhere.

Below are the proposed features for documentation

1. mkdocs Framework
2. readthedocs theme
3. Google Style docstrings
4. Hosted on readthedocs server

Any other comments, happy to address.

Thanks

[HenriqueAJNB]

@amjadraza, go on, you'll have my support.

[amjadraza]

@HenriqueAJNB @gventuri

I have completed the initial setup and you can view the results here. Can someone Try this out?

https://pandasai.readthedocs.io/en/latest/

https://github.com/amjadraza/pandas-ai

Any comments on Setup? The followings are TODO tasks

• [ ] Writing the Docstrings with more explanation
• [x] Writing examples as part of the Documentation
• [ ] Writing some details on how to Run Autotests
• [x] Some other refinements/explanations are needed.
• [x] A guide on how to write documentation in CONTRIBUTING Guide.
• [x] Docs generation should be part of release process

After we merge with the main repo, I will remove the test deployment of the documentation, we build from the released branch.

Thanks

[HenriqueAJNB]

That's nice. It's possible to add samples on how does function works at docstring in order to use doctests later.

I agree with the tasks!

[gventuri]

@amjadraza also please make sure not to edit all the files when submitting your PR!

[amjadraza]

@gventuri In order to add docstrings in files, files will be modified. How about, I create a sample docstring for one file and after mutual agreements, other files will be modified accordingly. But, I will make sure not to touch any code and it will pass all tests, etc. The first MR will be a bit of work but after than it will be ok.

[gventuri]

@amjadraza, sure, let's try with a sample! :)

[amjadraza]

@gventuri below is the link to example of Docstring using Google Style.

https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html

[amjadraza]

https://pandasai.readthedocs.io/en/feature-docs/

@gventuri @HenriqueAJNB

In my above TODO list, two tasks are yet to be done. After we decide on Docstring refactoring using Google Style, I will submit a PR.

Thanks