opendal icon indicating copy to clipboard operation
opendal copied to clipboard

Published 20 hours ago • verified publisher icon apache

feat(bindings/python): add auto-generated api docs

Open messense opened this issue 1 year ago • 8 comments

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

  • Move to mixed Python/Rust project layout to include type hints for opendal.layers
  • Add sphinx-autoapi based API docs
image

messense avatar Mar 15 '23 10:03 messense

It looks a bit complicated, I think pdoc might be much simpler.

PsiACE avatar Mar 15 '23 11:03 PsiACE

I don't mind switching to pdoc, though it can't document opendal.layers yet:

pdoc opendal/
pdoc server ready at http://localhost:8080
Warn: Error parsing type stubs for opendal:
Traceback (most recent call last):
  File "/Users/messense/.pyenv/versions/3.11.0/envs/test-3.11/lib/python3.11/site-packages/pdoc/doc_pyi.py", line 105, in include_typeinfo_from_stub_files
    imported_stub = _import_stub_file(module.modulename, stub_file)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/messense/.pyenv/versions/3.11.0/envs/test-3.11/lib/python3.11/site-packages/pdoc/doc_pyi.py", line 40, in _import_stub_file
    eval(code, m.__dict__, m.__dict__)
  File "opendal/__init__.pyi", line 19, in <module>
    from .layers import ConcurrentLimitLayer, ImmutableIndexLayer, RetryLayer
ImportError: attempted relative import with no known parent package
 (/Users/messense/.pyenv/versions/3.11.0/envs/test-3.11/lib/python3.11/site-packages/pdoc/doc_pyi.py:107)

messense avatar Mar 15 '23 11:03 messense

And pdoc doesn't pickup docstrings in .pyi files with current setup.

image

messense avatar Mar 15 '23 11:03 messense

And pdoc doesn't pickup docstrings in .pyi files with current setup.

Related to https://github.com/mitmproxy/pdoc/issues/239?

Xuanwo avatar Mar 15 '23 11:03 Xuanwo

And pdoc doesn't pickup docstrings in .pyi files with current setup.

Related to mitmproxy/pdoc#239?

Probably just need to move docstrings to Rust side.

messense avatar Mar 15 '23 11:03 messense

Just an example #1614

PsiACE avatar Mar 15 '23 11:03 PsiACE

I don't mind switching to pdoc, though it can't document opendal.layers yet:

pdoc opendal/
pdoc server ready at http://localhost:8080
Warn: Error parsing type stubs for opendal:
Traceback (most recent call last):
  File "/Users/messense/.pyenv/versions/3.11.0/envs/test-3.11/lib/python3.11/site-packages/pdoc/doc_pyi.py", line 105, in include_typeinfo_from_stub_files
    imported_stub = _import_stub_file(module.modulename, stub_file)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/messense/.pyenv/versions/3.11.0/envs/test-3.11/lib/python3.11/site-packages/pdoc/doc_pyi.py", line 40, in _import_stub_file
    eval(code, m.__dict__, m.__dict__)
  File "opendal/__init__.pyi", line 19, in <module>
    from .layers import ConcurrentLimitLayer, ImmutableIndexLayer, RetryLayer
ImportError: attempted relative import with no known parent package
 (/Users/messense/.pyenv/versions/3.11.0/envs/test-3.11/lib/python3.11/site-packages/pdoc/doc_pyi.py:107)

Can't solve this yet, this boils down to https://github.com/PyO3/pyo3/issues/759 and the workaround doens't work with pdoc.

messense avatar Mar 15 '23 11:03 messense

I am not satisfied with the current state of the Layers API. Therefore, I am fine if we do not document any API related to layers at this time. Maybe we can setup the Operator related API first?

Xuanwo avatar Mar 17 '23 10:03 Xuanwo

We can update here together.

https://github.com/apache/incubator-opendal/blob/27abd5123c14fbc6eb3438197183640744ac47b3/website/docusaurus.config.js#L92-L96

suyanhanx avatar Mar 18 '23 02:03 suyanhanx

Hello, @messense. I apologize for the broken CI. It has been fixed now. Kindly update your branch and try again.

Xuanwo avatar Mar 18 '23 05:03 Xuanwo