celeborn icon indicating copy to clipboard operation
celeborn copied to clipboard

[MINOR] add makefile to do docs gen

Open pingzh opened this issue 4 months ago • 1 comments

What changes were proposed in this pull request?

create a makefile to generate docs and later more targets can be added

Why are the changes needed?

inside the makefile, it uses a python docker container to build / serve docs, so that contributors dont need to install a specific python version, venv etc. (e.g. right now the requirements.txt does not support python 3.12)

Does this PR resolve a correctness bug?

No

Does this PR introduce any user-facing change?

No

How was this patch tested?

locally:

❯ make
Usage: make <target>

Targets:
  docs-clean    Remove the generated site directory
  docs-serve    Run mkdocs serve inside Docker and expose DOCS_PORT
  docs          Build the MkDocs site inside a Python 3.11 container
  help          Show help for available make targets
❯ make docs
INFO     -  DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  File "/usr/local/lib/python3.11/site-packages/mkdocs_macros/context.py", line 19, in <module>
    import pkg_resources
  File "/usr/local/lib/python3.11/site-packages/pkg_resources/__init__.py", line 98, in <module>
    warnings.warn(

INFO     -  [macros] - Macros arguments: {'module_name': 'main', 'modules': [], 'include_dir': '', 'include_yaml': [], 'j2_block_start_string': '', 'j2_block_end_string': '', 'j2_variable_start_string': '', 'j2_variable_end_string': '', 'on_undefined': 'keep', 'on_error_fail': False, 'verbose': False}
INFO     -  [macros] - Extra variables (config file): ['social']
INFO     -  [macros] - Extra filters (module): ['pretty']
INFO     -  Cleaning site directory
INFO     -  Building documentation to directory: /workspace/site
INFO     -  The following pages exist in the docs directory, but are not included in the "nav" configuration:
  - configuration/client.md
  - configuration/columnar-shuffle.md
  - configuration/ha.md
  - configuration/master.md
  - configuration/metrics.md
  - configuration/network-module.md
  - configuration/network.md
  - configuration/quota.md
  - configuration/worker.md
  - developers/release.md
WARNING  -  Documentation file 'developers/faulttolerant.md' contains a link to '../assets/img/revive.svg' which is not found in the documentation files.
WARNING  -  Documentation file 'developers/faulttolerant.md' contains a link to '../assets/img/batchrevive.svg' which is not found in the documentation files.
WARNING  -  Documentation file 'developers/overview.md' contains a link to '../assets/img/ess.svg' which is not found in the documentation files.
WARNING  -  Documentation file 'developers/overview.md' contains a link to '../assets/img/celeborn.svg' which is not found in the documentation files.
WARNING  -  Documentation file 'developers/shuffleclient.md' contains a link to '../assets/img/softsplit.svg' which is not found in the documentation files.
WARNING  -  Documentation file 'developers/storage.md' contains a link to '../assets/img/reducepartition.svg' which is not found in the documentation files.
WARNING  -  Documentation file 'developers/storage.md' contains a link to '../assets/img/mappartition.svg' which is not found in the documentation files.
WARNING  -  Documentation file 'developers/storage.md' contains a link to '../assets/img/multilayer.svg' which is not found in the documentation files.
WARNING  -  Documentation file 'developers/trafficcontrol.md' contains a link to '../assets/img/backpressure.svg' which is not found in the documentation files.
INFO     -  Documentation built in 1.33 seconds

~/code/apache-celeborn dev/pingz-makefile*                                                                                                                       5s apache-celeborn 14:52:43

make docs-serve locally image

pingzh avatar Nov 13 '25 22:11 pingzh

hi @SteNicholas @turboFei I created a make docs to easily generate the docs. could you please take a look at this?

pingzh avatar Nov 13 '25 22:11 pingzh