Doc automation broken in CI

[msaroufim]

🐛 Describe the bug

Our latest doc build is crashing https://github.com/pytorch/serve/runs/7361642275?check_suite_focus=true with this error ModuleNotFoundError: No module named 'ts_scripts'

@svekars does this look familiar?

Error logs

Run cd ./docs/sphinx ; export PATH=$PATH:$(python3 -m site --user-base)/bin/ ; make html
Running Sphinx v5.0.2
WARNING: Invalid configuration value found: 'language = None'. Update your configuration to a valid langauge code. Falling back to 'en' (English).
making output directory... done
WARNING: html_static_path entry '_static' does not exist
WARNING: logo file '_static/img/pytorch-logo-dark.svg' does not exist
[autosummary] generating autosummary for: FAQs.md, README.md, Troubleshooting.md, api/dev_api.rst, api/modules.rst, api/run_circleci_tests.rst, api/setup.rst, api/test.rst, api/torchserve_sanity.rst, api/ts.metrics.rst, ..., rest_api.md, server.md, snapshot.md, sphinx/requirements.txt, torchserve_on_win_native.md, torchserve_on_wsl.md, use_cases.md, workflow_inference_api.md, workflow_management_api.md, workflows.md
Failed to import ts_scripts.api_utils.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.backend_utils.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.frontend_utils.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.install_dependencies.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.install_from_src.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.marsgen.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.modelarchiver_utils.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.print_env_info.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.regression_utils.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.sanity_utils.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.shell_utils.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.torchserve_grpc_client.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.tsutils.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.utils.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.validate_model_on_gpu.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.workflow_archiver_utils.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import ts_scripts.
Possible hints:
* KeyError: 'ts_scripts'
* ModuleNotFoundError: No module named 'ts_scripts'
Failed to import torchserve_sanity.
Possible hints:
* KeyError: 'torchserve_sanity'
* ModuleNotFoundError: No module named 'torchserve_sanity'
Failed to import ts.torch_handler.unit_tests.test_base_handler.
Possible hints:
* AttributeError: module 'ts.torch_handler.unit_tests' has no attribute 'test_base_handler'
* ModuleNotFoundError: No module named 'pytest'
Failed to import ts.torch_handler.unit_tests.test_envelopes.
Possible hints:
* ModuleNotFoundError: No module named 'pytest'
* AttributeError: module 'ts.torch_handler.unit_tests' has no attribute 'test_envelopes'
Failed to import ts.torch_handler.unit_tests.test_image_classifier.
Possible hints:
* AttributeError: module 'ts.torch_handler.unit_tests' has no attribute 'test_image_classifier'
* ModuleNotFoundError: No module named 'pytest'
Failed to import ts.torch_handler.unit_tests.test_image_segmenter.
Possible hints:
* ModuleNotFoundError: No module named 'pytest'
* AttributeError: module 'ts.torch_handler.unit_tests' has no attribute 'test_image_segmenter'
Failed to import ts.torch_handler.unit_tests.test_mnist_kf.
Possible hints:
* AttributeError: module 'ts.torch_handler.unit_tests' has no attribute 'test_mnist_kf'
* ModuleNotFoundError: No module named 'pytest'
Failed to import ts.torch_handler.unit_tests.test_object_detector.
Possible hints:
* AttributeError: module 'ts.torch_handler.unit_tests' has no attribute 'test_object_detector'
* ModuleNotFoundError: No module named 'pytest'
Failed to import ts.torch_handler.request_envelope.kservev2.
Possible hints:
* AttributeError: module 'ts.torch_handler.request_envelope' has no attribute 'kservev2'
* ModuleNotFoundError: No module named 'numpy'
Failed to import test.regression_tests.
Possible hints:
* AttributeError: module 'test' has no attribute 'regression_tests'
* ModuleNotFoundError: No module named 'test.regression_tests'
Failed to import setup.
Possible hints:
* SystemExit: usage: sphinx-build [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: sphinx-build --help [cmd1 cmd2 ...]
   or: sphinx-build --help-commands
   or: sphinx-build cmd --help
error: option -M not recognized
* KeyError: 'setup'
Failed to import run_circleci_tests.
Possible hints:
* KeyError: 'run_circleci_tests'
* ModuleNotFoundError: No module named 'run_circleci_tests'
Failed to import ts.torch_handler.image_classifier.
Possible hints:
* AttributeError: module 'ts.torch_handler' has no attribute 'image_classifier'
* ModuleNotFoundError: No module named 'torchvision'
Failed to import ts.torch_handler.image_segmenter.
Possible hints:
* AttributeError: module 'ts.torch_handler' has no attribute 'image_segmenter'
* ModuleNotFoundError: No module named 'torchvision'
Failed to import ts.torch_handler.object_detector.
Possible hints:
* AttributeError: module 'ts.torch_handler' has no attribute 'object_detector'
* ModuleNotFoundError: No module named 'torchvision'
Failed to import ts.torch_handler.text_classifier.
Possible hints:
* AttributeError: module 'ts.torch_handler' has no attribute 'text_classifier'
* ModuleNotFoundError: No module named 'torchtext'
Failed to import ts.torch_handler.text_handler.
Possible hints:
* ModuleNotFoundError: No module named 'torchtext'
* AttributeError: module 'ts.torch_handler' has no attribute 'text_handler'
Failed to import ts.torch_handler.vision_handler.
Possible hints:
* AttributeError: module 'ts.torch_handler' has no attribute 'vision_handler'
* ModuleNotFoundError: No module named 'captum'
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
loading intersphinx inventory from https://docs.scipy.org/doc/numpy/objects.inv...
loading intersphinx inventory from https://pytorch.org/docs/stable/objects.inv...
intersphinx inventory has moved: https://docs.scipy.org/doc/numpy/objects.inv -> https://numpy.org/doc/stable/objects.inv
building [mo]: targets for 0 po files that are out of date
building [html]: targets for [4](https://github.com/pytorch/serve/runs/7361642275?check_suite_focus=true#step:4:5)9 source files that are out of date
updating environment: [new config] 49 added, 0 changed, 0 removed
reading sources... [  2%] FAQs
/usr/lib/python3.8/distutils/dist.py:274: UserWarning: Unknown distribution option: 'docs_url'
  warnings.warn(msg)
Extension error (sphinx_markdown_tables):
Handler <function process_tables at 0x7f8a0b9f2e[5](https://github.com/pytorch/serve/runs/7361642275?check_suite_focus=true#step:4:6)0> for event 'source-read' threw an exception (exception: __init__() missing 1 required positional argument: 'config')
make: *** [Makefile:2[8](https://github.com/pytorch/serve/runs/7361642275?check_suite_focus=true#step:4:9): html] Error 2
Error: Process completed with exit code 2.

Installation instructions

N/A

Model Packaing

N/A

config.properties

No response

Versions

N/A

Repro instructions

Run CI

Possible Solution

No response

[msaroufim]

This did not work - tagging @svekars who I'm planning on this discussing this more

EDIT: Problem seems to be our requirements.txt in docs is not actually used in the Github Action for doc build or anywhere for that matter https://github.com/pytorch/serve/blob/master/docs/sphinx/requirements.txt - will fix this and delete dead files . All our doc scripts are due for a cleanup.