DISC: Reorganising Options that are specific to pipeline stages

[mmcky]

trafficstars

It might be better to reorganise the options available for this extension into the different stages of how the extension works, in addition to global options for writing the notebooks. The options that only affect parts of the pipeline could be captured in a grouped dictionary format.

An example would be in the conf.py

jupyter_execution = {
    execute_notebooks : True,
    number_workers : 4,
    threads_per_worker : 1,
}

jupyter_html = {
    make_site : True,
    nbconvert_template : "<path>",
    download_nb : True,
    download_nb_urlpath : "<path>",
}

this will help group the options logically together and will make it much clearer which options affect what parts of the pipeline.

[mmcky]

@AakashGfude we should run through the entire list of options and review them to see if they should act globally or be inside one of the pipeline specific configuration dictionaries.

[mmcky]

The following is a list of options based on where they should be in the respective stages of the extension. Before we reorganise into these groups for greater clarity we will need to check if we can switch flags from the command line if we adopt these dict type configuration options.

• [ ] can we specify jupyter_html options from the cmd line if we cluster them into groups (see discussion below)

Global (IPYNB Production)

1. jupyter_kernels (@AakashGfude is this required)
2. jupyter_conversion_mode (update default value to "all")
3. jupyter_write_metadata
4. jupyter_static_file_path (@AakashGfude should this be a list or a string structure?)
5. jupyter_header_block
6. jupyter_options (@AakashGfude can you confirm if this is still used?)
7. jupyter_default_lang
8. jupyter_lang_synonyms
9. jupyter_drop_solutions
10. jupyter_drop_tests
11. jupyter_target_html
12. jupyter_images_markdown

Execution Pipeline

1. jupyter_execute_notebooks
2. jupyter_threads_per_worker
3. jupyter_number_workers
4. jupyter_ignore_no_execute
5. jupyter_ignore_skip_test
6. jupyter_dependency_lists (@AakashGfude it may be easier to specify this as a global option given its structure?)

would become:

jupyter_execute = {
     number_workers : 1,
     threads_per_worker : 1,
     ignore_no_execute : False,
     ignore_skip_test : False,
}

otherwise jupyter_execute = None. jupyter_execute_notebooks would be redundant if this settings is not None.

HTML Pipeline

1. jupyter_target_html = True by default when using make_site so shouldn't need to be specified
2. jupyter_generate_html and jupyter_make_site should be combined
3. jupyter_html_template
4. jupyter_make_site
5. jupyter_download_nb
6. jupyter_download_nb_urlpath
7. jupyter_images_urlpath (@AakashGfude should this be a download_nb only option?)

would become:

jupyter_html = {
     jupyter_target_html : True, #set the global option to True always
     nbconvert_template: "", #Path to tpl file for nbconvert
     download_nb : True,    #generate special download notebook set
     download_nb_urlpath:  "", 
     download_nb_assetpath : "", #allow for online assets for images and figures
}

jupyter_make_site and jupyter_generate_html would be redundnant if jupyter_html is not None

Coverage Pipeline

1. jupyter_make_coverage
2. jupyter_template_coverage_file_path

would become:

jupyter_coverage = {
    template_coverage_file_path : ""
}

`jupyter_make_coverage` would be redundant if `jupyter_coverage` is not `None`

[mmcky]

@AakashGfude if we setup these structures then I think the best way forward to implement the UX for what we need for our local operation is to setup our own jupyter command that overrides and switches the execution, html, and coverage off.

jupyter:
    sphinx-build <stuff> -D jupyter_execution=None -D jupyter_html=None

and then make website would just be the typical make jupyter call that respects the conf.py file.

can we pass through None via the cmd. I know 0=False, and 1=True but I wasn't sure how the cmd translates to None.

[AakashGfude]

@mmcky in the present setup, we are handling 0 as None as well, so as to allow for cmd options. It will work fine with 0 and 1.