Integration with breathe, various improvements, regexes, external linking

[rogerbarton]

Hi! I've been trying to get this to work with the sphinx breathe extension to document C# code with doxygen. This requires a working C# domain... I've had to make several changes and improvements, but its working quite nicely, have a look here. I use it for a C#/C++ Unity project.

1. Resolving references has improved
   • ignored built-in types and other strings, see IGNORE_XREF_TYPES
   • resolving to 'closest match' if the target is not inside any of the direct parent namespaces (this can still be improved), see resolve_xref
   • Searching for external links (previously just for MSDN), see below
2. External links can be added more easily
   • Direct API link is generated and checked if valid
   • If it is invalid a fallback link is used, usually a search
   • As a user, you need to specify the api link and a fallback link with %s for where to place the name
3. Improved parsing of signatures
   • more complicated types possible
   • Named regex groups for parsing (just makes it easier to extract the results)
4. Support for: events, interfaces, structs, proper variable parsing
5. Improved some of the rendering/visuals
   • use more of the sphinx addnodes over docutils nodes, also fixes some spacing issues
   • added nbsp for strings consisting of just spaces caf64fd4b95d30842933f64f22a4c2990e6955dc
6. Soft failure if a signature does not match its regex ac522b1e10a23277fde4b9aed9dc3c8c5308ee38
   • the string is just rendered as is
7. Important:
   • changed domain name to 'cs' from 'csharp' 2e8e21968f7fe58ac942241139d6b1c5b358c7ed
   • moved setup function to __init__ so we can import the extension with just sphinx_csharp instead of sphinx_csharp.csharp (its based on how breathe does it, not sure how good this is) 9cd500bcf1c886702a2d4b7f9594bc37009559ba
8. Use logger from sphinx to print info/warnings (found some deprecations with the current way of doing it) 9e7edbfcc5b79c99e861b464a9495e3596113dec
9. Test and integrate with breathe -> parses doxygen xml output
   • Handles some artifacts from doxygen, like generics being shown as template<T, U> MyFunc ()
   • ? being randomly added to the end of a name (no idea where this is coming from, doxygen might have interpreted my property as a nullable) 4398e3193ebedff56c94735366d775b048e03608

Todo:

• [x] xref resolution uses endswith in middle of type (so Material matches MyMaterial) 0a3fa65269116a8fef32016492d347ffca688ff5
• [x] Move external types to the config and concatenate with defaults 8c14a696b71741dfd4198741ed96add039ec0912 14fdafdab9d01fc23dd4f2d099bf5484e75d772d
  • [x] Also put in a debug/verbose option in the config 389f01cc32860a766564cdaf7a64dbc895bbed06 37ac3d9c5b451d8e8e15b8cb3473f22983f9f27c
• [x] Add some more test cases fb38af584c169c86058cb4fe6423a0cf7302acab

Future work:

• I'd recommend splitting the parsing/regexes into separate files, but I've left this out so there's less diffs. (It would make a lot of sense reducing the size/complexity of one file though)
• Figure out the correct combination for the domain vars: object_types, directives, roles (I have no idea how this is supposed to work)
• Improve property, attribute, indexer regexes/parsing

The current test cases seem to be working for me.

This PR is a dependency of another PR at the breathe repo: https://github.com/michaeljones/breathe/pull/550

fyi, I haven't really done much python and no sphinx at all before, so some stuff might be completely wrong/unconventionial. But I'm happy to fix that :)

[rogerbarton]

Added config variables that can be set in conf.py (defaults indicated): See CSharpDomain also for examples of the defaults

# Check if external links are valid with a http request. This can be slow but will enable the fallback url if a url is invalid.
sphinx_csharp_test_links = True

# Is this a multi language project? If true, C# will be indicated in the tooltips
sphinx_csharp_multi_language = False

""" Extra namespaces to hide in the displayed text """
sphinx_csharp_shorten_type_prefixes = None

""" Types and strings to ignore when looking for a reference """
sphinx_csharp_ignore_xref_types = None

"""
Contains recognised types and their namespaces as well as the package name.
The package name references the key in external_search_pages

Format the content like this:
Where Namespace1 is the namespace for the link (which may not match the real namespace)::

    'package': {
        'Namespace1': ['member1', ...],
        ...
    },
    ...
"""
sphinx_csharp_external_type_map = None

"""
Put special cases in here that should be renamed when used in links, use it for generics
The name is swapped *after* searching EXTERNAL_TYPE_MAP, just before constructing the url link
"""
sphinx_csharp_external_type_rename = None

"""
Where do we search for api documentation
Syntax:
'package': (api link, fallback search link)
Use %s for where to substitute item, every link *must* contain this
"""
sphinx_csharp_external_search_pages = None

# Debug options
sphinx_csharp_debug = False  # Sets all flags to true
sphinx_csharp_debug_parse = False  # Sets all parse flags to true
sphinx_csharp_debug_parse_func = False
sphinx_csharp_debug_parse_var = False
sphinx_csharp_debug_parse_prop = False
sphinx_csharp_debug_parse_attr = False
sphinx_csharp_debug_parse_idxr = False
sphinx_csharp_debug_parse_type = False
sphinx_csharp_debug_xref = False
sphinx_csharp_debug_ext_links = False

Example Usage:

sphinx_csharp_shorten_type_prefixes = [
    'System.IO',
]

sphinx_csharp_ignore_xref = [
    'Vector2',
    'Vector3',
]

sphinx_csharp_ext_search_pages = {
    'upm.xrtk': ('https://docs.unity3d.com/Packages/[email protected]/api/%s.html',
                 'https://docs.unity3d.com/Packages/[email protected]/?%s'),
    'upm.tmp': ('https://docs.unity3d.com/Packages/[email protected]/api/%s.html',
                'https://docs.unity3d.com/Packages/[email protected]/?%s'),
    'upm.ugui': ('https://docs.unity3d.com/Packages/[email protected]/manual/script-%s.html',
                 'https://docs.unity3d.com/Packages/[email protected]/manual/index.html?%s')
}

sphinx_csharp_ext_type_map = {
    'unity': {
        'XR': ['InputDeviceCharacteristics'],
        'Experimental.AssetImporters': ['AssetImportContext', 'MeshImportPostprocessor', 'ScriptedImporter'],
    },
    'upm.xrtk': {'UnityEngine.XR.Interaction.Toolkit': ['XRRayInteractor', 'XRBaseInteractable', 'XRController']},
    'upm.tmp': {'TMPro': ['TMP_Text']},
    'upm.ugui': {'': ['Image', 'Button', 'Toggle']},
}

sphinx_csharp_external_type_rename = {
    'NativeArray': 'NativeArray_1',
}

[rogerbarton]

Resolving external links (e.g. to msdn) and reltated functionality has been moved to a new file/class. Also the data is now in a separate file to the functionality.

[ffes]

@djungelorm Any change the PR gets a review. I would love to be able to use this

[jonnew]

This should probably be merged...

[rogerbarton]

I doubt this will be merged any time soon, haven't heard anything from @djungelorm yet. My repo is up to date, so you can just use that master branch directly.

pip install git+https://github.com/rogerbarton/sphinx-csharp.git

[red8888]

Is this project dead? Or has it been moved or forked?

[jonnew]

I mean its been working for me. I think its just kinda done?

[ffes]

I think you are right @red8888, this repo is dead. This PR is open for over 1,5 years without any reaction of @djungelorm.

And @jonnew, when you look at the work that @rogerbarton has put into it, it definitely is not done. C# is a language where new feature are added every major release. When you use the file-scoped namespace introduced in C# 10, it breaks various parsers (maybe this one as well, haven't tested yet).

Maybe we should just open an issue named "This repo is dead", with a link to the repo of @rogerbarton if @djungelorm doesn't react any time soon.

[jonnew]

Lol I was not paying attention and actually thought this thread was on @rogerbarton's repo as that is the one I've been using.