Matplotlib pulls numpy: make lib optional

[OverkillGuy]

I was surprised to find numpy as part of my dependencies when I don't do any maths.

NumPy is a fantastic tool, but it's heavy and clearly unnecessary for my project. It turns out, numpy is a required dependency of matplotlib, which is now part of needs deps since 0.5.0.

I won't personally be doing any pie-chart-ing, so wouldn't really benefit from use of matplotlib, nor numpy. Can we make the matplotlib an optional dependency please? This way, people who are in more "constrained" systems don't need to pull in the whole maths ecosystem.

From setuptools docs on Optional Dependencies, adapted for us:

    extras_require={
        "matplotlib":  ["matplotlib"],
    }

As for gating the feature in python, from SO:

try:
    import matplotlib
except ImportError:
    # dependency missing, issue a warning
    import warnings
    warnings.warn('matplotlib not found, please install 'matplotlib' extra to enable pie-charts')

I realize there's a documentation overhead to doing this, but think it's worth it.

[danwos]

Yes, I agree. There are some (huge) dependencies, which are used for some directives only and therefore are often not needed for most use-cases.

Instead of a warning I would like to throw an exception, as the final documentation build will not contain the expected content (e.g. no pie charts). Otherwise I'm afraid that a user may miss the warning, as sphinx-build logs are normally not so small.

This could also be configurable, e.g. needs_missing_import_exceptions = False (True is default).

Any concerns about raising an exception and stopping build in such a case?

[OverkillGuy]

100% agreed. I have no issues with errors instead of warnings, the only reason I showed warnings was when searching for "python check extra_require imports" docs, which got me a warnings-based example.

Another example I can think of is sphinx-plantuml (which I enjoy a lot!) but isn't necessary unless using graphs. It requires a jar file that isn't trivial to install/configure (a small pain of its own, sadly), and a lot of installs can just skip it.

[OverkillGuy]

Wrote up a (draft) PR to do this, as I just bumped into NumPy again on a new project =)

I do notice that no other extra dependency was specified in docs, that PR opens the way for optional dependencies, which means there's extra burden of documentation and testing on this first one. If I was to make the same PR for plantuml in a few months after this goes in, it'd likely be much smoother, as 1 more optional isn't hard, but the first needs attention.

Looking forward to the discussion with maintainers about optional deps in general (see PR checklist).

[danwos]

I like the idea to have it optional. But I have some bad feelings about having it opt-in only. So a default installation will not provide the necessary dependencies to be able to use the full feature set.

But it looks like opt-in is the only supported way for optional dependencies in poetry.

Have to think about it and as this would be a breaking change, I would like to merge the PR when we raise the major version (which may need some time)

[OverkillGuy]

Yeah, I know what you mean about "dropping" default features due to optional, doesn't feel great, breaks it for some. Agreed on it being semver-major.

I hope documentation can make this less of an issue, by defining an all option: pip install sphinxcontrib-needs[all] is now the "default" way that docs recommends, with lighter installs without it. Happy to update PR to do that.

If we're waiting on major release to merge this, do you mind if I prep a separate PR in the same style for the other big dep: plantuml? If breaking changes are coming, might as well make it count, group those up. Again, no hurry on any of this, just a thought.

[danwos]

Update: Missunderstood some parts, so 1 deleted 1-2 paragraphs.

Agree, having pip install sphinxcontrib-needs[all] as main installation note in the docs is helpful.

So we will have these options:

• all - Install everything
• not given - install without matplotlib and plantuml

[OverkillGuy]

Agreed with your point, see new commit, should reflect that.

One detail, I do aim for matplotlib and plantuml options to be separate (once plantuml is separated, in a PR I'll send soon), so the install options would be:

• sphinxcontrib-needs[all]: recommended way, equivalent to sphinxcontrib-needs[matplotlib,plantuml]
• sphinxcontrib-needs[matplotlib] (just matplotlib, no plantuml
• sphinxcontrib-needs[plantuml] (just plantuml, no matplotlib)
• sphinxcontrib-needs (no options, lightest)

Just in case one wants one of the options but not the other.

[OverkillGuy]

Bump: have you given more thought to this dependencies split?

We have a PR for both Matplotlib here and #485 = PlantUML, splitting the options as laid out in comment above: no deps, one or the other, or "all" (new default, as recommended in docs).

I'm not in a hurry to merge this (major/breaking change needs careful thought) but I'm keen to see if there's anything the PRs don't cover, for when we do want to push ahead.

[danieleades]

this is causing me a headache right now, as i'm having build issues when installing numpy

[OverkillGuy]

@danwos I see the milestone for a 1.0 version is being updated, can we consider integrating this feature, and its major version break impact as part of 1.0 milestone? I'd like to revive the discussion, with the perspective we have now on the feature.

I am worried that after a 1.0 version is out, the magnitude of the proposed change here is going to make it so this feature will never be merged, as it would force another major version.

[danwos]

Thanks for your concerns. 1.0 got already released, mainly because we changed the package name from sphinxcontrib-needs to sphinx-needs. I have no problem with creating new major releases if we introduce breaking changes.

As all discussions are done, let me try to get this into the next release.