cudf Update PandasCompat.py to resolve references

Description

This PR allows the PandasCompat sphinx ext to contain resolved references. For example, you can now add intersphinx mapping to the content of the admonition.

Motivation

I enjoy connecting the PyData communities and this PR allows for more opportunities to use intersphinx mapping to link back to the pandas docs.

History

I first tried this in a previous PR (https://github.com/rapidsai/cudf/pull/15383#discussion_r1537888240) and commented here (https://github.com/rapidsai/cudf/pull/15383#issuecomment-2028451487) that I may get around to investigating this further. I finally had to time to work on this and made a bit of progress.

Testing

I created a separate repo for this at https://github.com/raybellwaves/compatsphinxext which deploys straight to https://raybellwaves.github.io/compatsphinxext you can see it's working as expected here: https://raybellwaves.github.io/compatsphinxext/compat.html. You should be able to fork that and tinker pretty quickly.

Further work

This could be cleaned up (for example I couldn't get the [source] to display in the admonition as I worked from the latest sphinx todo extension (https://github.com/sphinx-doc/sphinx/blob/master/sphinx/ext/todo.py)). The existing pandas-compat Admonition's could be switched to this if agreed. In addition, the documentation around how to write pandas-compat entries going forward (https://github.com/rapidsai/cudf/blob/branch-24.06/docs/cudf/source/developer_guide/documentation.md#comparing-to-pandas) will also have to be updated.

Longer term the extension could be published and used across RAPIDS libraries where there are differences in compatibility with PyData libraries e.g. pandas, network, scikit-learn to simplify linking to those dos. I'm not sure if I'll have time to work on this though.

Checklist

[x] I am familiar with the Contributing Guidelines.
[ ] New or existing tests cover these changes.
[ ] The documentation is up to date with these changes.

May 09 '24 07:05 raybellwaves

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

May 09 '24 07:05 copy-pr-bot[bot]

/ok to test

May 15 '24 23:05 vyasr

@raybellwaves sorry this got a bit lost in the shuffle. @bdice is off for a couple of days so this may not get done until next week, but I've kicked off tests so that we can at least get a doc build going and make sure it works as expected.

May 15 '24 23:05 vyasr

@raybellwaves sorry this got a bit lost in the shuffle. @bdice is off for a couple of days so this may not get done until next week, but I've kicked off tests so that we can at least get a doc build going and make sure it works as expected.

Np. Sorry i'm slow to fix as well. I'll ping when ready for another review.

May 16 '24 00:05 raybellwaves

Ready for another round of reviews. Thanks for the code simplification suggestions @bdice. It still works as expected as shown here: https://raybellwaves.github.io/compatsphinxext/compat.html

A minor difference is the [source] link is now below the admonition (text box). I couldn't work out how to add it into the admonition box. I asked on the sphinx discussion page https://github.com/orgs/sphinx-doc/discussions/12366 but it hasn't been answered. I would say it's minor and only aesthetic and doesn't need to block the PR but of course you can disagree.

May 23 '24 21:05 raybellwaves

@raybellwaves thanks for coming back to this! Given the timing (we're about to freeze the 24.06 release), I'm going to retarget at 24.08 and merge the latest. Hope you don't mind.

May 23 '24 23:05 vyasr

/ok to test

May 23 '24 23:05 vyasr

@raybellwaves could you fix the style check here? Thanks!

May 28 '24 17:05 vyasr

/ok to test

May 29 '24 18:05 vyasr

It looks like docs builds failed in CI. https://github.com/rapidsai/cudf/actions/runs/9291509987/job/25571688171?pr=15704

We'll need to get those fixed before we can merge this. I'm not sure why it failed, but there is a warning:
WARNING: Domain 'PandasCompat::pandascompat' has not implemented a `resolve_any_xref` method [myst.domains]

Not obvious why this appeared. I'll got a minimal pipeline i've been testing this on https://github.com/raybellwaves/compatsphinxext/actions/runs/9342573682/job/25710886035 at it runs fine there.

Could be a sphinx version? I have 7.3.7 in my version (https://github.com/raybellwaves/compatsphinxext/actions/runs/9342573682/job/25710886035#step:5:86) whereas the docs env has 6.2.1 (https://github.com/rapidsai/cudf/actions/runs/9291509987/job/25571688171?pr=15704#step:9:483)

May not be this. Also get 6.2.1 in a fresh install using https://github.com/rapidsai/cudf/blob/branch-24.08/conda/environments/all_cuda-122_arch-x86_64.yaml#L86. Latest conda is at 7.3.7 https://github.com/conda-forge/sphinx-feedstock not sure if another package is causing the sphinx pin.

LibMambaUnsatisfiableError: Encountered problems while solving:
  - package breathe-4.35.0-pyhd8ed1ab_0 requires sphinx >=4.0,<6.0.0a, but none of the providers can be installed

Could not solve for environment specs
The following packages are incompatible
├─ breathe >=4.35.0  is installable with the potential options
│  ├─ breathe 4.35.0 would require
│  │  └─ sphinx >=4.0,<6.0.0a , which can be installed;
│  └─ breathe 4.35.0 would require
│     └─ sphinx >=4.0,<7.0.0a , which can be installed;
└─ sphinx >=7.3.7  is not installable because it conflicts with any installable versions previously reported.

I created https://github.com/conda-forge/breathe-feedstock/pull/60

The warning is similar to https://github.com/executablebooks/jupyter-book/issues/1226. Following https://myst-parser.readthedocs.io/en/latest/configuration.html#build-warnings I also tried add `suppress_warnings = ["myst. domains"]. I haven't tried building with this yet.

Jun 03 '24 01:06 raybellwaves

/ok to test

Jun 06 '24 16:06 vyasr

@raybellwaves I tried to review this but I don't see any changes in the docs. Does this PR affect anything yet, or is it just setup for some future change?

Jun 10 '24 16:06 bdice

The only difference I see is that the [source] link was moved out of the box:

Before

After

Jun 10 '24 16:06 bdice

The only difference I see is that the [source] link was moved out of the box:

Before

After

Correct. Follow on PR is https://github.com/rapidsai/cudf/pull/15846. Sorry if I forgot to link it before.

I assume the docs built ok this time. Let me know if I need to follow up on https://github.com/conda-forge/breathe-feedstock/pull/60

Jun 10 '24 17:06 raybellwaves

Great! That's fine. It looks like the docs built so I'll merge this and we can try it with #15846.

Jun 10 '24 19:06 bdice

/merge

Jun 10 '24 19:06 bdice