Guidance for use in a monorepo

[grahamhar]

I'm struggling to find any guidance or best practises on how or if pipenv should be used in a monorepo. Is the only way to use pipenv to have module specific pipenv files? Are there any tools that would facilitate running tests, creating the required pipenvs (virtualenvs) and then running them and on success creating packages across all modules in a repo?

[sarayourfriend]

@grahamhar I have the same request! We are using pipenv to manage dependencies in the Openverse API monorepo (https://github.com/wordpress/openverse-api) but are struggling to decide how to manage a dependencies like pre-commit, black and flake8 which should belong to the entire repository rather than each individual project we are developing inside the monorepo.

Furthermore, if someone is trying to use a single editor instance to work on the entire repo (which is common for monorepos), which Python interpreter should they use so that their language server works properly? Under the current set up we have there will be an individual virtualenv per project in the monorepo, but you can only point the editor to a single interpreter, not one per folder.

I've searched around a lot for ideas and there doesn't appear to be anyone discussing this use case outside of this issue.

I wonder if the maintainers have any insight into this as the issue has been in triage for quite some time!

[matteius]

@sarayourfriend sounds like what you are looking for is possibly the category feature request being tracked in: https://github.com/pypa/pipenv/issues/4745

[sarayourfriend]

A bit, but not quite. We have two different Pipfiles in our repository, with this structure:

| api
|-- Pipfile
| ingestion_server
|-- Pipfile

Where would we put something like these meta, devex focused dependencies that are required by both projects? We could duplicate them, but then upgrading their versions, especially with an automated tool like dependabot, would be a chore (it opens two PRs to update the same dependency across both Pipfiles instead of just once across everything in the repo).

We could have a top-level Pipfile with these dependencies. Maybe that is the correct solution. I'm just not sure. We haven't had a lot of time to dig into this on Openverse recently so it's fallen by the wayside. Maybe the categories would help but I'm not seeing right away how they would in our particular situation (thought they do seem useful for a flat-structured project with a single Pipfile and will look forward to the feature for those projects).