geostore icon indicating copy to clipboard operation
geostore copied to clipboard

Replace Pip, Poetry and NPM with Nix?

Open l0b0 opened this issue 2 years ago • 0 comments

Enabler

So that we have to deal with only a single packaging system, we want to use only Nix.

Pros:

  • We'd only ever need to update a single file (nixpkgs.json) to bump all the packages we use. If for any reason we need to use different packages in some part of the system that would be as simple as modifying and referencing a copy of nixpkgs.json, which could then trivially be merged back into the original nixpkgs.json when the packages end up being in sync again.
  • We wouldn't need to use poetry2nix, which, while excellent, is always playing catch-up with changing Python packages.
  • Faster builds, since we'd be using the official Nix cache for probably every package except linz-logger.
  • Reproducible builds, including Docker images.
  • ./reset-dev-env.bash would be replaced with nix-shell or direnv allow.

Cons:

  • Nix has a famously steep learning curve, but they are working on it.
  • We would not be able to support non-POSIX platforms like Windows. Nix only supports POSIX platforms (Linux, BSD, Mac OS, etc.).
  • Anyone who has set up their environment to work with Ubuntu would have to install and use Nix instead. This is much simpler than the Ubuntu setup, but it is an extra step.
  • We might want to package some things which are not yet in nixpkgs. For example, it looks like boto3-stubs is not yet in nixpkgs. I've already released some packages on nixpkgs, and it's not particularly difficult.

Acceptance Criteria

  • [ ] No mentions of Poetry in the repo.
  • [ ] No more npm commands in the repo.
  • [ ] No more pip commands in the repo.

Additional context

This would require developers to use Nix for development, rather than continuing to support a non-Nix workflow.

Tasks

  • [ ] Move the packages mentioned in the tool.poetry sections from pyproject.toml to a Nix file
  • [ ] Remove poetry.*
  • [ ] Remove geostore/*.txt
  • [ ] Change bundle.bash to build using Nix. This could possibly involve creating Docker containers using Nix.
  • [ ] Replace Dependabot with a tool which supports Nix (Renovate?)
  • [ ] Remove generate-requirements-files.bash
  • [ ] Remove pip, pip-tools, poetry, and poetry2nix from Nix shell definition
  • [ ] Remove package*.json
  • [ ] Remove Nix shell shellHook, since it would no longer be necessary
  • [ ] Remove reset-dev-env.bash
  • [ ] Convert infrastructure/constructs/lambda_layers/botocore to use Nix, if possible
  • [ ] Update relevant documentation
  • [ ] Ensure local development works easily. This should be as simple as copying the commands from the GitHub Actions.

l0b0 avatar Dec 07 '22 21:12 l0b0