Add a nix flake providing a development shell

[charludo]

Short description

(Since this has nothing to do with any issue or any plans we've discussed as a team, I feel that I need to preface this with: everything related to this was done on my own time, not during working hours.)

This PR adds a nix flake. It allows developers to utilize a standardized and completely reproducible development environment.

You might find this useful if you currently manage multiple python and/or node projects (possibly requiring conflicting versions), are a nix user already, or frequently re-setup the project; but more importantly I strongly believe this makes setting our project up significantly easier for new developers (speaking from experience from my own onboarding two years ago 😉):

Regardless of the system a person is on - any kind of Linux; MacOS; even Windows with WSL enabled - getting up and running with our project would boil down to

1. clone this repo
2. install nix (and only nix, not python/node/...), for example with the awesome installer script from zero-to-nix.com:

   curl --proto '=https' --tlsv1.2 -sSf -L https://install.determinate.systems/nix | sh -s -- install
   

3. run nix develop in your shell

As a result, you'll have the exact same development environment available as me/anyone else following these steps.

Proposed changes

• add a nix flake and corresponding lock file

Side effects

• none, except:
• I volunteer to give a bar camp about nix/nix develop/NixOS at the next conference 😄

Resolved issues

Fixes: n/a

---

Pull Request Review Guidelines

[codeclimate[bot]]

Code Climate has analyzed commit 1ebf26af and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 82.7% (0.0% change).

View more on Code Climate.

[JoeyStk]

I volunteer to give a bar camp about nix/nix develop/NixOS at the next conference 😄

Would be interested in hearing it!

[charludo]

Looks good code-wise but I ran into an error when starting the development server:

Good catch, the first time the database is attempted to be started in require_database pg-start is called, but the dirs do not yet exist. Should be fixed now. In case a similar error crops up now, please try to delete the .postgres dir, since it's not tracked by git but might have been created during the failed run from which you've shown the error message.

[charludo]

@svenseeberg @timobrembeck @david-venhoff the three of you seemed most interested at the conference, would one of you mind doing the second review? 😅

[david-venhoff]

I also get an error

The env seems to contain these values though :thinking:

[charludo]

I also get an error

The env seems to contain these values though 🤔

Interesting. Not sure why that would happen, but I'll have a look and try to find the source of the error.

Did ./tools/install.sh work without errors?

[charludo]

@david-venhoff this SO post seemed relevant, describing the exact same issue with postgres. The first answer is some hacky workarounds....

...and the second answer is "you didn't mention nix, but I'm guessing this is a nix issue" 😂

I've added glibcLocales to the packages list, would appreciate if you could check if the error is now resolved.

[david-venhoff]

Yes, the commit fixes my error. Now I get another one:

(.venv) user@laptop-pop-os:~/dev/integreat-cms$ ./tools/run.sh 
Checking if Integreat CMS is installed...
✔ Integreat CMS is installed
Checking if local Redis server is running...
✔ Running Redis server on port 6379 detected. Caching enabled.
✔ Running PostgreSQL database detected
Migrating database...
Traceback (most recent call last):
  File "/home/user/dev/integreat-cms/.venv/bin/integreat-cms-cli", line 54, in <module>
    main()
  File "/home/user/dev/integreat-cms/.venv/bin/integreat-cms-cli", line 50, in main
    execute_from_command_line(sys.argv)
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/django/core/management/__init__.py", line 416, in execute
    django.setup()
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/django/__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/django/apps/registry.py", line 91, in populate
    app_config = AppConfig.create(entry)
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/django/apps/config.py", line 123, in create
    mod = import_module(mod_path)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.pyenv/versions/3.11.9/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/home/user/dev/integreat-cms/integreat_cms/google_translate_api/apps.py", line 15, in <module>
    from google.cloud import translate_v2, translate_v3  # type: ignore[attr-defined]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/google/cloud/translate_v2/__init__.py", line 18, in <module>
    from google.cloud.translate import __version__
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/google/cloud/translate/__init__.py", line 21, in <module>
    from google.cloud.translate_v3.services.translation_service.async_client import (
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/google/cloud/translate_v3/__init__.py", line 21, in <module>
    from .services.translation_service import (
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/google/cloud/translate_v3/services/translation_service/__init__.py", line 16, in <module>
    from .async_client import TranslationServiceAsyncClient
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/google/cloud/translate_v3/services/translation_service/async_client.py", line 32, in <module>
    from google.api_core import gapic_v1
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/google/api_core/gapic_v1/__init__.py", line 16, in <module>
    from google.api_core.gapic_v1 import config
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/google/api_core/gapic_v1/config.py", line 23, in <module>
    import grpc
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/grpc/__init__.py", line 22, in <module>
    from grpc import _compression
  File "/home/user/dev/integreat-cms/.venv/lib/python3.11/site-packages/grpc/_compression.py", line 20, in <module>
    from grpc._cython import cygrpc
ImportError: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.38' not found (required by /nix/store/c2yb135iv4maadia5f760b3xhbh6jh61-gcc-13.2.0-lib/lib/libstdc++.so.6)

I found this post which suggests to unset LD_LIBRARY_PATH and then the server finally starts on my laptop. I don't really know what this means though or where that environment variable gets set :shrug:

[charludo]

I found this post which suggests to unset LD_LIBRARY_PATH and then the server finally starts on my laptop. I don't really know what this means though or where that environment variable gets set 🤷

We actually set that ourselves, currently:

LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath [ pkgs.stdenv.cc.cc.lib pkgs.file ];

This is necessary to make libstdc++ available to the python version which gets linked in the process of creating a venv. As far as I can tell, the "setting LD_LIBRARY_PATH at all is a massive red flag" comment from the thread you linked refers to the context of the original post, where LD_LIBRARY_PATH is set on a system-level, which is indeed heavily discouraged.

Inside of development shells on the other hand, it's the preferred way to get python venvs working (see for example this thread, where this distinction is also made).

However, I just realized that stdenv.cc.cc.lib is not part of the input packages. I've added it, hopefully this will make the required glibc available on non-nix systems (btw, not exporting LD_LIBRARY_PATH leads to very similar errors as you've posted above for me 😂 I'm guessing that the glibc existing on my system is always the required one (since coming from nixpkgs both for my system, and for the packages installed in the shell), while the one provided by your distribution differs from the one the packages in the shell rely on).

In any case, I'm sorry that I can't be more certain if this will fix it, since I can't reproduce the error 🙈