system icon indicating copy to clipboard operation
system copied to clipboard

Trying to build the perfect system

  • System Configuration

Everything needed to build my NixOS system. Note this is highly customized configuration that's not targeting a wide audience. However, feel free to copy and use any parts of if you like in your own configuration.

** Table of Contents :TOC_3:

  • [[#system-configuration][System Configuration]]
    • [[#folder-structure][Folder Structure]]
      • [[#config][config]]
      • [[#config-backups][config-backups]]
      • [[#emacs--nvim][emacs / nvim]]
      • [[#hosts][hosts]]
      • [[#keyboard-firmware][keyboard-firmware]]
      • [[#modules-profiles-and-users][modules, profiles, and users]]
      • [[#secrets][secrets]]
      • [[#xmonad][xmonad]]
    • [[#nixos][NixOS]]
      • [[#build-custom-iso][Build Custom ISO]]
      • [[#write-custom-iso-to-a-usb-stick][Write Custom ISO to a USB stick]]
      • [[#install][Install]]
      • [[#update][Update]]
      • [[#nix-repl][Nix Repl]]
      • [[#switch][Switch]]
      • [[#flake-learning-resources][Flake Learning Resources]]
      • [[#nixos-on-wsl2][NixOS on WSL2]]
  • [[#license][License]]

** Folder Structure

The following is 1 level deep directory structure generated using ~tree -L 1 -d~.

#+begin_src . ├── config ├── config-backups ├── emacs ├── hosts ├── keyboard-firmware ├── modules ├── nvim ├── profiles ├── secrets ├── users └── xmonad #+end_src

*** config

Config files used in NixOS that are symlinked to ~/.config/~ and managed with [[https://github.com/rycee/home-manager][home manager]].

*** config-backups

Miscellaneous configuration files that I'm backing up and are not deployed automatically.

*** emacs / nvim

My emacs configuration code. See its [[file:emacs/readme.org][readme]] for more information.

I typically only use Vim if Emacs is not available for some reason. In the ~init.vim~ file there are just the most important key bindings in my [[keyboard-firmware/readme.org][customized Evil / Vim keybindings]]

*** hosts

host specific configuration files.

*** keyboard-firmware

Holds my keyboard firmware code managed by Nix and the [[file:keyboard-firmware/readme.org][readme]] that goes into depth on my key layout and customized Evil / Vim keybindings.

*** modules, profiles, and users

similar to and inspired by: https://devos.divnix.com/index.html

*** secrets

A place for me to store encrypted secrets using [[https://github.com/AGWA/git-crypt][git-crypt]].

*** xmonad

My Xmonad configuration code which is symlinked by home-manager. See its [[file:xmonad/readme.org][readme]] for more information.

** NixOS

*** Build Custom ISO

I have a custom bootable ISO that provides a graphical environment similar to the one in nixpkgs with a few nice to have tools and my Emacs configuration for installing NixOS.

To build it:

#+begin_src shell git clone https://github.com/willbush/system.git cd system nix build ".#iso" #+end_src

The ISO will be in the ~./result/iso~ build result folder.

*** Write Custom ISO to a USB stick

Use ~lsblk~ to determine the device and replace ~/dev/sdX~ in the example below with your device. You should not have the device mounted before running the following command. In addition, the exact file name will probably be different from the example below. Also see [[https://nixos.org/manual/nixos/stable/index.html#sec-booting-from-usb][the manual]] on this topic for additional information.

#+begin_src shell sudo dd if=./result/iso/nixos-21.03.20201209.e9158ec-x86_64-linux.iso of=/dev/sdX status=progress bs=4M #+end_src

If you're in Windows try https://rufus.ie/.

*** Install

The primary source of documentation for installing NixOS is the [[https://nixos.org/manual/nixos/stable/][manual]]. The following are my own notes on how I install things in various situations. I thought about making a install script that [[https://xkcd.com/1319/][automates]] most of this for me, but it just doesn't seem worth it with the varying use cases and low frequency of re-formatting.

**** [[https://virt-manager.org/][Virt Manager]]

  1. File > New virtual machine > Forward.
  2. Choose ISO > Forward (it should auto-detect that it's NixOS).
  3. Choose Memory and CPU amount (~4 CPUs and ~6148 RAM when host can afford it)

    Forward.

  4. Choose available disk size (I usually increase to 30 GiB) > Forward.
  5. Check the ~Customize configuration before install~ > Forward.
  6. In the Overview section consider changing the Firmware from BIOS to UEFI > Apply
  7. In the Video QXL section change the model Virtio and enable 3D acceleration (allows increasing desktop resolution) > Apply > Begin Installation.
  8. Once booted into the ISO the first thing I do is go to View > Scale Display Always. In addition, I increase the resolution using Display Configuration for plasma5.

**** Dual booting NixOS with Windows

These are my abridged notes on dual booting Windows 10 along side NixOS on a single drive and UEFI system. The following outlines how to ensure the EFI partition that Windows creates isn't ridiculously small, how to remove the Recovery partition that Windows creates, and how to shrink the Primary partition to make space for NixOS.

***** Create custom sized EFI partition that NixOS will share and Install

  1. Boot into the Windows setup and select custom install.
  2. Delete all partitions on the drive until only unallocated space remains.
  3. Press Shift+F10 to open an elevated command prompt.
  4. Open the disk partitioning tool's REPL by entering ~diskpart~.
  5. Enter ~list disk~ and ~select disk n~ where n is your disk number.
  6. ~create partition efi size=512~ to create a EFI system partition 512MiB.
  7. ~format quick fs=fat32 label=System~
  8. Enter ~exit~ twice to get out of the REPL and command prompt.
  9. Select the unallocated space and hit next (it will automatically utilize the created EFI partition).

***** Delete Recovery partition

  1. Once booted into Windows open another elevated command prompt and ~diskpart~.
  2. Again enter ~list disk~ and ~select disk n~ where n is your disk number.
  3. ~list partition~ and ~select partition n~ where n is the Recovery partition number.
  4. ~delete partition override~ and ~exit~ twice.

***** Shrink Primary partition to make space for NixOS

  1. Open the Disk management GUI application.
  2. Right click on the large ~C:~ NTFS partition and shrink it to your desired size.
  3. Continue with the UEFI version of the next section.

**** Basic partitioning and formatting

The following is mostly just following the manual instructions [[https://nixos.org/manual/nixos/stable/index.html#sec-installation-partitioning-UEFI][here]]. However, I switched to using =cfdisk= instead of =parted= because I've hit the [[https://unix.stackexchange.com/questions/38164/create-partition-aligned-using-parted][misalignment warning]] using =parted= when following the manual's instructions. Most suggestions to fix the warning involve using percentages, but that's annoying when you want to create 3 or more partitions. The =cfdisk= utility allows specifying sizes and auto aligns partitions optimally.

  • Note that one can wipe all the file systems on a device using ~sudo wipefs -a /dev/sdX~ which is useful to start over or prepare a device.

  • Note your system booted in UEFI mode when ~/sys/firmware/efi/~ folder exists and has files in it.

***** BIOS

  1. ~sudo cfdisk~
  2. select dos
  3. new > 2G (adjust size to your liking) > primary > type > Linux swap / Solaris.
  4. Select free space > new > enter (for remaining space) > primary.
  5. Write > yes > quit

****** Check alignment

At this point I like to use =parted= just to double check the alignment.

  1. Open a parted REPL: ~sudo parted~
  2. Print all devices to determine the device to format: ~print all~
  3. Select device: ~select /dev/sda~ (Note the exact disk depends on your ~print all~ output.)
  4. ~align-check optimal 1~ and repeat for ~2~ (both should output aligned)
  5. ~q~ to quit.

****** Formatting

#+begin_src shell sudo mkswap -L swap /dev/sda1 sudo mkfs.ext4 -L nixos /dev/sda2 #+end_src

****** Mount

Mount the target NixOS file system:

#+begin_src shell sudo mount /dev/disk/by-label/nixos /mnt #+end_src

Continue to the Installation section.

***** UEFI

  1. ~sudo cfdisk~
  2. select gpt (if prompted)
  3. new > 512M > type > EFI System (skip this step if dual booting with Windows)
  4. select free space
  5. new > 2G (adjust size to your liking) > type > Linux swap
  6. Select free space > new > enter (for remaining space)
  7. Write > yes > quit

****** Check alignment

At this point I like to use parted just to double check the alignment.

  1. Open a parted REPL: ~sudo parted~
  2. Print all devices to determine the device to format: ~print all~
  3. Select device: ~select /dev/sda~ (Note the exact disk depends on your ~print all~ output.)
  4. ~align-check optimal 1~ and repeat all partitions (all should output aligned)
  5. ~q~ to quit.

****** Formatting

#+begin_src shell sudo mkfs.fat -F 32 -n boot /dev/sda1 # Skip when dual booting Windows. sudo mkswap -L swap /dev/sda2 sudo mkfs.ext4 -L nixos /dev/sda3 #+end_src

****** Mount

Mount the target NixOS file system:

#+begin_src shell sudo mkdir -p /mnt/boot sudo mount /dev/disk/by-label/boot /mnt/boot #by-label/SYSTEM if partition created by Windows sudo mount /dev/disk/by-label/nixos /mnt #+end_src

**** Installation

  1. If you have swap, then enable it now.

    When the ~hardware-configuration.nix~ is generated it won't include an entry for the swap partition unless you enable it before generating the file.

    #+begin_src shell sudo swapon /dev/sda2 #+end_src

  2. Prepare the configuration

    #+begin_src shell sudo nixos-generate-config --root /mnt cd /mnt/etc/nixos sudo git clone https://github.com/willbush/system.git #+end_src

    For a new machine setup the new machine in ~flake.nix~ and under ~./system/hosts/~ using the generated ~hardware-configuration.nix~.

    Change the hashed user passwords in ~user.nix~. These hashed passwords can be generated with ~mkpasswd -m sha-512~.

    Disable anything else that depends on unlocking the encrypted secrets folder (just =syncthing= at the time of writing this).

    Be sure to stage all new files created in repository as required for flakes.

  3. Install

    Replace betelgeuse in the following command with your machine name.

    #+begin_src shell sudo nixos-install --flake /mnt/etc/nixos/system/#betelgeuse --no-root-passwd --impure reboot #+end_src

    Notes:

    • No root password needs to be created because it should be specified in ~user.nix~ as a hashed password.
    • Impure flag is needed because nix thinks ~/mnt/nix/store~ as a non-store path.
  4. Move system repository

    After booting into the newly installation, I personally prefer my system repository in my home directory (change the user as needed):

    #+begin_src shell sudo mv /etc/nixos/system ~/ sudo chown -R will ~/system/ #+end_src

*** Update

To update all the flake inputs:

#+begin_src shell nix flake update #+end_src

To up update a single input:

#+begin_src shell nix flake update --update-input nixpkgs #+end_src

*** Nix Repl

To open the flake in a ~nix repl~ do the following:

#+begin_src shell nix run '.#repl' #+end_src

*** Switch

Build and switch to the generation provided by the flake for machine / host name:

#+begin_src shell sudo nixos-rebuild switch --flake <PATH_TO_GIT_REPO>#<MACHINE> #+end_src

For example:

#+begin_src shell git clone https://github.com/willbush/system.git cd system sudo nixos-rebuild switch --flake '.#betelgeuse' #+end_src

The ~.~ before the ~#~ is the path to the git repository. What comes after the ~#~ is the machine / host name which can be found enumerated in the ~flakes.nix~ file.

*** Flake Learning Resources

  • https://nixos.wiki/wiki/Flakes
  • [[https://github.com/tweag/rfcs/blob/flakes/rfcs/0049-flakes.md][Original RFC]]
  • [[https://www.youtube.com/watch?v=UeBX7Ide5a0][Flake talk at NixConf]]
  • https://www.tweag.io/blog/2020-05-25-flakes/
  • https://www.tweag.io/blog/2020-06-25-eval-cache/
  • https://www.tweag.io/blog/2020-07-31-nixos-flakes/

**** Flake Examples

  • https://github.com/colemickens/nixos-flake-example
  • https://github.com/hlissner/dotfiles
  • https://github.com/lexuge
  • https://github.com/divnix/devos

*** NixOS on WSL2

Thanks to [[https://github.com/Trundle/NixOS-WSL][NixOS-WSL]] I can run NixOS in WSL2 on Windows.

Anytime I do this, I need to check what's changed since last time and update code copied from that repo into [[./hosts/nixos-wsl]]. Code was copied there because they don't yet [[https://github.com/Trundle/NixOS-WSL/pull/19][have modular flake support]].

I can build my own tarball:

#+begin_src shell nix build .#nixosConfigurations.nixos-wsl.config.system.build.tarball #+end_src

The tarball ends up in ~./result/tarball/nixos-system-x86_64-linux.tar.gz~

Copy it over to Windows and install:

#+begin_src PS C:\Users\will\Downloads> mkdir C:\NixOS PS C:\Users\will\Downloads> wsl --import NixOS C:\NixOS\ .\nixos-system-x86_64-linux.tar.gz --version 2 PS C:\Users\will\Downloads> wsl -d NixOS $ /nix/var/nix/profiles/system/activate $ exit #+end_src

Set NixOS as the default ~wsl -s NixOS~ and ~wsl~ will now take me into the NixOS distribution.

**** Bootstrap my WSL2 flake using NixOS-WSL's provided tarball

  1. Download the [[https://github.com/Trundle/NixOS-WSL/releases/latest/download/nixos-system-x86_64-linux.tar.gz][tarball]] from their GitHub page.

  2. Perform same steps above to install the tarball.

  3. ~sudo nano /etc/nixos/configuration.nix~

    Edit the configuration to enable nix flakes and add the git package.

    #+begin_src nix {

    ...

    nix = { package = pkgs.nixUnstable; extraOptions = '' experimental-features = nix-command flakes ''; };

    List packages installed in system profile.

    environment.systemPackages = with pkgs; [ git ];

    ...

    } #+end_src

    Rebuild and switch:

    #+begin_src shell sudo nixos-rebuild switch #+end_src

  4. Clone this repository and rebuild switch:

    #+begin_src shell sudo nixos-rebuild switch --flake '/home/nixos/system/#nixos-wsl' #+end_src

  • License

Dual licensed under either:

  • Apache License, Version 2.0 ([[http://www.apache.org/licenses/LICENSE-2.0][LICENSE-APACHE]])

  • MIT license ([[http://opensource.org/licenses/MIT][LICENSE-MIT]])

at your option.

However, code under the ~./hosts/nixos-wsl/~ directory is distributed solely under the Apache License, Version 2.0. See its [[file:hosts/nixos-wsl/readme.org][readme]] for more information.