NixOS

NixOS is a Linux distribution based upon the Nix package manager.

Introduction to Nix

NixOS's fundamental difference is the idea that all software is never installed globally on the system but rather placed in a central content store on your system, also known as the Nix store. Everything a package needs - the binaries, global default configuration, libraries - are placed together within a directory in the Nix store. When a package is 'installed', it is simply added to your system's PATH so that it appears within your environment.

The consequence of this design choice makes a lot of different things possible:

1. Multiple versions of the same software can be installed on the same system. It's even possible to run these different versions at the same time by having different PATHs defined for each environment.
2. Rollbacks are simple. You have the old version of the software installed already. Just change the previous version back as the default.
3. Upgrades are faster. You can leave the current version of some software running while you install and test the new version. Once you're ready, you can then cut over to the new version. Since everything is already on the system, all the system needs to do is update a few symlinks and you're upgraded.
4. Entire system upgrades can be rolled back. This idea of having all previous software available on the Nix store means you don't need to do A/B partitioning.

Packages under the hood

Nix packages are also known as derivations and are stored under in the Nix store at /nix/store/hash-name-version. All packages are named with a cryptographic hash that uniquely identifies the package name and version number. All package files are also immutable and should not be changed once it is installed. The Nix package manager will arrange the binaries and libraries in your environment appropriately (by tweaking the PATHs) based on your global system and per-user configuration.

The Nix database is stored at /nix/var/nix/db.

Profiles and generations

Profiles define what packages and what version of packages should be available in your environment. Based on your selected profile, Nix will gather all the necessary components spread among different paths into a single unified path. When we upgrade a system, Nix installs all the new packages and creates a new profile referencing the updated packages. In order to facilitate rolling back to a previous version, Nix version controls all profiles into what's known as a generation. A generation in essence is a specific version of a profile. After a system upgrade, if we ever need to roll back to the system prior to the upgrade, we can reboot into an older generation in the Grub menu.

Profiles are stored under ~/.nix-profile which is a symlink to /nix/var/nix/profiles/default. Different generations of the profile are stored /nix/var/nix/profiles/.

nix-env handles the profiles and all the symlinks in this location. Paths are also merged by nix-env.

Dependencies are called closures

Closures of a derivation is a recursive list of all dependencies that are required.

You can generate a list by running: nix-store -qR `which man` or nix-store -q --tree `which man`

Package repositories are called channels

If you ever dealt with Anaconda (the software manager), you will be familiar with the concept of channels. You can think of channels as a package repository. Nix has a few channels which you can see at https://nixos.org/channels:

• Stable (eg. nixos-20.03): minor bug fixes, no major kernel changes
• Unstable (eg. nixos-unstable): Main development branch
• Small (nixos-20.03-small, nixos-unsable-small): Contains fewer binary pakages and faster update cycles. Intended for server environments.

Cheatsheet

Various commands that were introduced in Chapter 3 of the Nix Pills.

Command	Description
nix-env	Manages environments, profiles, and their generations
nix-env -i hello	installs the hello environment
nix-env --list-generations	lists all environment generations
nix-env -q	lists all derivations
nix-env -q --references `which hello`	shows all derivations that are required by the given binary
nix-env -q --referrers `which hello`	shows all derivations that are dependent on the given binary
nix-env --rollback	rolls back the generation by one
nix-env -G 3	sets the environment generation to the given generation (3)
nix-env -u	upgrades all packages in the environment
nix-env -e '*'	Uninstalls everything from the current environment. You can recover this by running nix-env from the nix store and reinstalling the nix-env derivative or roll back the environment generation.
nix-store -q --tree /run/current-system	List all package dependencies
nix-store --gc --print-roots	See what is using a dependency

Command	Description
nix-channel --list	lists all channels. This is configured in ~/.nix-channels
nix-channel --update	Synchronizes the channel repo data
nix-channel --add https://nixos.org/channels/nixos-20.03 nixos	Adds a channel

To upgrade the system, you can add a new channel and then run nixos-rebuild switch --upgrade. This is equivalent to nix-channel --update nixos ; nixos-rebuild switch.

Remember, users can have their own channels. Only root can update channels that will affect the entire system (and /etc/nixos/configuration.nix).

Installing NixOS

The installation steps are listed in section 2.3 of the manual. I'll go over the steps to NixOS set up on a non-UEFI system.

Download the NixOS ISO and boot it. Start a terminal and become root using sudo su. Format the primary disk with at least one root partition and optionally a swap partition. You may either use fdisk or gparted. Mount the primary root partition as /mnt.

# fdisk /dev/sda  # Create a partition for the system, another for swap if desired
# mkfs.ext4 -L nixos /dev/sda1
# mount /dev/sda1 /mnt

Change the editor

If you want to use emacs to edit the initial config, run nix-env -f '<nixpkgs>' -iA emacs first.

Setup NixOS by generating an initial configuration file. Edit the configuration file and ensure that the boot loader is set up. Run nixos-install to finish the installation.

# nixos-generate-config --root /mnt 
# cd /mnt   # You can take a look at what's generated

## Review the configuration file. 
## BIOS systems: ensure that boot.loader.grub.device is set
## UEFI systems: ensure that boot.loader.systemd-boot.enable is set to true
# vi /mnt/etc/nixos/configuration.nix

## When you're ready, set up the system
# nixos-install

The nixos-install command will download all the required derivations, set up the boot loader, etc. The last step of the command will prompt for a root password. Once complete, you should be able to reboot into the new installation without the ISO image.

After the system comes up, you should be able to log in as root with the password you provided in the installation step. You can change the system by editing the same /etc/nixos/configuration.nix file as you did originally. For changes to apply, run nixos-rebuild switch. Any system services that are affected should be automatically restarted by this rebuild process. If so desired, you can make a separate profile for the rebuild with the -p profile flag so that it shows up as a separate grub option.

You may also use nixos-rebuild test to test the configuration without making it boot by default. You may also use nixos-rebuild boot to do everything as switch did but without switching over until the next reboot.

Administration tasks

All options that can be set are listed in the NixOS manual.

• https://nixos.org/nixos/manual/options.html

Section 5 through 42 covers some of the administration tasks:

• https://nixos.org/nixos/manual/index.html

Installing a package in NixOS

For system-wide packages, you can define them in the configuration.nix configuration file under environment.systemPackages. Note that all NixOS packages are provided by the pkgs channel.

environment.systemPackages = with pkgs; [ 
   wget vim 
];

## or

environment.systemPackages = [ pkgs.wget pkgs.vim ];

After changing configuration.nix, rerun nixos-rebuild switch to apply your changes.

Installing packages as a user or ad-hoc

You can alternatively install packages using nix-env rather than the declarative approach described above. Non-root users can also use nix-env which will update the user's profile.

Install: nix-env -iA nixos.wget

Uninstall: nix-env -e wget

Remember that any changes to the environment are versioned automatically into a new generation and can be rolled back through grub or with nix-env --rollback.

System Upgrade

See channels.

Update the channel first

nix-channel --update nixos

Then upgrade all packages

nix-env -u '*'

Automatic updates can be enabled by adding to configuration.nix:

system.autoUpgrade.enable = true;
system.autoUpgrade.allowReboot = true;

Garbage collection

See: https://nixos.org/manual/nix/stable/package-management/garbage-collection.html

Description	Command
To delete all old (non-current) generations of your current profile:	nix-env --delete-generations old
Delete a list of generations	nix-env --delete-generations 10 11 14
Delete generations older than 14 days	nix-env --delete-generations 14d
Clean up nix store (after deleting generations)	nix-store --gc
Delete all old generations of all profiles in /nix/var/nix/profiles	nix-collect-garbage -d

User Management

By default, entries in /etc/passwd and /etc/group are added ad-hoc by the system. You could use useradd/groupadd to modify the system and have them persist (though, this defeats the point of NixOS).

You can ask NixOS to rewrite the /etc/passwd and /etc/group files as required by the system configuration by setting the

users.mutableUsers = false

To have NixOS create the user for you, define the following in the configuration.nix file:

users.users.alice = {
  isNormalUser = true;
  home = "/home/alice";
  description = "Alice Foobar";
  extraGroups = [ "wheel" "networkmanager" ];
  openssh.authorizedKeys.keys = [ "ssh-dss AAAAB3Nza... alice@foobar" ];
};

Re-run nixos-rebuild to apply.

Note: Password is set using passwd and is persistent with the system.

Networking

The default configuration uses dhcpd. A static address can be set for a specific interface.

networking.interfaces.eth0.ipv4.addresses = [ {
  address = "192.168.1.2";
  prefixLength = 24;
} ];

networking.defaultGateway = "192.168.1.1";
networking.nameservers = [ "8.8.8.8" ];

Enable SSH root authentication

services.openssh = {
    enable = true;
    settings.PermitRootLogin = "yes";
    settings.PasswordAuthentication = true;
  };

Tasks

Join FreeIPA domain

FreeIPA is configured using the security/ipa.nix module. For my home environment with the FreeIPA server at ipa.home.steamr.com, this is the config that I used:

security.ipa = {
  enable = true;
  domain = "home.steamr.com";
  realm = "HOME.STEAMR.COM";
  server = "ipa.home.steamr.com";
  cacheCredentials = true;
  offlinePasswords = true;
  basedn = "dc=home,dc=steamr,dc=com";
  certificate = pkgs.fetchurl {
    url = "http://ipa.home.steamr.com/ipa/config/ca.crt";
    sha256 = "1svxgn6mq8v1w8jqpn2jx99jwm1zfcpkzslra4mhb365f0fniw6s";
  };
};


system.activationScripts.binbash = lib.stringAfter [ "stdio" ]
  ''
    # Create the required /bin/bash symlink;  User profiles use /bin/bash as their shell
    mkdir -m 0755 -p /bin
    ln -sfn "${config.system.build.binsh}/bin/bash" /bin/.bash.tmp
    mv /bin/.bash.tmp /bin/bash # atomically replace /bin/bash
  '';

Note the following:

• SHA256 of the server CA certificate was obtained by running: nix-prefetch-url http://$server/ipa/config/ca.crt
• The activation script creates /bin/bash as a symlink to bash. This is necessary because users' shell attribute is set to /bin/bash and the login would fail if it doesn't resolve to a shell. You might be able to do some sssd overrides, but that's super annoying to deal with as the entire sssd config is handled by the security/ipa.nix module and I'd rather not have to override what it's configuring.
• I ran this on a LXC under Proxmox. The hostname was set to 'nx' but the domain for some reason could never be set to 'home.steamr.com' despite my best efforts. (/etc/hosts is correct -- IP resolves to the FQDN then aliases, /etc/resolv.conf has both domain and search set to home.steamr.com, nsswitch.conf seems to be correct in using these files.) My guess here is that there's something not quite right in the LXC environment which is preventing the domain from being set. My work around is to set the hostname to the FQDN (nx.home.steamr.com) and then run ipa-join.

After rebuilding the system, there will be a message that gets printed with instructions to complete the domain join. SSSD will likely fail until you manually join the domain. See the following output:

[root@nixos:~/nix-configs]# nixos-rebuild switch --flake .#nx
warning: Git tree '/root/nix-configs' is dirty
building the system configuration...
warning: Git tree '/root/nix-configs' is dirty
stopping the following units: nscd.service
activating the configuration...
setting up /etc...

    In order to complete FreeIPA integration, please join the domain by completing the following steps:
    1. Authenticate as an IPA user authorized to join new hosts, e.g. kinit admin@HOME.STEAMR.COM
    2. Join the domain and obtain the keytab file: ipa-join
    3. Install the keytab file: sudo install -m 600 krb5.keytab /etc/
    4. Restart sssd systemd service: sudo systemctl restart sssd

Do what it tells you to do:

1. Get a Kerberos ticket for an account with joining capabilities: kinit admin@HOME.STEAMR.COM
2. with the ticket in place, run ipa-join
3. You _should_ now be joined to the domain and there should be a keytab file for the machine. Verify with klist -kte /etc/krb5.conf
4. Restart sssd: systemctl restart sssd

NFS mounts

If you intend to use NFS after enabling FreeIPA (or if you have a krb5.keytab file), NixOS will try to enable the rpcgss module. If you're using NixOS in LXC, the module load will always fail since the rpcgss kernel module isn't where it thinks it is. If you don't use kerberos for mounting NFS, you can just disable the service by adding:

systemd.services.auth-rpcgss-module.enable = false;

If you do need to use kerberos authentication, you will have to add the NFS service to the host in the FreeIPA interface (Identity -> Services -> Add) and then add the service principal to the kerberos keytab file with something like: ipa-getkeytab -s ipa.home.steamr.com -p nfs/nx.home.steamr.com -k /etc/krb5.keytab

Override nixpkgs package version

If you need to override specific versions or attributes that are defined by the nixpkgs repo, you can use overlays to override specific things. In the example below, I override the microsoft-edge package to use the most recent version.

Add the following line to your /etc/configuration.nix file:

nixpkgs.overlays = [ (import /etc/nixos/overlays/overlay.nix) ];

And then create /etc/nixos/overlays/overlay.nix with the following:

self: super:
{
  microsoft-edge = super.microsoft-edge.overrideAttrs (oldAttrs: {
    src = super.fetchurl {
      url = "https://packages.microsoft.com/repos/edge/pool/main/m/microsoft-edge-stable/microsoft-edge-stable_115.0.1901.188-1_amd64.deb";
      sha256 = "0xf2d2wxc2hjxz070nkrbgsszylwhck6x655ginjmh0qm76kf4wr";
    };
  });
}

If you need to get the appropriate hash, use the nix-prefetch-url command.

$ nix-prefetch-url https://packages.microsoft.com/repos/edge/pool/main/m/microsoft-edge-stable/microsoft-edge-stable_115.0.1901.188-1_amd64.deb
path is '/nix/store/kqjnmgrnhgsg56v16rnvv4j4d8fzxlb7-microsoft-edge-stable_115.0.1901.188-1_amd64.deb'
0xf2d2wxc2hjxz070nkrbgsszylwhck6x655ginjmh0qm76kf4wr

If you'd like to update the nixpkgs repo, you'll have to specify the hash in SRI format (which is just the base64 encoded). That can be accomplished using the nix hash command:

$ nix --extra-experimental-features nix-command  hash to-base64 --type sha256 0xf2d2wxc2hjxz070nkrbgsszylwhck6x655ginjmh0qm76kf4wr
mRM3zakYwCptfKWYbiaDnPqv9Vt5WnDA7xIK1rlownU=

Enable IP forwarding

Add the following config:

boot.kernel.sysctl."net.ipv4.ip_forward" = 1;

Troubleshooting

nixos-rebuild: line 13: exec: man: not found

You are using nixos-rebuild incorrectly and it's trying to show you the man page for it. But you don't have man installed, so instead you're getting this error.

The fix is to just figure out what's wrong with your command. You likely forgot to give it the subcommand (such as 'switch'):

[root@nixos:~/nix-configs]# nixos-rebuild --flake .#nx
/run/current-system/sw/bin/nixos-rebuild: line 13: exec: man: not found

## You probably meant to run nixos-rebuild switch
[root@nixos:~/nix-configs]# nixos-rebuild switch --flake .#nx