Managing mutable files in NixOS
Some weeks ago, I’ve made a post describing how to combine traditional dotfiles and NixOS config with Nix flakes. That seems all well and good but it comes with a few problems.
-
Changes from the dotfiles have to be pushed first to the remote repo.
-
Then, the dotfile flake input have to be updated. After pushing changes from the dotfiles, you have to update it as depicted from Workflows and its caveats. While managing with flakes is less tedious than managing it with fetchers, you’re essentially just reducing the source of updates from two to one.
While it is reproducible, it’s a tedious process especially if you need immediate changes from your dotfiles.
One of the responses I got from the aforementioned post is an alternative solution for managing dotfiles.
I’m more likely to use home-manager’s mkOutOfStoreSymlink if I find myself in situations where I need non-generational or mutable file content. I have a trivial personal HM module that git-clones things to a desired path as part of the activation script if they’re absent and then manage them out of band from then on.
Long story short, I tried this approach and I found it to be a better solution overall. It is more flexible and lends itself as a great solution for managing mutable files — files that are not meant to be managed by Nix. This also reduces the things I need to do post-installation which is already contained in a script so that’s a nice benefit. Anyways, here’s my take on the posted solution.
A better way to manage traditional dotfiles in home-manager
As hinted from the quoted statement, mkOutOfStoreSymlink is a home-manager function that accepts a path string and returns a derivation.
This derivation contains a builder script that links the given path to a store path.
Using it should be simple.
The following listing simply links my dotfiles located on /home/foo-dogsquared/dotfiles and creates a path on the Nix store.
mkOutOfStoreSymlink "/home/foo-dogsquared/dotfiles"
This pretty much allows us to interact with various options from home-manager that normally accepts a store path.
In my case, I mainly use it for linking various files with home.file.<name>.source, xdg.configFile.<name>.source, and so forth.
To give some more context, here’s an example usage of the function with my use case.
home.nix, this time with mkOutOfStoreSymlink instead of flakes{ config, lib, pkgs, ... }:
let
dotfiles = config.lib.file.mkOutOfStoreSymlink "/home/foo-dogsquared/dotfiles";
in
{
# Putting the dotfiles in their rightful place.
xdg.configFile = {
doom.source = "${dotfiles}/emacs";
wezterm.source = "${dotfiles}/wezterm";
nvim.source = "${dotfiles}/nvim";
};
}Compared to the approach of making the dotfiles as a flake input, this reduces the reproducibility of our home-manager configuration a little bit. Instead of fully including the dotfiles, we only assume we have the dotfiles at the given location. However, this does remove the workflow of managing the flake input and its caveats altogether.
You don’t have to do nix flake update or anything else in your NixOS config and manage them separately.
We’re compromising reproducibility with this but it is worth it considering I want the changes immediately.
Adding a declarative interface for fetching mutable files
Take note with the above method, we did reduce from fully including the dotfiles to only assuming we have the dotfiles. I still want to include the dotfiles declared somewhere in the configuration. The closest we’ll ever get is to create a module that accepts a list of files to download and put it in the filesystem which is exactly what I did. Anyhoo, here’s how I would use the imaginary module.
{
home.mutableFile = {
"${config.xdg.userDirs.documents}/dotfiles" = {
url = "https://github.com/foo-dogsquared/dotfiles.git";
type = "git";
};
"${config.xdg.userDirs.documents}/top-secret" = {
url = "https://example.com/file.zip";
type = "fetch";
};
};
}
This module is meant to be used for fetching mutable files.
It would have different methods for fetching the file indicated by the type attribute.
For the initial version of this module, we’ll consider two use cases: cloning the Git repos and downloading the file.
Let’s first create the skeleton for the module.
modules/home-manager/fetch-mutable-files.nix{ config, options, lib, pkgs, ... }:
let
cfg = config.home.mutableFiles;
file = { config, name, ... }: {
};
in
{
options.home.mutableFile = lib.mkOption {
type = with lib.types; attrsOf (submodule file);
default = { };
description = lib.mkDoc ''
An attribute set of mutable files and directories to be fetched into the
home directory.
'';
example = lib.literalExpression ''
"''${config.xdg.userDirs.documents}/dotfiles" = {
url = "https://github.com/foo-dogsquared/dotfiles.git";
type = "git";
};
"''${config.xdg.userDirs.documents}/top-secret" = {
url = "https://example.com/file.zip";
type = "fetch";
};
'';
};
config = {
systemd.user.services.fetch-mutable-files = {
};
};
}
We have yet to define certain parts including what each attribute could contain.
Each of the attribute in the home.mutableFile.<name> expects at least two attributes: the URL to be downloaded and the download method.
The file should only be downloaded if the path doesn’t exist.
|
At this point, updates to the code are shown as diffs.
It is meant to be used with |
diff --git b/modules/home-manager/fetch-mutable-files.nix a/modules/home-manager/fetch-mutable-files.nix
index 9c66e05..aa00cac 100644
--- b/modules/home-manager/fetch-mutable-files.nix
+++ a/modules/home-manager/fetch-mutable-files.nix
@@ -2,12 +2,47 @@
let
cfg = config.home.mutableFiles;
- file = { config, name, ... }: {
+ file = baseDir: { config, name, ... }: {
+ options = {
+ url = lib.mkOption {
+ type = lib.types.str;
+ description = lib.mkDoc ''
+ The URL of the file to be fetched.
+ '';
+ example = "https://github.com/foo-dogsquared/dotfiles.git";
+ };
+
+ path = lib.mkOption {
+ type = lib.types.str;
+ description = lib.mkDoc ''
+ Output path of the resource relative to ${baseDir}.
+ '';
+ default = name;
+ apply = p:
+ if lib.hasPrefix "/" p then p else "${baseDir}/${p}";
+ };
+
+ type = lib.mkOption {
+ type = lib.types.enum [ "git" "fetch" ];
+ description = lib.mkDoc ''
+ Type that configures the behavior for fetching the URL.
+
+ This accept only certain keywords.
+
+ - For `fetch`, the file will be fetched with `curl`.
+ - For `git`, it will be fetched with `git clone`.
+
+ The default type is `fetch`.
+ '';
+ default = "fetch";
+ example = "git";
+ };
+ };
};
in
{
options.home.mutableFile = lib.mkOption {
- type = with lib.types; attrsOf (submodule file);
+ type = with lib.types; attrsOf (submodule (file config.home.homeDirectory));
default = { };
description = lib.mkDoc ''
An attribute set of mutable files and directories to be fetched into the
Take note we also added the path attribute that comes with a function to handle the path.
It’s for cleanly passing absolute paths and relative paths when it needs to.
{
# Absolute paths should be acceptable.
home.mutableFile."${config.xdg.userDirs.documents}/top-secret" = { };
home.mutableFile."${config.xdg.configHome}/doom" = { };
home.mutableFile."${config.home.homeDirectory}/hello" = { };
home.mutableFile."/home/foo-dogsquared/writings" = { };
# So does relative paths...
home.mutableFile."dotfiles" = { };
home.mutableFile."library" = { };
}With the interface done, we can then proceed with the implementation which is just a shell script managed by systemd. Let’s first build the systemd service before we proceed with the shell script.
|
Remember, we’re using systemd to manage the service.
The service is run in a completely new environment and isn’t in a shell with programming features like Bash and zsh.
This means you cannot run the following command on |
diff --git b/modules/home-manager/fetch-mutable-files.nix a/modules/home-manager/fetch-mutable-files.nix
index aa00cac..3d75414 100644
--- b/modules/home-manager/fetch-mutable-files.nix
+++ a/modules/home-manager/fetch-mutable-files.nix
@@ -63,6 +63,25 @@ in
config = {
systemd.user.services.fetch-mutable-files = {
+ Unit = {
+ Description = "Fetch mutable files from home-manager";
+ After = [ "default.target" "network-online.target" ];
+ Wants = [ "network-online.target" ];
+ };
+
+ Service = {
+ # We'll assume this service will download lots of files. We want the
+ # temporary files to only last along with the service.
+ PrivateUsers = true;
+ PrivateTmp = true;
+
+ Type = "oneshot";
+ RemainAfterExit = true;
+ # TODO: Complete this
+ ExecStart = "";
+ };
+
+ Install.WantedBy = [ "default.target" ];
};
};
}
Creating the shell script should be trivial.
We could generate the entire script by iterating each of the file from home.mutableFile.<name> and mapping the methods from it.
Here’s one way to let Nix generate our shell script featuring writeShellScript builder.
diff --git b/modules/home-manager/fetch-mutable-files.nix a/modules/home-manager/fetch-mutable-files.nix
index 3d75414..c3e349b 100644
--- b/modules/home-manager/fetch-mutable-files.nix
+++ a/modules/home-manager/fetch-mutable-files.nix
@@ -77,8 +77,22 @@ in
Type = "oneshot";
RemainAfterExit = true;
- # TODO: Complete this
- ExecStart = "";
+ ExecStart = let
+ curl = "${lib.getBin pkgs.curl}/bin/curl";
+ git = "${lib.getBin pkgs.curl}/bin/git";
+ fetchCmds = lib.mapAttrsToList (file: value:
+ let
+ inherit (value) type;
+ path = lib.escapeShellArg value.path;
+ url = lib.escapeURL value.url;
+ in ''
+ ${lib.optionalString (type == "git") "[ -d ${path} ] || ${git} clone ${url} ${path}"}
+ ${lib.optionalString (type == "fetch") "[ -d ${path} ] || ${curl} ${url} --output ${path}"}
+ '') cfg;
+ shellScript = pkgs.writeShellScript "fetch-mutable-files" ''
+ ${lib.concatStringsSep "\n" fetchCmds}
+ '';
+ in builtins.toString shellScript;
};
Install.WantedBy = [ "default.target" ];|
For those who want a complete version of the module directly without applying all of the above patches, you can see it with this link. |
With the module being complete for the most part, we just have to include it to our home-manager configuration…
flake.nix{
outputs = { nixpkgs, home-manager, ... }@inputs: {
homeConfigurations.foodogsquared = home-manager.lib.homeManagerConfiguration {
modules = [
./modules/home-manager/fetch-mutable-files.nix
./home.nix
];
};
};
}…and finally use it. Like I said previously, the nice thing with this module for me is allowing me beyond fetching my dotfiles. I could, for example, fetch Doom Emacs alongside my home-manager configuration. Very nice!
home.nix{ config, lib, pkgs, ... }:
let
dotfiles = config.lib.file.mkOutOfStoreSymlink config.home.mutableFile."dotfiles".path;
in
{
home.mutableFile = {
"dotfiles" = {
url = "https://github.com/foo-dogsquared/dotfiles.git";
type = "git";
};
"${config.xdg.configHome}/emacs" = {
url = "https://github.com/doomemacs/doomemacs";
type = "git";
};
};
# Putting the dotfiles in their rightful place.
xdg.configFile = {
doom.source = "${dotfiles}/emacs";
wezterm.source = "${dotfiles}/wezterm";
nvim.source = "${dotfiles}/nvim";
};
}Mutable files in NixOS
So far, we only manage them mutable files in home-manager.
I cannot find an equivalent in NixOS but it should be pretty trivial to recreate it especially that the things that made mkOutOfStoreSymlink possible is readily available in nixpkgs.
All we have to do is to recreate them.
For this case, we’ll use the runCommandLocal builder typically used for cheap commands.
This also what mkOutOfStoreSymlink uses in the source code.
let
path = lib.escapeShellArg "/etc/dotfiles";
in
pkgs.runCommandLocal "nixos-mutable-file-${builtins.baseNameOf path}" { } ''ln -s ${path} $out''
Similarly, this can be used for various NixOS options that normally accepts a store path (e.g., environment.etc.<name>.source).
For example, if you have a minimal i3 setup that you want to link from a non-NixOS-managed folder, all you have to do is to link it from the dotfiles.
{ config, lib, pkgs, ... }:
let
path = lib.escapeShellArg "/etc/dotfiles";
dotfiles = pkgs.runCommandLocal "nixos-mutable-file-${builtins.baseNameOf path}" { } ''ln -s ${path} $out'';
in
{
enviroment.etc = {
"i3".source = "${dotfiles}/i3";
"polybar".source = "${dotfiles}/polybar";
"zathurarc".source = "${dotfiles}/zathura/zathurarc";
};
}