DevOps with Dhawos

Cloud, infrastructure and programming

Using Nix or Docker for reproducible Development Environments

Introduction

Having an efficient and robust CI/CD pipeline is one of the keys to developing high-quality services. However, setting up such a system is not always straightforward, and maintaining it over time is even more challenging. One of the main reasons is the difficulty in achieving a consistent execution environment between the CI system and developers’ local machines.

Unfortunately, such systems are sometimes underdeveloped because they are not always perceived as worthwile. Yet, without them, it is impossible to deliver a service under good conditions and at a sustained pace.

In my view, the rise of GitHub Actions is a symptom of these issues. When you think about it, the promise of GitHub Actions is to spend as little time as possible on your CI by maximizing the use of actions from the marketplace. The promise is enticing, and it’s true that the catalog is massive. But as soon as you dig a little deeper, you realize that this paradigm can quickly lead to several problems.

Notably:

In the rest of this article, we will try to implement best practices to limit these problems while still enjoying the benefits of GitHub Actions.


The Scenario

Let’s set a goal starting from a simple application: a web server in Go.

To avoid limiting ourselves to a simple “hello-world”, we will add a third-party library: templ for HTML rendering. If you are not familiar with templ, don’t worry, it’s not the core of the subject. Just remember that this tool requires running a command to generate code before building our application.

From there, we want to be able to:

But we will impose several constraints to keep our CI/CD maintainable:

Let’s get started!


Code Linting

Let’s start with this aspect, which will be the simplest. If you are somewhat familiar with GitHub Actions, you probably know that it can be done very simply. We will use the golangci-lint tool, an excellent tool that checks whether our code complies with a number of standards, such as not ignoring errors.

We just need to make sure our code is generated via templ generate beforehand.

on:
  push:
    branches:
      - main

jobs:
  codegen:
    name: "Ensure code is generated"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version-file: "go.mod"
      - name: Setup templ
        run: go install github.com/a-h/templ/cmd/templ@latest
      - name: "Ensure code is generated"
        run: scripts/codegen.sh
  lint:
    name: "Lint code"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version-file: "go.mod"
      - name: golangci-lint
        uses: golangci/golangci-lint-action@v9

The honeymoon with GitHub Actions is at its peak: the code will be linted on every commit with just 15 lines of YAML. Everything seems perfect…

Except it’s not. Several issues remain:


Let’s Improve Things

We can already do better without introducing new tools. To do this, we need to “pin” our actions to specific commits. This is actually a security recommendation from GitHub Actions, but it seems to me that it is rarely implemented in practice.

The reference is here: “Pin actions to a full-length commit SHA”.

Here’s what it looks like after this change:

name: Run workflow with pinned dependencies
on:
  push:

jobs:
  codegen:
    name: "Ensure code is generated"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
        with:
          go-version-file: "go.mod"
      - name: Setup templ
        run: go install github.com/a-h/templ/cmd/templ@latest
      - name: "Ensure code is generated"
        run: scripts/codegen.sh
  lint:
    name: "Lint code"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
        with:
          go-version-file: "go.mod"
      - uses: golangci/golangci-lint-action@1e7e51e771db61008b38414a730f564565cf7c20 # v9.2.0

Here, we protect ourselves against a major attack vector. However, a few points should be noted:

Let’s take the opportunity to add tests. Our workflow should now look like this:

name: Run workflow with pinned dependencies
on:
  push:

jobs:
  codegen:
    name: "Ensure code is generated"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
        with:
          go-version-file: "go.mod"
      - name: Setup templ
        run: go install github.com/a-h/templ/cmd/templ@latest
      - name: "Ensure code is generated"
        run: scripts/codegen.sh
  lint:
    name: "Lint code"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
        with:
          go-version-file: "go.mod"
      - uses: golangci/golangci-lint-action@1e7e51e771db61008b38414a730f564565cf7c20 # v9.2.0
  test:
    name: "Test code"
    runs-on: ubuntu-latest
    needs:
      - codegen
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6.4.0
        with:
          go-version-file: "go.mod"
      - name: "Tests"
        shell: bash
        run: go test ./...

As you can see, we will download the same version of Go three times in a row, which is a waste. Additionally, it is difficult to test this CI locally under the same conditions. The two main problems with this approach are therefore:

If these two points are not a problem for you, you probably don’t need the practices we will see next. If it works for you and you have no trouble maintaining your CI, it’s best not to complicate it.

This is not my case. Having worked on CIs many times, I believe that being able to modify and test scripts locally is a valuable help that will save you a lot of time. It will also allow you to share the same development environment with all contributors, so why deprive yourself?


The Quest for Reproducibility

If we want to be able to test our CI locally and avoid re-downloading all or part of our toolchain, we need to solve a major problem: setting up a reproducible environment, regardless of the machine running it.

Until now, my immediate answer to this question was Docker, but I recently tried to tame Nix, which has a slightly steeper learning curve. These two technologies allow us to address this issue in completely different ways, each with its own advantages and disadvantages.

Rather than diving into an abstract comparison, let’s try to adapt our CI using these two tools.


Docker

Docker is now widely used, so I won’t go into detail about what it is and in what other cases it can be useful. Let’s just recall that a Docker image produces a minimal filesystem that allows you to run programs with only their dependencies.

In our case, here’s what such an image might look like:

FROM golang@sha256:fcdb3e42c5544e9682a635771eac76a698b66de79b1b50ec5b9ce5c5f14ad775 # golang:1.26.2
ARG SCALEWAY_CLI_VERSION="2.54.0"
ARG GOLANGCI_LINT_VERSION="2.11.4"

RUN apt-get update && apt-get install -y wget

RUN wget "https://github.com/scaleway/scaleway-cli/releases/download/v${SCALEWAY_CLI_VERSION}/scaleway-cli_${SCALEWAY_CLI_VERSION}_linux_amd64" -O /tmp/scaleway-cli && \
    chmod +x /tmp/scaleway-cli && \
    mv /tmp/scaleway-cli /bin/scw && \
    scw version
RUN wget "https://github.com/golangci/golangci-lint/releases/download/v${GOLANGCI_LINT_VERSION}/golangci-lint-${GOLANGCI_LINT_VERSION}-linux-amd64.tar.gz" -O /tmp/golangci-lint.tar.gz && \
    tar -xf /tmp/golangci-lint.tar.gz -C /tmp && \
    mv /tmp/golangci-lint-${GOLANGCI_LINT_VERSION}-linux-amd64/golangci-lint /tmp/golangci-lint && \
    chmod +x /tmp/golangci-lint && \
    mv /tmp/golangci-lint /bin/golangci-lint && \
    golangci-lint --version
RUN go install github.com/a-h/templ/cmd/templ@v0.3.1001 && \
    templ version
RUN rm -rf /tmp/* && mkdir /src
WORKDIR /src

Let’s take the time to break down this image. We mainly have three RUN steps that will allow us to download and install the binaries needed for our CI: golangci-lint, templ, and scaleway-cli (used for deployment but this is beyond our scope here). Go and its toolchain will already be installed because we start from a base image that contains them: golang:1.26.2. We also need to pin that base image just to be sure to pull the same everytime.

Then, all you have to do is build this Docker image:

docker build . -t ghcr.io/dhawos/nix-docker-blog-post-ci:v0.3.0 -f Dockerfile.ci

Once this command is executed, anyone running a container from this image will have exactly the same set of tools. Indeed, an image corresponds to a “snapshot” of a filesystem, which is quite similar to a virtual machine image.

However, this filesystem is isolated from the host machine and therefore does not contain the sources of our application. We could put them there, but that would require recreating a Docker image for every small change, which would be very slow. For this use case, it is better to mount a volume in our container. Locally, this would look like:

docker run -it -v $(pwd):/src ghcr.io/dhawos/nix-docker-blog-post-ci:v0.3.0

The -v option allows us to map a directory from our host machine to a location in our container (here /src). From there, we will be able to run all the commands needed to work on the project.

However, at this point, we can only use it locally. For everyone to be able to actually use the same image, it must be pushed to a registry:

docker push ghcr.io/dhawos/nix-docker-blog-post-ci:v0.3.0

Then, we can use it directly in our GitHub workflows, provided the registry is public. If it is not, you will need to manage authentication so that GitHub can pull the relevant image.

name: Run workflow with docker
on:
  push:

env:
  IMAGE_TAG: ${{ github.sha }}-nix

permissions:
  contents: read
  id-token: write

jobs:
  codegen:
    name: "Ensure code is generated"
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/dhawos/nix-docker-blog-post-ci:v0.3.0
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - run: scripts/codegen.sh
  lint:
    name: "Lint code"
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/dhawos/nix-docker-blog-post-ci:v0.3.0
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - run: golangci-lint run
  test:
    name: "Test code"
    runs-on: ubuntu-latest
    needs:
      - codegen
    container:
      image: ghcr.io/dhawos/nix-docker-blog-post-ci:v0.3.0
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - run: go test ./...

Here, the checkout step is what mounts the sources in the container; it’s a bit of GitHub Actions magic, but it works.


Nix

Now let’s see how to do the same thing with Nix.

First, unlike Docker, Nix is a package manager. So these are two fundamentally different technologies. Nix’s proposition is to guarantee the reproducibility of environments.

To provide these guarantees, Nix uses lockfiles and checksums so that every package download, performed from the same files, yields the same result. Nix comes with its own declarative language, which can be intimidating (it was for me).

In our case, we mainly want to set up an environment—a shell, to be precise—with the programs installed, available, and in the right version.

For this, we can use Nix flakes. You can think of a flake as a “function” that takes one or more inputs and can produce one or more outputs. I realize this is very abstract, but we will clarify. For example, in our case, we will only have one input, nixpkgs, which you can think of as a repository containing the packages we can install. It’s a simple GitHub repository.

inputs = {
  nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
};

And we will also only have one output: a shell containing the desired binaries. For this, Nix provides a simple function: mkShell.

pkgs = import nixpkgs { inherit system; };
shell = pkgs.mkShell {
  packages = [
    pkgs.go
    pkgs.golangci-lint
    pkgs.templ
    pkgs.scaleway-cli
  ];
};

If we want to be able to use this flake on macOS machines as well as Linux machines or servers, we will need to complicate our code a bit, but let’s do it step by step.

First, let’s define the list of systems we want to support:

systems = [
  "x86_64-linux"
  "aarch64-darwin"
];

Before going further, two things need to be clarified about the Nix language:

  1. You can declare variables using the let {} in {} construct. Variables declared in the let block will be known in the in block. Example:
let
  h = "Hello";
  X = "World";
in
{
  helloWorld = h + " " + X;
}

Result: { helloWorld = "Hello World"; }

  1. To apply a function to all elements of a list, you can use the map function. Example:
builtins.map (x: x*2) [1 2 3]

Result: [ 2 4 6 ]

Here, we apply the function f(x) = x*2 to the list [1,2,3], which returns the list [2,4,6].

With that clarified, let’s return to our subject. We will need to define a shell for each system we want to support. Here’s what it will look like:

builtins.map (system: let
  pkgs = import nixpkgs { inherit system; };
  shell = pkgs.mkShell {
    packages = [
      pkgs.go
      pkgs.golangci-lint
      pkgs.templ
      pkgs.scaleway-cli
    ];
  };
in {
  name = system;
  value = {
    default = shell;
  };
}) systems

At a high level, this function will return something like:

[
  {
    name = "x86_64-linux",
    value = {
      default = $shell
    }
  },
  {
    name = "aarch64-darwin",
    value = {
      default = $shell
    }
  }
]

Then, we will apply the listToAttrs function, which transforms a list into an attribute set (the Nix term for what we call dictionaries in Python or maps in Go). The result will look something like:

{
  "x86_64-linux": {
    default = $shell
  }
  "aarch64-darwin": {
    default = $shell
  }
}

This is the kind of object that Nix expects to create a shell. All you have to do is associate this value with the devShells key for Nix to understand that we are defining development environments.

Putting it all together, here’s what we get:

// flake.nix
{
  description = "Development environments for this project";

  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs?ref=nixos-unstable";
  };

  outputs = {nixpkgs, ...}: let
    systems = [
      "x86_64-linux"
      "aarch64-darwin"
    ];
  in {
    devShells = builtins.listToAttrs (
      builtins.map (system: let
        pkgs = import nixpkgs { inherit system; };
        shell = pkgs.mkShell {
          packages = [
            pkgs.go
            pkgs.golangci-lint
            pkgs.templ
            pkgs.scaleway-cli
          ];
        };
      in {
        name = system;
        value = {
          default = shell;
        };
      }) systems
    );
  };
}

If the syntax is not yet entirely clear, don’t worry—you need to get your hands dirty to learn it. But it’s worth it, because then all you have to do is run this command:

nix develop
$ which go
/nix/store/ckcq2mj8zk0drhaaacy6mp9d924hnr4m-go-1.26.1/bin/go
$ which golangci-lint
/nix/store/j4w0z1lgrs4fdds7npk2bnby5ks612cf-golangci-lint-2.11.4/bin/golangci-lint
$ which scw
/nix/store/mn077hkqy1bs3dwxkvxi15l0k7vz03mk-scaleway-cli-2.54.0/bin/scw
$ which templ
/nix/store/v4qj3mml7lyps2nicavbys97w56dxvjy-templ-0.3.1001/bin/templ

After downloading the binaries if they are not already present, you are immersed in a shell with all the tools you need to work. Two things should be immediately clarified:

To use it in CI, we need to adapt our workflows a bit. Here’s what it looks like:

name: Run workflow with nix
on:
  push:

env:
  IMAGE_TAG: ${{ github.sha }}-nix

jobs:
  codegen:
    name: "Ensure code is generated"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - uses: cachix/install-nix-action@616559265b40713947b9c190a8ff4b507b5df49b # v31.10.4
      - uses: DeterminateSystems/magic-nix-cache-action@565684385bcd71bad329742eefe8d12f2e765b39 # v13
      - shell: "nix develop -c bash -e {0}"
        run: scripts/codegen.sh
  lint:
    name: "Lint code"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - uses: cachix/install-nix-action@616559265b40713947b9c190a8ff4b507b5df49b # v31.10.4
      - uses: DeterminateSystems/magic-nix-cache-action@565684385bcd71bad329742eefe8d12f2e765b39 # v13
      - shell: "nix develop -c bash -e {0}"
        run: golangci-lint run
  test:
    name: "Test code"
    runs-on: ubuntu-latest
    needs:
      - codegen
    steps:
      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
      - uses: cachix/install-nix-action@616559265b40713947b9c190a8ff4b507b5df49b # v31.10.4
      - uses: DeterminateSystems/magic-nix-cache-action@565684385bcd71bad329742eefe8d12f2e765b39 # v13
      - shell: "nix develop -c bash -e {0}"
        run: go test ./...

The only action left here is the one that installs Nix in GitHub runners, as it is not available by default. To avoid re-downloading everything each time, it is also necessary to set up a cache, which the DeterminateSystems/magic-nix-cache-action action does automatically.


Conclusion

I hope I have shown you how to set up a reliable, maintainable CI/CD pipeline thanks to its reproducibility, and secure by avoiding re-downloading everything, which limits a good portion of supply-chain attacks.

The goal of this article is not to say that Docker is better than Nix or vice versa, but rather to present two ways of addressing the same issue. If we try to summarize the implications of using one or the other, here’s what we could say:

If you want to go further, the source code related to this article is available here: https://github.com/dhawos/nix-docker-blog-post. Feel free to browse it.

Whether this article was useful to you or you noticed errors or inaccuracies, don’t hesitate to let me know on my Bluesky.

Thanks for reading and see you soon!