Development environment setup with flox and direnv

The status quo

When you are working on a project, one of the core requirements would be to choose a programming language to use. If you work alone, as long as everything is reproducible on your development machine, things would be more or less all right. If you need to use other packages, you can install them as you go along, which could be a brew install $package, curl https://example.com/get | sh, apt install $package or anything in between. But so far your project is humming along nicely, nothing to worry about.

It has to work on other machines

The fun starts when you are working in a team, or have to collaborate with other teams on the same project. Let's say the project uses Node.js, you'll have to make sure that everyone working on this project uses the same node version. But this means each person has to do this on their end, there's no central mechanism to ensure that the node version used in this project is standardized. You can use .nvmrc to specify node version, but you'll have to make sure that they have nvm on their machine in the first place.

As for other project dependencies that's not the runtime (node, python, go, rust, etc.) itself, when you install them, by default, latest version will be installed. What if when a feature is created, it was developed on a machine where $package had been installed last year. And when a new developer works on this project and is setting up their dev environment, they install this $package, but the version installed this time is a new version, with breaking changes. At this point, you'll have to hope that you can still contact the original developer so you can obtain the $package version used in this project. If this is not possible, you'll have to test against each version of this $package to find the version that works with this project.

And you need to initialize your workspace

For simple projects you can run a single command to start a process and that's all there is to run a project. But some projects require multiple processes running before you can start working on it.

For example, a backend project with a database, you'll need to spin up a database prior to running a backend server.

A frontend project, if it needs to talk to a backend, you'll also need to spin up a backend server (and probably a database as well).

Or a terraform project talking to a kubernetes cluster with argocd, you'll need to run ssh tunneling to expose kubernetes api server, and kubectl port forwarding to expose argocd.

And these are multiple processes, which means multiple terminal windows, and you need to read the logs for relevant processes your code is talking to.

Environment variables also need to be initialized, and sometimes you'll forget to set it before starting your services. If you work on multiple projects and forget to start a new terminal session, your environment variables can get all jumbled from different projects.

(You also need to tear down these processes once you've finished working on a project as well, otherwise you are wasting compute on unused resources).

Flox and direnv can help you

With flox, you can define packages used in a project. They are pinned by a revision hash, so given the same flox configuration, it will use the same packages, down to the version. You can also define processes attached to a project. Logs are also available, either for a specific process or all processes.

Which means, when you activate a flox shell (flox activate), it will use packages defined in a flox configuration. You can also set processes (flox activate --start-services) to start once the flox shell is activated. This means you don't need to open multiple terminal tabs and start upstream processes. If you need to see logs, flox services logs -f $serviceName or flox services logs -f to see logs for all processes. If you need to restart processes, just run flox services restart.

Example flox services config:

[services]
backend.command = "air"
frontend.command = "cd frontend && yarn install && yarn dev"

The best part is when you exit flox shell, it will automatically tear down the processes used in a project.

As for environment variables, direnv will automatically register variables for current directory. Once you leave this directory, environment variables will be unset.

There is mise, which can do what flox and direnv do. But mise's approach to running processes in the background isn't as smooth as flox's. Namely, there's no clean mechanism to view process logs, you'll have to watch them or pipe them.

Do I really need flox and direnv

When you are working on a project and you are perfectly content with:

• initializing environment variables
• spin up a database
• start backend server
• start frontend server
• packages whack-a-mole to find out why it works on your machine, but doesn't work on your teammate's

No, you don't need flox and direnv.

But that's four commands already, and three terminals (or two, if you run a database in the background) need to be left open.

But with flox and direnv, to achieve all this, you only need to run flox activate --start-services. Less time and less cognitive load, what's not to love?