Using Dotenvy in Elixir Releases

The last article I wrote about Dotenvy didn’t leave much room for demonstrating specific use-cases. So in this article I wanted to take one on: using Dotenvy with Elixir releases.

When you create an Elixir release, you create a self-contained directory that consists of your application code, all of its dependencies, plus the whole Erlang Virtual Machine (VM) and runtime. Releases are frequently used when you are deploying production code to a server. The trick is ensuring that they can still read the configuration on the environment where they are running (and not from the environment where they were built).

As a quick reminder, the general trick with using Dotenvy is specifying a list of env files that your runtime.exs will attempt to load, and if you do it properly, you can set specific values for each environment by leveraging the config_env() function. Your config runtime.exs might look like this:

import Config
import Dotenvy
source!([
  "config/.env",
  "config/.#{config_env()}.env",
  "config/.#{config_env()}.local.env"
])
config :foo,
  goalkeeper: env!("GOALKEEPER", :string), 
  fullback: env!("FULLBACK", :string), 
  sweeper: env!("SWEEPER", :string)

In the dev environment, for example, this would cause following files to be loaded in order: config/.env , config/dev.env , config/dev.local.env and if they were present, they would be parsed and their values loaded.

There are a couple important things to remember here:

1. Make sure you put the last pattern into your .gitignore (e.g. *.local.env) because that will allow values to be overridden.
2. The config/ directory not included in a release! (Keep reading for how to deal with this)
3. The values required by the app can be set globally on the server (e.g. in shell profile) or in any of the files listed and passed to the Dotenvy.source!/2 function. As long as the values have been declared somewhere, the app can bootstrap.

You might think that the above is sufficient to make your app work within the context of a release. You can build the app and run a command, e.g. _build/dev/rel/foo/bin/foo start_iex and it will start up on your local machine. However, if you start sniffing around, you will realize that it is still reading the .env files from your repository’s config/ directory (!!!), and that isn’t included in a release by default! Unless you are planning to manually set all ENV vars (e.g. inside something like Heroku’s dashboard), this strategy won’t work because the needed files will not be present.

Including your .envs inside a release

There are ways to package up our .env files into a release and reference them on the target deployment machine. The trick revolves around two things:

• the RELEASE_ROOT environment variable which gets declared by your build artifact (e.g. _build/dev/rel/myapp/bin/myapp — in it, you can see several important environment variables set)
• the overlays option inside your mix.exs releases setting.

The RELEASE_ROOT variable is set automatically when you are running your app as a release, and it won’t be set when you are running your app normally (e.g. via iex during normal development). So we can use this variable to adjust the paths of where we want Dotenvy to look for our .env files.

Before we edit our runtime.exs let’s take a look at the overlays option. Inside our mix.exs we can define a releases key for our app:

releases: [
  myapp: [
    include_executables_for: [:unix],
    steps: [:assemble, :tar],
    overlays: ["envs/"],
    path: "_build/rel"
  ]
]

In this example, we are going to move our .env files into a dedicated directory named envs/ — this might seem strange, but it will be less confusing when we overlay the directory contents into the release. How does this work? When you run mix release the contents of the envs/ directory will be “overlaid” (i.e. copied) to the root of the release.

Now when you run mix release you can see that your .env files get copied (i.e. “overlayed”) into the _build/rel/ folder. That’s great, but things won’t work until you update your runtime.exs

Note that the pathoption has also been set in this example: this means all environments build to the exact same location (dev, prod, etc). This may not be appropriate for your app, but if you have moved all your configuration into the runtime.exs and your app is the same in all environments, then you can save a bit of time by avoiding re-compiling for each environment.

To make the configuration work with the new location during regular development and within a release, we need to update the runtime.exs so it looks in the new envs/ directory during normal development OR to the RELEASE_ROOT when it is running as a release:

# For local development, read env files inside the envs/ dir;
# for releases, read them at the RELEASE_ROOT
config_dir_prefix = System.fetch_env("RELEASE_ROOT")
  |> case do
    :error -> "envs/"
    {:ok, value} ->
      IO.puts("Loading dotenv files from #{value}")
      "#{value}/"
  end
source!([
  "#{config_dir_prefix}.env",
  "#{config_dir_prefix}.#{config_env()}.env",
  "#{config_dir_prefix}.#{config_env()}.local.env"
])

Now you can edit the .env files within the envs/ directory during normal development and testing. When you build a release via mix release , a copy of those files is overlaid into the repository root. You can edit the values in the copy at _build/rel/.envand then verify that the release is reading those values at runtime, e.g. from your terminal:

_build/rel/bin/foo start_iex
iex> Application.get_env(:foo, :goalkeeper)
"Glorb"

Your CI/CD code can now write a version of the .env when it deploys the release (e.g. .prod.local.env) and the app will read the values declared in the file.

There is a sample repository which demonstrates this strategy: https://github.com/fireproofsocks/mermaid-demo (this is the same repo I used to demonstrate how to add Mermaid Charts to Elixir documentation).

Feel the release!