Xfile: a better way to work with files and directories in Elixir

Necessity is the mother of invention… or in the case of software, need begets packages. And packages require names, preferably something cool that maybe invokes the supernatural and/or throwbacks to ’90s television. So here’s my little PSA about Xfile, my latest Elixir package.

The truth is out there… if you can find it

Recursively List Files

I kept encountering shortcomings when using Elixir’s built-in File module. One of the bits of boilerplate code that I added to a few different projects was functionality that let me recursively list files stored in a nested directory structure. In adherence to DRY principles, I turned that boilerplate into its own package. Now I can easily traverse deeply nested directories in a uniform way:

iex(1)> Xfile.ls!("deps") 
|> Enum.to_list()
["deps/metrics/.hex", "deps/metrics/LICENSE", "deps/metrics/.fetch",
 "deps/metrics/README.md", "deps/metrics/hex_metadata.config",
 "deps/metrics/rebar.lock", ...]

A couple things to point out:

1. Xfile returns its results as a lazily-evaluated stream. This has important performance benefits if you ever wade into folders with indecent numbers of files. You just have to enumerate the stream (here, this is done by converting it to a list via Enum.to_list/1 )
2. Xfile returns the full path segment, e.g. dir/file.txt whereas the built-in File.ls/1 returns just the basename, e.g. file.txt . When I used the File.ls/1, I often found myself needing to concatenate the parent directory onto the results before I could do anything useful with them.
3. Xfile.ls/2 supports recursively listing files, so I can easily peer into deeply nested directory structures without re-inventing wheels.

Having the above functionality in a reusable package was my MVC, but I realized there was some room for some useful scope creep. (Is “useful scope creep” a trigger phrase for project managers?). Adding a few bits of polish here made the package even more handy as I found myself needing to deal with intermediary file byproducts of long-running ETL processes. And here’s where Xfile.ls/2 gets more horsepower:

4. Xfile.ls/2 supports a :filter option used to evaluate whether or not the given file should be included in the results. The :filter can be a regular expression OR an arity-1 function. To be fair, you could always filter the resulting enumerable AFTER listing the files, but filtering here in the same breath allows us to traverse the list only once.

Grep File Contents

One of my favorite bash commands is grep -rl — I usually rely on it instead of my IDE when I’m searching a project. Now I can return a list of files that contain any line that matches the pattern:

iex> Xfile.grep_rl("raise Error", "projekt", recursive: true) 
|> Enum.to_list()
      [
        "projekt/lib/foo/service.ex",
        "projekt/lib/db.ex"
      ]

Since this exists in code, the pattern can be more than a simple string: you can supply your own arity-1 function to receive each line in the file.

Although I use grep -rl more, I also included a grep function inside Xfile for searching single files and returning the matching lines. This can be useful for cherry-picking relevant lines from a file.

Head, Tail, and counting lines

Finally, I will mention a couple other convenience functions that can spare you from writing more boilerplate code after researching solutions on Stackoverflow: head , tail , and line_count .

Xfile.head/2 will return the first n lines of the given file:

iex> Xfile.head(".gitignore", 3) 
|> Enum.to_list()
      [
        "# The directory Mix will write compiled artifacts to.\n",
        "/_build/\n",
        "\n"
      ]

Xfile.tail/2 does the same, but for returning the last n lines from a file, and Xfile.line_count/1 is more or less an implementation of wc -l .

This package has made my life easier this week, and I hope it’s useful to the community. Happy coding!