Xfile: a better way to work with files and directories in Elixir
Necessity is the mother of invention… or in the case of software, need begets packages. And packages require names, preferably something cool that maybe invokes the supernatural and/or throwbacks to ’90s television. So here’s my little PSA about Xfile, my latest Elixir package.
Recursively List Files
I kept encountering shortcomings when using Elixir’s built-in
File module. One of the bits of boilerplate code that I added to a few different projects was functionality that let me recursively list files stored in a nested directory structure. In adherence to DRY principles, I turned that boilerplate into its own package. Now I can easily traverse deeply nested directories in a uniform way:
|> Enum.to_list()["deps/metrics/.hex", "deps/metrics/LICENSE", "deps/metrics/.fetch",
A couple things to point out:
Xfilereturns its results as a lazily-evaluated stream. This has important performance benefits if you ever wade into folders with indecent numbers of files. You just have to enumerate the stream (here, this is done by converting it to a list via
Xfilereturns the full path segment, e.g.
dir/file.txtwhereas the built-in
File.ls/1returns just the basename, e.g.
file.txt. When I used the
File.ls/1, I often found myself needing to concatenate the parent directory onto the results before I could do anything useful with them.
Xfile.ls/2supports recursively listing files, so I can easily peer into deeply nested directory structures without re-inventing wheels.
Having the above functionality in a reusable package was my MVC, but I realized there was some room for some useful scope creep. (Is “useful scope creep” a trigger phrase for project managers?). Adding a few bits of polish here made the package even more handy as I found myself needing to deal with intermediary file byproducts of long-running ETL processes. And here’s where
Xfile.ls/2 gets more horsepower:
Xfile.ls/2 supports a
:filter option used to evaluate whether or not the given file should be included in the results. The
:filter can be a regular expression OR an arity-1 function. To be fair, you could always filter the resulting enumerable AFTER listing the files, but filtering here in the same breath allows us to traverse the list only once.
Grep File Contents
One of my favorite bash commands is
grep -rl — I usually rely on it instead of my IDE when I’m searching a project. Now I can return a list of files that contain any line that matches the pattern:
iex> Xfile.grep_rl("raise Error", "projekt", recursive: true)
Since this exists in code, the pattern can be more than a simple string: you can supply your own arity-1 function to receive each line in the file.
Although I use
grep -rl more, I also included a
grep function inside
Xfile for searching single files and returning the matching lines. This can be useful for cherry-picking relevant lines from a file.
Head, Tail, and counting lines
Finally, I will mention a couple other convenience functions that can spare you from writing more boilerplate code after researching solutions on Stackoverflow:
tail , and
Xfile.head/2 will return the first
n lines of the given file:
iex> Xfile.head(".gitignore", 3)
"# The directory Mix will write compiled artifacts to.\n",
This package has made my life easier this week, and I hope it’s useful to the community. Happy coding!