Custom Commands in Elixir

Automate the annoying stuff away, and manage an app in prod a bit easier

Something I tried to do recently in writing Elixir code was to expand what commands my project supports. In some projects I have been a part of we released a set of "blessed" commands that function both in our development environment as well as the production environment, things like starting a server, operating a database migration during release, data operations, so on so forth.

Out of the box, Elixir offers the commands start, stop, daemon, pid, eval, remote, start_iex, daemon_iex, rpc, remote, and version to every project distributed as a release, the time tested best practice for deploying your executable code. From that list of commands, the deployed artifact is able to execute code within your project using the eval and rpc commands. Aside from these two commands in deployments, adding "commands" in the forms of mix tasks is simple and a well documented pattern and mix is shipped with most every developers install of Elixir.

(main) johnny@Johns-Laptop gov_republish % bin/gov_republish
Usage: gov_republish COMMAND [ARGS]

The known commands are:

    start          Starts the system
    start_iex      Starts the system with IEx attached
    daemon         Starts the system as a daemon
    daemon_iex     Starts the system as a daemon with IEx attached
    eval "EXPR"    Executes the given expression on a new, non-booted system
    rpc "EXPR"     Executes the given expression remotely on the running system
    remote         Connects to the running system via a remote shell
    restart        Restarts the running system via a remote command
    stop           Stops the running system via a remote command
    pid            Prints the operating system PID of the running system via a remote command
    version        Prints the release name and version to be booted

I'd like "migrate" to be added to this list somehow (if its possible).

So whats the problem, why not just use mix? The answer: mix is not shipped as part of the deployed artifact in an Elixir release, thus any mix task that you might have following the very well documented pattern, will not be able to run using the release artifact. Any easy management you set up with mix task_name will not be available in your production environment. This also means the default method of running database migrations with Ecto, is not possible. You could deploy mix, but this is not considered a standard pattern and there are other issues with it.

These are baked in issues with mix versus the release software from Elixir itself.

Fortunately, someone has already solved this problem, enter Phoenix Framework. Phoenix has already built two "custom commands" that are available to every install using the framework: migrate and server. Reviewing their docs these two commands are available in the same artifact directory when a release is made and each of these commands are really a shell script that wraps one of the commands available in the "binary" for a project.

# example for the "migrate" command
_build/prod/rel/my_app/bin/my_app eval "MyApp.Release.migrate"

Phoenix makes these two commands available by running a mix task prior to deployment called phx.gen.release and does two things:

1. generates a file in your project directory called release.ex that contains the code in functions called migrate and rollback accessible by code inside your project OR an evocation of bin/your_project eval "YourProject.Release.migrate()"

2. generates shell scripts for each function it intends to create and puts them in a directory called rel/overlays for each of these functions those will be

   1. rel/overlays/migrate.sh

   2. rel/overlays/rollback.sh
      Once these steps have been taken, the Phoenix docs will tell you that you are safe to generate the artifact, and these shell scripts will be available to your project. The Phoenix framework project notes that any files inside rel/overlays is copied in the artifact as default behavior by mix release.

Now thats awesome. Theres just three tiny problems:

1. I am not using Phoenix (directly) in my project as it is not a web service

2. I have management tasks that are more than just migrating and rolling back a release

3. I do not know yet how to make my custom commands show up in the releaseable artifact myself

In software there is no magic, there is always a direct reason for a thing that "just works" to be utilized, understood, and expanded on. I could take the easy route and write some shell scripts that call some functions deep in the core of my app but I want to automate away any of that thinking. I also want my commands I call in my mix tasks to be as identical as possible to how I manage the application in production

For example, these two should feel and be the same.

$> mix generate_new_account --account-name=hello.world

# executable not at beginning is a little annoying but fine
$> generate_new_account --account-name=hello.world 

So lets go look at the source code for phx.gen.release.

From lines 101 to 124 we get these calls which copy templates (editing them during the copy) for server and migrate to rel/overlays/bin sets the file permissions to Read and Execute for all users and Write given only to the owner.

Mix.Phoenix.copy_from(paths(), "priv/templates/phx.gen.release", binding, [
      {:eex, "rel/server.sh.eex", "rel/overlays/bin/server"},
      {:eex, "rel/server.bat.eex", "rel/overlays/bin/server.bat"}
    ])

    if opts.ecto do
      Mix.Phoenix.copy_from(paths(), "priv/templates/phx.gen.release", binding, [
        {:eex, "rel/migrate.sh.eex", "rel/overlays/bin/migrate"},
        {:eex, "rel/migrate.bat.eex", "rel/overlays/bin/migrate.bat"},
        {:eex, "release.ex", Mix.Phoenix.context_lib_path(app, "release.ex")}
      ])
    end

    if opts.docker do
      gen_docker(binding, opts)
    end

    File.chmod!("rel/overlays/bin/server", 0o755)
    File.chmod!("rel/overlays/bin/server.bat", 0o755)

    if opts.ecto do
      File.chmod!("rel/overlays/bin/migrate", 0o755)
      File.chmod!("rel/overlays/bin/migrate.bat", 0o755)
    end

Below these lines they define the output of the command string which provides instructions on how these files get run. All of this is HARD CODED and takes advantage of other provided automations.

How can we take advantage of these learnings?

First, putting shell scripts that call commands in your projects is easy, they just need to be executable (typically chmod 755 will do) and live in rel/overlays. Knowing that templates exist for shell scripts with some limited bindings, we can automate the creation of a new script.

Automate the generation of shell scripts

I will for this exercise not use elixir bindings, but instead take advantage of simple string replace tools. Our goal is to take the below shell script which has {task} written in it and formulate new scripts with those tasks filled out

This file lives in priv/templates/release_task.sh.txt

# rel/overlays/{task}.sh
#!/bin/sh
# Runs the {task} function
bin/gov_republish eval "GovRepublish.Release.{task}" "$@"

Just a quick note for readability: the final line calls eval on my provided function name which lives in a module called GovRepublish.Release and all arguments from the shell are forwarded to the function with the "$@". On an invocation of ./hello_world.sh --name=john my underlying program will receive --name=john as the provided arguments.

The function I came up with to create the scripts looks like this:

def sed_file(filename, replace_pattern, replace_value, output_location \\ :stdout) do
    {:ok, file_data} = File.read(filename)
    n_data = String.replace(file_data, replace_pattern, "#{replace_value}", [global: true])
    case output_location do
        :stdout -> IO.puts("File for #{replace_value}.sh would resemble:\n#{n_data}")
        _-> File.write(output_location, n_data)
            File.chmod!(output_location, 0o755)
    end
end

Its really simple, we read our template file, do a single string replace for {task} and the name of the thing we want to execute. File gets generated with the correct permissions in the right place. Providing :stdout prints a "dry run" to the console. (If anyone has a better idea than the print via IO.puts for testing and not using a log statement let me know.)

Okay so i generated the file, will it run? No, not yet. We still need to make functions available to run in our release.ex module.

Here is an example of release.ex that I created.

defmodule GovRepublish.Release do
    
    @doc """
    Conducts schema migrations for the application. Defaults to run all of them.
    """
    def migrate do
        Tasks.EctoMigrate.migrate(System.argv())
    end
    
    @doc """
    Conducts a schema rollback for the application.
    """
    def rollback do
        Tasks.EctoMigrate.rollback(System.argv())
    end
    
    @doc """
    Print the migration status for configured Repos' migrations.
    """
    def migration_status do
        Tasks.EctoMigrate.migration_status()
    end
    
    def bsky_resolve_did_to_handle do
        Tasks.BskyTasks.resolve_did_to_handle(System.argv())
    end
    
    def bsky_update_handle do
        Tasks.BskyTasks.update_handle(System.argv())
    end
end

In another module called tasks I have put all of the actual logic that exists for these functions, and others. In this convenient file I have decided what functions I want to expose in my deployed executable, and make sure that each gets System.argv() provided. All handling logic for arguments happens in these Task functions like this one:

def resolve_did_to_handle(args) do
    # we want to just have the output of the command in the stdout.
    Logger.configure(level: :error)
    Application.ensure_all_started(:gov_republish)
    {parsed, _, _} = OptionParser.parse(args, strict: [handle: :string])
    {:ok, did} = AtProto.IdentityResolution.resolve_handle_to_did(parsed[:handle])
    IO.puts("#{did}")
end

This gives us the benefit of doing all handling in these somewhat obscured functions, and allows us to design test cases for handling input and business logic separately.

Additionally by having all of these "blessed" functions in a file, I can do something else with my code: I can automate getting the list of functions that are supported and have it available to my build every time it runs. A consistent, generated list of commands. How?

Enter Sourceror. This library does the difficult job of getting the parse for code files and making their contents available for analysis or modification in programs. I will not get into the specifics here of how I utilized it's parse functions other than to say a function defined as get_task_names traverses a hard coded file location for the release.ex file and gets a list of every public function. In the case of the file defined above that list looks like this

[
    "migrate",
    "rollback",
    "migration_status",
    "bsky_resolve_did_to_handle",
    "bsky_update_handle"
]

Each entry is handed to our "find-and-replace" function that generates files and voila:

johnny@Johns-Laptop gov_republish % ls -ltra rel/overlays 
total 40
drwxr-xr-x  7 johnny  staff  224 Jul 14 11:47 ..
drwxrwxrwx  7 johnny  staff  224 Jul 14 11:47 .
-rwxr-xr-x  1 johnny  staff  126 Jul 14 13:12 rollback.sh
-rwxr-xr-x  1 johnny  staff  150 Jul 14 13:12 migration_status.sh
-rwxr-xr-x  1 johnny  staff  123 Jul 14 13:12 migrate.sh
-rwxr-xr-x  1 johnny  staff  156 Jul 14 13:12 bsky_update_handle.sh
-rwxr-xr-x  1 johnny  staff  180 Jul 14 13:12 bsky_resolve_did_to_handle.sh

So when we run MIX_ENV=prod mix release they are all available in the final product like so

johnny@Johns-Laptop gov_republish % ls -ltra _build/prod/rel/gov_republish 
total 112
drwxr-xr-x   3 johnny  staff     96 Jul 14 00:00 ..
drwxr-xr-x   3 johnny  staff     96 Jul 14 00:00 erts-15.2.3
drwxr-xr-x   3 johnny  staff     96 Jul 14 00:00 bin
drwxr-xr-x   5 johnny  staff    160 Jul 14 13:12 releases
-rwxr-xr-x   1 johnny  staff    126 Jul 14 13:12 rollback.sh
-rwxr-xr-x   1 johnny  staff    150 Jul 14 13:12 migration_status.sh
-rwxr-xr-x   1 johnny  staff    123 Jul 14 13:12 migrate.sh
-rwxr-xr-x   1 johnny  staff    156 Jul 14 13:12 bsky_update_handle.sh
-rwxr-xr-x   1 johnny  staff    180 Jul 14 13:12 bsky_resolve_did_to_handle.sh
drwxr-xr-x  62 johnny  staff   1984 Jul 14 13:12 lib

The one note here is that the file generation methods are run before compile time. The files must be generated before compilation executes to make sure they are available in a release.

Thats cool and all but what about Mix Tasks that resemble these?

Bringing us back to making sure we can use our dev env the same way we use our production env, we should make these commands available as mix tasks. Naming a mix task requires making a new module in the following directory for your project lib/mix/tasks and the file named new_task.ex typically looks like

defmodule Mix.Tasks.NewTask do
    use Mix.Task
    
    @impl Mix.Task
    def run(args) do
        #logic goes here
    end
end

Args are the system arguments that come in and typically must be parsed with System.argv() just like in our release.ex file. So....if we can generate a shell script, why not a mix task?

Going with our previous marker of {task} and trusting we want to expose all of the functions in release.ex as mix tasks, the next steps required are quite simple:

1. define a template for a mix task

2. make sure that we sed the mix task templates too, when we generate our new files.

3. test and see

Mix task template

defmodule Mix.Tasks.{task_camel} do
  use Mix.Task
  @impl Mix.Task
  def run(args) do
    GovRepublish.Release.{task}(args)
  end
end

Our new placeholder here is {task_camel} as module names must be camel case in Elixir.

Lets modify our sed and manager functions from before to support these types of actions as well as the commands earlier:

@doc """
Replaces values in the file provided and writes to a new location or prints to stdout
"""
def sed_file(filename, replacement_pairs, output_location \\ :stdout) do
    Logger.info("Attempting to find and replace for #{inspect(replacement_pairs)} in output location #{inspect(output_location)}")
    {:ok, file_data} = File.read(filename)
    n_data = String.replace(file_data, Map.keys(replacement_pairs), fn rep -> Map.get(replacement_pairs, rep) end, [global: true])
    case output_location do
        :stdout -> IO.puts("resembles:\n#{n_data}")
        _-> File.write(output_location, n_data)
            File.chmod!(output_location, 0o755)
    end
    Logger.info("Completed find and replace for #{inspect(replacement_pairs)}")
end

@doc """
Generates files for a given set of task names generated by the parse function above.
"""
def generate_task_files(template_filename, [task, task_camel] \\ ["{task}", "{task_camel}"], output_location \\ :stdout, extension \\ "sh") do
    Logger.info("Generating #{extension} files.")
    Enum.each(get_task_names(), fn elem ->
        stringed_elem = "#{elem}"
        module_name = Macro.camelize(stringed_elem)
        replacement_pairs = %{task=> stringed_elem, task_camel=> module_name}
        case output_location do
            :stdout -> sed_file(template_filename, replacement_pairs, :stdout)
            _-> sed_file(template_filename, replacement_pairs, output_file_format(output_location, elem, extension))
        end
    end)
    Logger.info("Completed generating #{extension} files. They are viewable in #{output_location}")
end

Now running the generate_task_file function for both release level commands as well as mix looks like the following:

@doc """
Runs the whole show for command and file generation
"""
def generate_files_and_commands() do
    #hard code here - config later
    generate_task_files("priv/templates/release_task.sh.txt", ["{task}", "{task_camel}"], "rel/overlays", "sh")
    generate_task_files("priv/templates/mix_task.ex.txt", ["{task}", "{task_camel}"], "lib/mix/tasks", "ex")
end

Caveats

First things first, we need to have all of our required shell scripts for a release generated BEFORE the release command is run. This also means that the command files need to be generated before compilation occurs. In order to do this, our call to sourceror happens in a spot where the files are not compiled (yet). All of the functions above that do "find-and-replace" and writing to various locations happen in a .exs script. When I want to generate these for a release I have a manually created mix task that looks like this to make the files:

defmodule Mix.Tasks.Gen.Commands do

    use Mix.Task
    
    @impl Mix.Task
    def run(_args) do
        Code.compile_file("scripts/elixir_helpers/code_generation_utils.exs")
        commands = CodeGenerationUtils.generate_files_and_commands()
        file = "#{inspect(commands)}"
        File.write("config/commands.exs", file)
    end
end    

If I do not invoke the .exs script before I compile code, or before I build a release, new mix tasks will not be available, nor will any new commands. To get around this I created an alias that runs the above mix task before compilation OR release. These aliases live in my project's mix.exs file like this:

defp aliases do
    [
        ...
        compile: ["gen.commands", "compile"],
        release: ["gen.commands", "release"]
    ]
end

This forces the gen commands to run before compile gets started. This may not be the best practice but I am open to correction. Please reach out.

Take Aways

My biggest piece of advice coming from this experiment has been: really get to know your tools. Typically this might mean reading the documentation and knowing what your config options are, but that might not be enough.

There comes a point where you should go and take apart your favorite toys and know how they accomplish a goal. For anyone working in software: you need to go to the github and start poking around. Find the line that executes an option and know exactly what happens after that point, see what happens before it. You should know and be able to show a fellow developer the calls from invocation of a main operation until the return boomerang. Once you see how a "blessed" project a kind of work but doesnt quite solve your problem, synthesize the steps taken and replicate it using a non-trivial example. Generalize what you came up with. Add some test cases. See what breaks.

On a personal note:

This was a long one but its the result of a lot of experimentation and putting together some important concepts while I was trying to unblock myself.

The code for a sample project is available in this github repository. Please peruse and feel free to try it in your own project.