elixir-patterns/patterns/testing.md

Testing Patterns in Elixir

Patterns extracted from the Elixir standard library source code — how the core team writes and organizes tests.

---

1. Module-Level Async Declaration

Source: lib/elixir/test/elixir/gen_server_test.exs:9, lib/elixir/test/elixir/enum_test.exs:8, nearly all test files

What it does: Every test module declares async: true or async: false at the module level, making concurrency intent explicit.

Why: Tests that don't mutate global state run concurrently, dramatically speeding up the suite. The explicit opt-in forces developers to think about whether their test touches shared resources.

Pattern:

defmodule GenServerTest do
  use ExUnit.Case, async: true
  # ...
end

# When global state is modified (e.g. registered processes):
defmodule TaskTest do
  use ExUnit.Case  # async defaults to false
  # ...
end

Key insight: The vast majority of Elixir's own tests use async: true. Only tests that register global names, modify Logger config, or interact with the filesystem use synchronous mode.

---

2. Parameterized Tests

Source: lib/elixir/test/elixir/registry_test.exs:12-22

What it does: Runs the same test suite against multiple configurations using the :parameterize option (since v1.18).

Why: Avoids duplicating test modules for combinatorial configurations. The Registry needs testing with :unique/:duplicate keys and varying partition counts.

Pattern:

defmodule Registry.Test do
  use ExUnit.Case,
    async: true,
    parameterize:
      for(
        keys <- [:unique, :duplicate, {:duplicate, :pid}, {:duplicate, :key}],
        partitions <- [1, 8],
        do: %{keys: keys, partitions: partitions}
      )

  setup config do
    name = :"#{config.test}_#{config.partitions}_#{inspect(config.keys)}"
    opts = [keys: config.keys, name: name, partitions: config.partitions]
    {:ok, _} = start_supervised({Registry, opts})
    %{registry: name}
  end

  test "clean up registry on process crash", %{registry: registry, partitions: partitions} do
    # Test body uses parameters from context
  end
end

Warning from docs: "If you use parameterized tests and then find yourself adding conditionals in your tests to deal with different parameters, then parameterized tests may be the wrong solution."

---

3. Setup with start_supervised/2

Source: lib/ex_unit/lib/ex_unit/callbacks.ex:277-340, lib/elixir/test/elixir/registry_test.exs:31

What it does: Starts processes under a test supervisor that guarantees cleanup before the next test.

Why: Eliminates manual cleanup. The test supervisor terminates children in reverse order before on_exit callbacks run. No leaked processes between tests.

Pattern:

setup config do
  {:ok, _} = start_supervised({Registry, keys: :unique, name: config.test})
  %{registry: config.test}
end

Contrast with anti-pattern:

# BAD — process may leak if test crashes before cleanup
setup do
  {:ok, pid} = Registry.start_link(keys: :unique, name: :my_reg)
  on_exit(fn -> Process.exit(pid, :kill) end)
  %{registry: :my_reg}
end

---

4. Named Setup Functions (Composable Pipelines)

Source: lib/ex_unit/lib/ex_unit/callbacks.ex:100-120 (docs)

What it does: Defines setup as a list of named functions rather than anonymous blocks.

Why: Each step is independently testable, reusable, and the setup pipeline reads like a declaration of preconditions.

Pattern:

setup [:clean_up_tmp_directory, :start_server, :seed_data]

defp clean_up_tmp_directory(_context), do: [tmp_dir: "/tmp/test"]
defp start_server(context), do: {:ok, server: start_supervised!({MyServer, context.tmp_dir})}
defp seed_data(context), do: :ok

---

5. on_exit for Reversing Global Side Effects

Source: lib/elixir/test/elixir/task_test.exs:1128-1131, lib/logger/test/logger_test.exs:12-17

What it does: Registers cleanup callbacks that always run, even if the test fails.

Why: Guarantees global state (Logger config, ETS tables, process registrations) is restored regardless of test outcome.

Pattern:

setup do
  translator = :logger.get_primary_config().filters[:logger_translator]
  assert :ok = :logger.remove_primary_filter(:logger_translator)
  on_exit(fn -> :logger.add_primary_filter(:logger_translator, translator) end)
end

Key design: on_exit runs in a separate process from the test, so it cannot interfere with test assertions.

---

6. Pattern Match Assertions

Source: lib/ex_unit/lib/ex_unit/assertions.ex:145-175

What it does: Uses assert with = for structural pattern matching in assertions.

Why: Provides rich failure messages showing both sides. More expressive than assert x == y when you only care about shape.

Pattern:

# Assert structure, ignore specifics
assert {:ok, %{id: id}} = create_user("alice")
assert is_integer(id)

# Pin variables in patterns
x = 5
assert {:count, ^x} = get_counter()

# match? for complex guards
assert match?([%{id: id} | _] when is_integer(id), records)

---

7. assert_receive / refute_receive for Process Communication

Source: lib/ex_unit/lib/ex_unit/assertions.ex:466-526, lib/elixir/test/elixir/process_test.exs:90-100

What it does: Waits for messages matching a pattern within a timeout (default 100ms).

Why: Tests asynchronous process communication without Process.sleep. The test either receives the expected message or fails with a helpful mailbox dump.

Pattern:

# Basic message assertion
test "send_after/3 sends messages once expired" do
  Process.send_after(self(), :hello, 10)
  assert_receive :hello
end

# Pattern matching with pins
test "monitor/2 with monitor options" do
  ref_and_alias = Process.monitor(pid, alias: :explicit_unalias)
  send(pid, {:ping, ref_and_alias})
  assert_receive :pong
  assert_receive {:DOWN, ^ref_and_alias, _, _, _}
end

# Negative assertion
test "exit(pid, :normal) does not cause the target process to exit" do
  Process.exit(pid, :normal)
  refute_receive {:EXIT, ^pid, :normal}, 100
end

---

8. Testing GenServers via Public API (No Internal State Inspection)

Source: lib/elixir/test/elixir/gen_server_test.exs:87-106

What it does: Tests GenServer behavior exclusively through GenServer.call/cast/stop — never peeks at internal state.

Why: Tests the contract, not the implementation. Internal state changes don't break tests.

Pattern:

test "start_link/2, call/2 and cast/2" do
  {:ok, pid} = GenServer.start_link(Stack, [:hello])

  assert GenServer.call(pid, :pop) == :hello
  assert GenServer.cast(pid, {:push, :world}) == :ok
  assert GenServer.call(pid, :pop) == :world
  assert GenServer.stop(pid) == :ok
end

---

9. catch_exit for Testing Process Failures

Source: lib/ex_unit/lib/ex_unit/assertions.ex:950-960, lib/elixir/test/elixir/gen_server_test.exs:118-137

What it does: Catches exit signals from linked processes for assertion, or uses Process.flag(:trap_exit, true) + assert_receive {:EXIT, ...}.

Why: Testing error conditions in OTP requires intercepting exit signals. The two approaches serve different needs.

Pattern:

# catch_exit for synchronous exit testing
test "call/3 exit messages" do
  assert catch_exit(GenServer.call(pid, :noreply, 1)) ==
           {:timeout, {GenServer, :call, [pid, :noreply, 1]}}
end

# trap_exit for linked process exits
test "exits on task error" do
  Process.flag(:trap_exit, true)
  task = Task.async(fn -> raise "oops" end)
  assert {{%RuntimeError{}, _}, {Task, :await, [^task, 5000]}} = catch_exit(Task.await(task))
end

---

10. @tag capture_log: true for Suppressing Expected Log Output

Source: lib/elixir/test/elixir/gen_server_test.exs:114, lib/elixir/test/elixir/task_test.exs:10

What it does: Captures log output during the test, only printing it if the test fails.

Why: Tests that intentionally trigger error conditions produce noisy log output. Capturing keeps the test output clean while preserving diagnostics on failure.

Pattern:

# Per-test tag
@tag capture_log: true
test "call/3 exit messages" do
  # This test triggers error logs — they're captured
end

# Module-level for all tests
@moduletag :capture_log

---

11. capture_log / capture_io for Content Assertions

Source: lib/ex_unit/lib/ex_unit/capture_log.ex:1-50, lib/elixir/test/elixir/task_test.exs:1138-1150

What it does: Captures log/IO output and returns it as a string for assertion.

Why: Tests that the right messages are logged/printed without relying on side effects.

Pattern:

# capture_log for asserting log content
test "logs a terminated task" do
  assert ExUnit.CaptureLog.capture_log(fn ->
           ref = Process.monitor(pid)
           send(pid, :go)
           receive do: ({:DOWN, ^ref, _, _, _} -> :ok)
         end) =~ ~r/Task .* terminating/
end

# with_io returns both result and output (since v1.13)
{result, output} = with_io(fn ->
  IO.puts("a")
  2 + 2
end)
assert result == 4
assert output == "a\n"

Important for async tests: Use =~ instead of == for :stderr captures because output from other tests may interleave.

---

12. describe Blocks for Logical Grouping

Source: lib/elixir/test/elixir/task_test.exs:218,272,365, lib/elixir/test/elixir/process_test.exs:146

What it does: Groups related tests under a named describe block. Setup inside describe only applies to that group.

Why: Organizes tests by function/feature. Makes test output readable. Allows scoped @describetag and scoped setup.

Pattern:

describe "await/2" do
  test "exits on timeout" do
    task = %Task{ref: make_ref(), owner: self(), pid: nil, mfa: {__MODULE__, :test, 1}}
    assert catch_exit(Task.await(task, 0)) == {:timeout, {Task, :await, [task, 0]}}
  end

  test "exits on normal exit" do
    task = Task.async(fn -> exit(:normal) end)
    assert catch_exit(Task.await(task)) == {:normal, {Task, :await, [task, 5000]}}
  end
end

Constraint: Describe blocks cannot be nested. setup_all cannot appear inside describe.

---

13. ExUnit.CaseTemplate for Shared Test Infrastructure

Source: lib/mix/test/test_helper.exs:79-140, lib/logger/test/test_helper.exs:24-65

What it does: Defines reusable test case templates with shared setup, helpers, and imports.

Why: Eliminates duplication across test modules. Provides domain-specific test DSLs.

Pattern:

# In test_helper.exs
defmodule Logger.Case do
  use ExUnit.CaseTemplate

  using _ do
    quote do
      import Logger.Case
    end
  end

  setup do
    on_exit(fn ->
      # Shared cleanup for all tests using this template
    end)
    :ok
  end

  def capture_log(level \\ :debug, fun) do
    Logger.configure(level: level)
    capture_io(:user, fn ->
      fun.()
      Logger.flush()
    end)
  after
    Logger.configure(level: :debug)
  end
end

# In test file:
defmodule LoggerTest do
  use Logger.Case
  # Gets all imports and setup from the template
end

---

14. doctest Integration

Source: lib/ex_unit/lib/ex_unit/doc_test.ex:1-80, lib/elixir/test/elixir/agent_test.exs:9

What it does: Generates tests from @doc and @moduledoc code examples.

Why: Documentation examples are always verified. Prevents docs from rotting.

Pattern:

defmodule AgentTest do
  use ExUnit.Case, async: true
  doctest Agent
end

# Selective doctesting:
doctest Kernel, except: [===: 2, !==: 2, and: 2, or: 2]

---

15. Process.sleep(:infinity) as a Process Parking Pattern

Source: lib/elixir/test/elixir/task_test.exs:417, lib/elixir/test/elixir/registry_test.exs:71

What it does: Spawns processes that block forever, used as test subjects that need to exist until explicitly killed.

Why: Creates stable process references for testing supervision, monitoring, and registry behavior. The process stays alive until the test supervisor shuts it down.

Pattern:

# Process exists solely to be registered/monitored
{:ok, task} =
  Task.start(fn ->
    send(parent, Registry.register(registry, key, value))
    Process.sleep(:infinity)
  end)

# Then kill it to test cleanup:
Process.exit(task, :kill)
assert_receive {:DOWN, ^ref, _, _, _}

Important distinction: This is NOT Process.sleep(100) for timing — it's an intentional "park this process" pattern where the process is always explicitly terminated by the test.

---

16. Helper Functions for Test-Specific Behavior

Source: lib/elixir/test/elixir/task_test.exs:12-36, lib/elixir/test/elixir/supervisor_test.exs:278-285

What it does: Defines private helper functions within test modules for common test operations.

Why: Keeps tests DRY without over-abstracting. Helpers like wait_until_down, assert_kill, create_dummy_task encapsulate recurring patterns.

Pattern:

defmodule TaskTest do
  use ExUnit.Case

  # Helper to create a known-state task for testing edge cases
  defp create_dummy_task(reason) do
    {pid, ref} = spawn_monitor(Kernel, :exit, [reason])
    receive do
      {:DOWN, ^ref, _, _, _} ->
        %Task{ref: ref, pid: pid, owner: self(), mfa: {__MODULE__, :create_dummy_task, 1}}
    end
  end

  # Helper that properly waits for process termination
  def wait_until_down(task) do
    ref = Process.monitor(task.pid)
    assert_receive {:DOWN, ^ref, _, _, _}
  end

  # Helper for asserting process kill
  defp assert_kill(pid, reason) do
    ref = Process.monitor(pid)
    Process.exit(pid, reason)
    assert_receive {:DOWN, ^ref, _, _, _}
  end
end

---

17. @tag :tmp_dir for Filesystem Tests

Source: lib/ex_unit/lib/ex_unit/case.ex:281-304, lib/elixir/test/elixir/path_test.exs:12

What it does: ExUnit automatically creates a unique temporary directory and passes its path via the test context.

Why: Filesystem tests need isolation. Each test gets its own directory, removed before creation to ensure a clean slate.

Pattern:

@tag :tmp_dir
test "writes files", %{tmp_dir: tmp_dir} do
  path = Path.join(tmp_dir, "test.txt")
  File.write!(path, "hello")
  assert File.read!(path) == "hello"
end

---

18. assert_raise with Message Matching

Source: lib/ex_unit/lib/ex_unit/assertions.ex:815-885

What it does: Asserts both the exception type AND the message content (string or regex).

Why: Verifying the exception type alone is insufficient — the message tells users what went wrong. Testing it ensures error UX.

Pattern:

# Exact message match
assert_raise ArgumentError, ~r"expected :name option to be one of the following:", fn ->
  GenServer.start_link(Stack, [:hello], name: "my_gen_server_name")
end

# Regex for dynamic content
assert_raise RuntimeError, ~r/^today's lucky number is 0\.\d+!$/, fn ->
  raise "today's lucky number is #{:rand.uniform()}!"
end

---

19. @moduletag / @describetag for Cross-Cutting Configuration

Source: lib/elixir/test/elixir/system_test.exs:104,163, lib/elixir/test/elixir/task_test.exs:10

What it does: Sets tags that apply to all tests in a module or describe block, used for filtering and configuration.

Why: Enables running subsets of tests (mix test --include unix) and applying configuration (like :capture_log) without repeating it on every test.

Pattern:

defmodule SystemTest do
  use ExUnit.Case, async: true

  describe "Windows" do
    @describetag :windows
    # All tests here tagged :windows
  end

  describe "Unix" do
    @describetag :unix
    # All tests here tagged :unix
  end
end

---

20. Context Pattern Matching in Test Signatures

Source: lib/ex_unit/lib/ex_unit/case.ex:57-80, lib/elixir/test/elixir/gen_server_test.exs:166

What it does: Destructures the test context directly in the test function signature.

Why: Makes dependencies explicit. You see exactly what each test needs from setup.

Pattern:

test "abcast/3", %{test: name} do
  {:ok, _} = GenServer.start_link(Stack, [], name: name)
  assert GenServer.abcast(name, {:push, :hello}) == :abcast
end

# Using test name for unique naming — prevents collision in async tests

The %{test: name} pattern is ubiquitous — the test name is unique per module, making it perfect for naming registered processes in async tests.