elixir-patterns/patterns/testing.md

# Testing Patterns in Elixir

Patterns extracted from the Elixir standard library source code — how the core team writes and organizes tests.

---

## 1. Module-Level Async Declaration

**Source:** `lib/elixir/test/elixir/gen_server_test.exs:9`, `lib/elixir/test/elixir/enum_test.exs:8`, nearly all test files

**What it does:** Every test module declares `async: true` or `async: false` at the module level, making concurrency intent explicit.

**Why:** Tests that don't mutate global state run concurrently, dramatically speeding up the suite. The explicit opt-in forces developers to think about whether their test touches shared resources.

**Pattern:**
```elixir
defmodule GenServerTest do
  use ExUnit.Case, async: true
  # ...
end

# When global state is modified (e.g. registered processes):
defmodule TaskTest do
  use ExUnit.Case  # async defaults to false
  # ...
end
```

**Key insight:** The vast majority of Elixir's own tests use `async: true`. Only tests that register global names, modify Logger config, or interact with the filesystem use synchronous mode.

---

## 2. Parameterized Tests

**Source:** `lib/elixir/test/elixir/registry_test.exs:12-22`

**What it does:** Runs the same test suite against multiple configurations using the `:parameterize` option (since v1.18).

**Why:** Avoids duplicating test modules for combinatorial configurations. The Registry needs testing with `:unique`/`:duplicate` keys and varying partition counts.

**Pattern:**
```elixir
defmodule Registry.Test do
  use ExUnit.Case,
    async: true,
    parameterize:
      for(
        keys <- [:unique, :duplicate, {:duplicate, :pid}, {:duplicate, :key}],
        partitions <- [1, 8],
        do: %{keys: keys, partitions: partitions}
      )

  setup config do
    name = :"#{config.test}_#{config.partitions}_#{inspect(config.keys)}"
    opts = [keys: config.keys, name: name, partitions: config.partitions]
    {:ok, _} = start_supervised({Registry, opts})
    %{registry: name}
  end

  test "clean up registry on process crash", %{registry: registry, partitions: partitions} do
    # Test body uses parameters from context
  end
end
```

**Warning from docs:** "If you use parameterized tests and then find yourself adding conditionals in your tests to deal with different parameters, then parameterized tests may be the wrong solution."

---

## 3. Setup with `start_supervised/2`

**Source:** `lib/ex_unit/lib/ex_unit/callbacks.ex:277-340`, `lib/elixir/test/elixir/registry_test.exs:31`

**What it does:** Starts processes under a test supervisor that guarantees cleanup before the next test.

**Why:** Eliminates manual cleanup. The test supervisor terminates children in reverse order before `on_exit` callbacks run. No leaked processes between tests.

**Pattern:**
```elixir
setup config do
  {:ok, _} = start_supervised({Registry, keys: :unique, name: config.test})
  %{registry: config.test}
end
```

**Contrast with anti-pattern:**
```elixir
# BAD — process may leak if test crashes before cleanup
setup do
  {:ok, pid} = Registry.start_link(keys: :unique, name: :my_reg)
  on_exit(fn -> Process.exit(pid, :kill) end)
  %{registry: :my_reg}
end
```

---

## 4. Named Setup Functions (Composable Pipelines)

**Source:** `lib/ex_unit/lib/ex_unit/callbacks.ex:100-120` (docs)

**What it does:** Defines setup as a list of named functions rather than anonymous blocks.

**Why:** Each step is independently testable, reusable, and the setup pipeline reads like a declaration of preconditions.

**Pattern:**
```elixir
setup [:clean_up_tmp_directory, :start_server, :seed_data]

defp clean_up_tmp_directory(_context), do: [tmp_dir: "/tmp/test"]
defp start_server(context), do: {:ok, server: start_supervised!({MyServer, context.tmp_dir})}
defp seed_data(context), do: :ok
```

---

## 5. `on_exit` for Reversing Global Side Effects

**Source:** `lib/elixir/test/elixir/task_test.exs:1128-1131`, `lib/logger/test/logger_test.exs:12-17`

**What it does:** Registers cleanup callbacks that always run, even if the test fails.

**Why:** Guarantees global state (Logger config, ETS tables, process registrations) is restored regardless of test outcome.

**Pattern:**
```elixir
setup do
  translator = :logger.get_primary_config().filters[:logger_translator]
  assert :ok = :logger.remove_primary_filter(:logger_translator)
  on_exit(fn -> :logger.add_primary_filter(:logger_translator, translator) end)
end
```

**Key design:** `on_exit` runs in a *separate process* from the test, so it cannot interfere with test assertions.

---

## 6. Pattern Match Assertions

**Source:** `lib/ex_unit/lib/ex_unit/assertions.ex:145-175`

**What it does:** Uses `assert` with `=` for structural pattern matching in assertions.

**Why:** Provides rich failure messages showing both sides. More expressive than `assert x == y` when you only care about shape.

**Pattern:**
```elixir
# Assert structure, ignore specifics
assert {:ok, %{id: id}} = create_user("alice")
assert is_integer(id)

# Pin variables in patterns
x = 5
assert {:count, ^x} = get_counter()

# match? for complex guards
assert match?([%{id: id} | _] when is_integer(id), records)
```

---

## 7. `assert_receive` / `refute_receive` for Process Communication

**Source:** `lib/ex_unit/lib/ex_unit/assertions.ex:466-526`, `lib/elixir/test/elixir/process_test.exs:90-100`

**What it does:** Waits for messages matching a pattern within a timeout (default 100ms).

**Why:** Tests asynchronous process communication without `Process.sleep`. The test either receives the expected message or fails with a helpful mailbox dump.

**Pattern:**
```elixir
# Basic message assertion
test "send_after/3 sends messages once expired" do
  Process.send_after(self(), :hello, 10)
  assert_receive :hello
end

# Pattern matching with pins
test "monitor/2 with monitor options" do
  ref_and_alias = Process.monitor(pid, alias: :explicit_unalias)
  send(pid, {:ping, ref_and_alias})
  assert_receive :pong
  assert_receive {:DOWN, ^ref_and_alias, _, _, _}
end

# Negative assertion
test "exit(pid, :normal) does not cause the target process to exit" do
  Process.exit(pid, :normal)
  refute_receive {:EXIT, ^pid, :normal}, 100
end
```

---

## 8. Testing GenServers via Public API (No Internal State Inspection)

**Source:** `lib/elixir/test/elixir/gen_server_test.exs:87-106`

**What it does:** Tests GenServer behavior exclusively through `GenServer.call/cast/stop` — never peeks at internal state.

**Why:** Tests the contract, not the implementation. Internal state changes don't break tests.

**Pattern:**
```elixir
test "start_link/2, call/2 and cast/2" do
  {:ok, pid} = GenServer.start_link(Stack, [:hello])

  assert GenServer.call(pid, :pop) == :hello
  assert GenServer.cast(pid, {:push, :world}) == :ok
  assert GenServer.call(pid, :pop) == :world
  assert GenServer.stop(pid) == :ok
end
```

---

## 9. `catch_exit` for Testing Process Failures

**Source:** `lib/ex_unit/lib/ex_unit/assertions.ex:950-960`, `lib/elixir/test/elixir/gen_server_test.exs:118-137`

**What it does:** Catches exit signals from linked processes for assertion, or uses `Process.flag(:trap_exit, true)` + `assert_receive {:EXIT, ...}`.

**Why:** Testing error conditions in OTP requires intercepting exit signals. The two approaches serve different needs.

**Pattern:**
```elixir
# catch_exit for synchronous exit testing
test "call/3 exit messages" do
  assert catch_exit(GenServer.call(pid, :noreply, 1)) ==
           {:timeout, {GenServer, :call, [pid, :noreply, 1]}}
end

# trap_exit for linked process exits
test "exits on task error" do
  Process.flag(:trap_exit, true)
  task = Task.async(fn -> raise "oops" end)
  assert {{%RuntimeError{}, _}, {Task, :await, [^task, 5000]}} = catch_exit(Task.await(task))
end
```

---

## 10. `@tag capture_log: true` for Suppressing Expected Log Output

**Source:** `lib/elixir/test/elixir/gen_server_test.exs:114`, `lib/elixir/test/elixir/task_test.exs:10`

**What it does:** Captures log output during the test, only printing it if the test fails.

**Why:** Tests that intentionally trigger error conditions produce noisy log output. Capturing keeps the test output clean while preserving diagnostics on failure.

**Pattern:**
```elixir
# Per-test tag
@tag capture_log: true
test "call/3 exit messages" do
  # This test triggers error logs — they're captured
end

# Module-level for all tests
@moduletag :capture_log
```

---

## 11. `capture_log` / `capture_io` for Content Assertions

**Source:** `lib/ex_unit/lib/ex_unit/capture_log.ex:1-50`, `lib/elixir/test/elixir/task_test.exs:1138-1150`

**What it does:** Captures log/IO output and returns it as a string for assertion.

**Why:** Tests that the right messages are logged/printed without relying on side effects.

**Pattern:**
```elixir
# capture_log for asserting log content
test "logs a terminated task" do
  assert ExUnit.CaptureLog.capture_log(fn ->
           ref = Process.monitor(pid)
           send(pid, :go)
           receive do: ({:DOWN, ^ref, _, _, _} -> :ok)
         end) =~ ~r/Task .* terminating/
end

# with_io returns both result and output (since v1.13)
{result, output} = with_io(fn ->
  IO.puts("a")
  2 + 2
end)
assert result == 4
assert output == "a\n"
```

**Important for async tests:** Use `=~` instead of `==` for `:stderr` captures because output from other tests may interleave.

---

## 12. `describe` Blocks for Logical Grouping

**Source:** `lib/elixir/test/elixir/task_test.exs:218,272,365`, `lib/elixir/test/elixir/process_test.exs:146`

**What it does:** Groups related tests under a named describe block. Setup inside describe only applies to that group.

**Why:** Organizes tests by function/feature. Makes test output readable. Allows scoped `@describetag` and scoped setup.

**Pattern:**
```elixir
describe "await/2" do
  test "exits on timeout" do
    task = %Task{ref: make_ref(), owner: self(), pid: nil, mfa: {__MODULE__, :test, 1}}
    assert catch_exit(Task.await(task, 0)) == {:timeout, {Task, :await, [task, 0]}}
  end

  test "exits on normal exit" do
    task = Task.async(fn -> exit(:normal) end)
    assert catch_exit(Task.await(task)) == {:normal, {Task, :await, [task, 5000]}}
  end
end
```

**Constraint:** Describe blocks cannot be nested. `setup_all` cannot appear inside describe.

---

## 13. `ExUnit.CaseTemplate` for Shared Test Infrastructure

**Source:** `lib/mix/test/test_helper.exs:79-140`, `lib/logger/test/test_helper.exs:24-65`

**What it does:** Defines reusable test case templates with shared setup, helpers, and imports.

**Why:** Eliminates duplication across test modules. Provides domain-specific test DSLs.

**Pattern:**
```elixir
# In test_helper.exs
defmodule Logger.Case do
  use ExUnit.CaseTemplate

  using _ do
    quote do
      import Logger.Case
    end
  end

  setup do
    on_exit(fn ->
      # Shared cleanup for all tests using this template
    end)
    :ok
  end

  def capture_log(level \\ :debug, fun) do
    Logger.configure(level: level)
    capture_io(:user, fn ->
      fun.()
      Logger.flush()
    end)
  after
    Logger.configure(level: :debug)
  end
end

# In test file:
defmodule LoggerTest do
  use Logger.Case
  # Gets all imports and setup from the template
end
```

---

## 14. `doctest` Integration

**Source:** `lib/ex_unit/lib/ex_unit/doc_test.ex:1-80`, `lib/elixir/test/elixir/agent_test.exs:9`

**What it does:** Generates tests from `@doc` and `@moduledoc` code examples.

**Why:** Documentation examples are always verified. Prevents docs from rotting.

**Pattern:**
```elixir
defmodule AgentTest do
  use ExUnit.Case, async: true
  doctest Agent
end

# Selective doctesting:
doctest Kernel, except: [===: 2, !==: 2, and: 2, or: 2]
```

---

## 15. `Process.sleep(:infinity)` as a Process Parking Pattern

**Source:** `lib/elixir/test/elixir/task_test.exs:417`, `lib/elixir/test/elixir/registry_test.exs:71`

**What it does:** Spawns processes that block forever, used as test subjects that need to exist until explicitly killed.

**Why:** Creates stable process references for testing supervision, monitoring, and registry behavior. The process stays alive until the test supervisor shuts it down.

**Pattern:**
```elixir
# Process exists solely to be registered/monitored
{:ok, task} =
  Task.start(fn ->
    send(parent, Registry.register(registry, key, value))
    Process.sleep(:infinity)
  end)

# Then kill it to test cleanup:
Process.exit(task, :kill)
assert_receive {:DOWN, ^ref, _, _, _}
```

**Important distinction:** This is NOT `Process.sleep(100)` for timing — it's an intentional "park this process" pattern where the process is always explicitly terminated by the test.

---

## 16. Helper Functions for Test-Specific Behavior

**Source:** `lib/elixir/test/elixir/task_test.exs:12-36`, `lib/elixir/test/elixir/supervisor_test.exs:278-285`

**What it does:** Defines private helper functions within test modules for common test operations.

**Why:** Keeps tests DRY without over-abstracting. Helpers like `wait_until_down`, `assert_kill`, `create_dummy_task` encapsulate recurring patterns.

**Pattern:**
```elixir
defmodule TaskTest do
  use ExUnit.Case

  # Helper to create a known-state task for testing edge cases
  defp create_dummy_task(reason) do
    {pid, ref} = spawn_monitor(Kernel, :exit, [reason])
    receive do
      {:DOWN, ^ref, _, _, _} ->
        %Task{ref: ref, pid: pid, owner: self(), mfa: {__MODULE__, :create_dummy_task, 1}}
    end
  end

  # Helper that properly waits for process termination
  def wait_until_down(task) do
    ref = Process.monitor(task.pid)
    assert_receive {:DOWN, ^ref, _, _, _}
  end

  # Helper for asserting process kill
  defp assert_kill(pid, reason) do
    ref = Process.monitor(pid)
    Process.exit(pid, reason)
    assert_receive {:DOWN, ^ref, _, _, _}
  end
end
```

---

## 17. `@tag :tmp_dir` for Filesystem Tests

**Source:** `lib/ex_unit/lib/ex_unit/case.ex:281-304`, `lib/elixir/test/elixir/path_test.exs:12`

**What it does:** ExUnit automatically creates a unique temporary directory and passes its path via the test context.

**Why:** Filesystem tests need isolation. Each test gets its own directory, removed before creation to ensure a clean slate.

**Pattern:**
```elixir
@tag :tmp_dir
test "writes files", %{tmp_dir: tmp_dir} do
  path = Path.join(tmp_dir, "test.txt")
  File.write!(path, "hello")
  assert File.read!(path) == "hello"
end
```

---

## 18. `assert_raise` with Message Matching

**Source:** `lib/ex_unit/lib/ex_unit/assertions.ex:815-885`

**What it does:** Asserts both the exception type AND the message content (string or regex).

**Why:** Verifying the exception type alone is insufficient — the message tells users what went wrong. Testing it ensures error UX.

**Pattern:**
```elixir
# Exact message match
assert_raise ArgumentError, ~r"expected :name option to be one of the following:", fn ->
  GenServer.start_link(Stack, [:hello], name: "my_gen_server_name")
end

# Regex for dynamic content
assert_raise RuntimeError, ~r/^today's lucky number is 0\.\d+!$/, fn ->
  raise "today's lucky number is #{:rand.uniform()}!"
end
```

---

## 19. `@moduletag` / `@describetag` for Cross-Cutting Configuration

**Source:** `lib/elixir/test/elixir/system_test.exs:104,163`, `lib/elixir/test/elixir/task_test.exs:10`

**What it does:** Sets tags that apply to all tests in a module or describe block, used for filtering and configuration.

**Why:** Enables running subsets of tests (`mix test --include unix`) and applying configuration (like `:capture_log`) without repeating it on every test.

**Pattern:**
```elixir
defmodule SystemTest do
  use ExUnit.Case, async: true

  describe "Windows" do
    @describetag :windows
    # All tests here tagged :windows
  end

  describe "Unix" do
    @describetag :unix
    # All tests here tagged :unix
  end
end
```

---

## 20. Context Pattern Matching in Test Signatures

**Source:** `lib/ex_unit/lib/ex_unit/case.ex:57-80`, `lib/elixir/test/elixir/gen_server_test.exs:166`

**What it does:** Destructures the test context directly in the test function signature.

**Why:** Makes dependencies explicit. You see exactly what each test needs from setup.

**Pattern:**
```elixir
test "abcast/3", %{test: name} do
  {:ok, _} = GenServer.start_link(Stack, [], name: name)
  assert GenServer.abcast(name, {:push, :hello}) == :abcast
end

# Using test name for unique naming — prevents collision in async tests
```

The `%{test: name}` pattern is ubiquitous — the test name is unique per module, making it perfect for naming registered processes in async tests.