10218813d3
- Add ## Contents and ## Decision Tree to all 10 existing pattern files - Fix embed_as/1 semantics inversion in types.md (:self → :dump) - Fix fabricated __meta__.changes reference in changesets.md - Fix default primary key type (:integer → :id) in schemas.md - Combine @impl subsections into single "Minimal Callback Annotation"
1775 lines
57 KiB
Markdown
1775 lines
57 KiB
Markdown
# Testing Patterns in Elixir
|
|
|
|
Patterns extracted from the Elixir standard library source code — how the core team writes and organizes tests.
|
|
|
|
## Contents
|
|
|
|
1. [Module-Level Async Declaration](#1-module-level-async-declaration)
|
|
2. [Parameterized Tests](#2-parameterized-tests)
|
|
3. [Setup with `start_supervised/2`](#3-setup-with-start_supervised2)
|
|
4. [Named Setup Functions (Composable Pipelines)](#4-named-setup-functions-composable-pipelines)
|
|
5. [`on_exit` for Reversing Global Side Effects](#5-on_exit-for-reversing-global-side-effects)
|
|
6. [Pattern Match Assertions](#6-pattern-match-assertions)
|
|
7. [`assert_receive` / `refute_receive` for Process Communication](#7-assert_receive--refute_receive-for-process-communication)
|
|
8. [Testing GenServers via Public API (No Internal State Inspection)](#8-testing-genservers-via-public-api-no-internal-state-inspection)
|
|
9. [`catch_exit` for Testing Process Failures](#9-catch_exit-for-testing-process-failures)
|
|
10. [`@tag capture_log: true` for Suppressing Expected Log Output](#10-tag-capture_log-true-for-suppressing-expected-log-output)
|
|
11. [`capture_log` / `capture_io` for Content Assertions](#11-capture_log--capture_io-for-content-assertions)
|
|
12. [`describe` Blocks for Logical Grouping](#12-describe-blocks-for-logical-grouping)
|
|
13. [`ExUnit.CaseTemplate` for Shared Test Infrastructure](#13-exunitcasetemplate-for-shared-test-infrastructure)
|
|
14. [`doctest` Integration](#14-doctest-integration)
|
|
15. [`Process.sleep(:infinity)` as a Process Parking Pattern](#15-processsleepinfinity-as-a-process-parking-pattern)
|
|
16. [Helper Functions for Test-Specific Behavior](#16-helper-functions-for-test-specific-behavior)
|
|
17. [`@tag :tmp_dir` for Filesystem Tests](#17-tag-tmp_dir-for-filesystem-tests)
|
|
18. [`assert_raise` with Message Matching](#18-assert_raise-with-message-matching)
|
|
19. [`@moduletag` / `@describetag` for Cross-Cutting Configuration](#19-moduletag--describetag-for-cross-cutting-configuration)
|
|
20. [Context Pattern Matching in Test Signatures](#20-context-pattern-matching-in-test-signatures)
|
|
|
|
---
|
|
|
|
## 1. Module-Level Async Declaration
|
|
|
|
**Source:** [lib/elixir/test/elixir/gen_server_test.exs#L9](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/gen_server_test.exs#L9), [lib/elixir/test/elixir/enum_test.exs#L8](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/enum_test.exs#L8), nearly all test files
|
|
|
|
**What it does:** Every test module declares `async: true` or `async: false` at the module level, making concurrency intent explicit.
|
|
|
|
**Why:** Tests that don't mutate global state run concurrently, dramatically speeding up the suite. The explicit opt-in forces developers to think about whether their test touches shared resources.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
defmodule GenServerTest do
|
|
use ExUnit.Case, async: true
|
|
# ...
|
|
end
|
|
|
|
# When global state is modified (e.g. registered processes):
|
|
defmodule TaskTest do
|
|
use ExUnit.Case # async defaults to false
|
|
# ...
|
|
end
|
|
```
|
|
|
|
**Key insight:** The vast majority of Elixir's own tests use `async: true`. Only tests that register global names, modify Logger config, or interact with the filesystem use synchronous mode.
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- You're creating any new test module (always explicitly declare async intent)
|
|
- Tests are pure — no global name registration, no shared ETS tables, no filesystem writes to the same path
|
|
- You want faster test suite execution
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
defmodule UserTest do
|
|
use ExUnit.Case
|
|
# Defaults to async: false — runs serially for no reason
|
|
test "parses name" do
|
|
assert parse_name("Alice Bob") == {"Alice", "Bob"}
|
|
end
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
defmodule UserTest do
|
|
use ExUnit.Case, async: true
|
|
# Pure tests run concurrently — suite finishes faster
|
|
test "parses name" do
|
|
assert parse_name("Alice Bob") == {"Alice", "Bob"}
|
|
end
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- Tests register global process names (`:global`, `Process.register/2`)
|
|
- Tests modify shared application config (`Application.put_env`)
|
|
- Tests write to the same filesystem path without `@tag :tmp_dir`
|
|
- Tests depend on Logger configuration or capture stderr in exact mode
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
defmodule ConfigTest do
|
|
use ExUnit.Case, async: true # DANGEROUS
|
|
|
|
test "updates global config" do
|
|
Application.put_env(:my_app, :key, :value)
|
|
assert Application.get_env(:my_app, :key) == :value
|
|
end
|
|
# Another async test might see or clobber this config!
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
defmodule ConfigTest do
|
|
use ExUnit.Case # async: false — global state mutation
|
|
|
|
setup do
|
|
original = Application.get_env(:my_app, :key)
|
|
on_exit(fn -> Application.put_env(:my_app, :key, original) end)
|
|
end
|
|
|
|
test "updates global config" do
|
|
Application.put_env(:my_app, :key, :value)
|
|
assert Application.get_env(:my_app, :key) == :value
|
|
end
|
|
end
|
|
```
|
|
|
|
**Why:** Async tests run in parallel. Global state mutations in parallel tests produce race conditions — intermittent failures that are nightmares to debug.
|
|
|
|
---
|
|
|
|
## 2. Parameterized Tests
|
|
|
|
**Source:** [lib/elixir/test/elixir/registry_test.exs#L12](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/registry_test.exs#L12)
|
|
|
|
**What it does:** Runs the same test suite against multiple configurations using the `:parameterize` option (since v1.18).
|
|
|
|
**Why:** Avoids duplicating test modules for combinatorial configurations. The Registry needs testing with `:unique`/`:duplicate` keys and varying partition counts.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
defmodule Registry.Test do
|
|
use ExUnit.Case,
|
|
async: true,
|
|
parameterize:
|
|
for(
|
|
keys <- [:unique, :duplicate, {:duplicate, :pid}, {:duplicate, :key}],
|
|
partitions <- [1, 8],
|
|
do: %{keys: keys, partitions: partitions}
|
|
)
|
|
|
|
setup config do
|
|
name = :"#{config.test}_#{config.partitions}_#{inspect(config.keys)}"
|
|
opts = [keys: config.keys, name: name, partitions: config.partitions]
|
|
{:ok, _} = start_supervised({Registry, opts})
|
|
%{registry: name}
|
|
end
|
|
|
|
test "clean up registry on process crash", %{registry: registry, partitions: partitions} do
|
|
# Test body uses parameters from context
|
|
end
|
|
end
|
|
```
|
|
|
|
**Warning from docs:** "If you use parameterized tests and then find yourself adding conditionals in your tests to deal with different parameters, then parameterized tests may be the wrong solution."
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- The same logic must work across multiple configurations (backends, modes, partition counts)
|
|
- You'd otherwise copy-paste an entire test module with minor variations
|
|
- The behavior under test is identical regardless of parameter — only setup differs
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
# Two nearly identical modules
|
|
defmodule CacheEtsTest do
|
|
use ExUnit.Case, async: true
|
|
setup do: %{cache: start_cache(:ets)}
|
|
test "get/set", %{cache: c}, do: # ...
|
|
end
|
|
|
|
defmodule CacheRedisTest do
|
|
use ExUnit.Case, async: true
|
|
setup do: %{cache: start_cache(:redis)}
|
|
test "get/set", %{cache: c}, do: # ... # Same test!
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
defmodule CacheTest do
|
|
use ExUnit.Case,
|
|
async: true,
|
|
parameterize: [%{backend: :ets}, %{backend: :redis}]
|
|
|
|
setup %{backend: backend} do
|
|
%{cache: start_cache(backend)}
|
|
end
|
|
|
|
test "get/set", %{cache: c} do
|
|
# Runs once per backend — no duplication
|
|
end
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- Different parameters need different assertions (sign you're testing different behavior)
|
|
- You find yourself adding `if config.backend == :redis` inside tests
|
|
- There are only 2 simple variations (just write two tests with descriptive names)
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
defmodule ApiTest do
|
|
use ExUnit.Case,
|
|
parameterize: [%{method: :get}, %{method: :post}]
|
|
|
|
test "request", %{method: method} do
|
|
if method == :get do
|
|
assert get("/users") == 200
|
|
else
|
|
assert post("/users", %{name: "x"}) == 201 # Different assertion!
|
|
end
|
|
end
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
test "GET /users returns 200" do
|
|
assert get("/users") == 200
|
|
end
|
|
|
|
test "POST /users creates a user" do
|
|
assert post("/users", %{name: "x"}) == 201
|
|
end
|
|
```
|
|
|
|
**Why:** Parameterized tests assert the *same* behavior across configurations. If the assertions diverge per parameter, you're testing different things — write separate tests with clear names.
|
|
|
|
---
|
|
|
|
## 3. Setup with `start_supervised/2`
|
|
|
|
**Source:** [lib/ex_unit/lib/ex_unit/callbacks.ex#L277](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/ex_unit/lib/ex_unit/callbacks.ex#L277), [lib/elixir/test/elixir/registry_test.exs#L31](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/registry_test.exs#L31)
|
|
|
|
**What it does:** Starts processes under a test supervisor that guarantees cleanup before the next test.
|
|
|
|
**Why:** Eliminates manual cleanup. The test supervisor terminates children in reverse order before `on_exit` callbacks run. No leaked processes between tests.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
setup config do
|
|
{:ok, _} = start_supervised({Registry, keys: :unique, name: config.test})
|
|
%{registry: config.test}
|
|
end
|
|
```
|
|
|
|
**Contrast with anti-pattern:**
|
|
```elixir
|
|
# BAD — process may leak if test crashes before cleanup
|
|
setup do
|
|
{:ok, pid} = Registry.start_link(keys: :unique, name: :my_reg)
|
|
on_exit(fn -> Process.exit(pid, :kill) end)
|
|
%{registry: :my_reg}
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Your test needs a running process (GenServer, Registry, Supervisor, etc.)
|
|
- You want guaranteed cleanup even if the test crashes
|
|
- The process should have the same lifecycle as the test
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
setup do
|
|
{:ok, pid} = MyServer.start_link(name: :test_server)
|
|
on_exit(fn ->
|
|
if Process.alive?(pid), do: GenServer.stop(pid)
|
|
end)
|
|
%{server: pid}
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
setup do
|
|
pid = start_supervised!(MyServer)
|
|
%{server: pid}
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- You're testing the startup/shutdown behavior itself (need raw `start_link`)
|
|
- The process must outlive the test (rare — usually a test design smell)
|
|
- You need to test what happens when start_link fails
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Testing that start_link fails with bad args
|
|
test "rejects invalid config" do
|
|
# start_supervised! will raise, hiding the actual error you want to test
|
|
assert {:error, _} = start_supervised({MyServer, invalid: true})
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
test "rejects invalid config" do
|
|
assert {:error, {:bad_config, _}} = MyServer.start_link(invalid: true)
|
|
end
|
|
```
|
|
|
|
**Why:** `start_supervised` is for processes your test *needs running*. When you're testing failure modes of the process itself, call `start_link` directly so you can assert on the error tuple.
|
|
|
|
---
|
|
|
|
## 4. Named Setup Functions (Composable Pipelines)
|
|
|
|
**Source:** [lib/ex_unit/lib/ex_unit/callbacks.ex#L100](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/ex_unit/lib/ex_unit/callbacks.ex#L100) (docs)
|
|
|
|
**What it does:** Defines setup as a list of named functions rather than anonymous blocks.
|
|
|
|
**Why:** Each step is independently testable, reusable, and the setup pipeline reads like a declaration of preconditions.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
setup [:clean_up_tmp_directory, :start_server, :seed_data]
|
|
|
|
defp clean_up_tmp_directory(_context), do: [tmp_dir: "/tmp/test"]
|
|
defp start_server(context), do: {:ok, server: start_supervised!({MyServer, context.tmp_dir})}
|
|
defp seed_data(context), do: :ok
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Setup has multiple independent steps that read better as named operations
|
|
- Different `describe` blocks need different combinations of setup steps
|
|
- You want to reuse individual setup steps across describe blocks
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
setup do
|
|
# 30 lines of mixed concerns in one block
|
|
{:ok, _} = start_supervised(Database)
|
|
user = insert(:user)
|
|
token = generate_token(user)
|
|
{:ok, socket} = connect(token)
|
|
%{user: user, socket: socket, token: token}
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
setup [:start_database, :create_user, :connect_socket]
|
|
|
|
defp start_database(_ctx), do: {:ok, db: start_supervised!(Database)}
|
|
defp create_user(_ctx), do: {:ok, user: insert(:user)}
|
|
defp connect_socket(%{user: user}), do: {:ok, socket: connect!(generate_token(user))}
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- Setup is 1-3 lines (named function adds indirection without clarity)
|
|
- Steps are tightly coupled and can't be composed independently
|
|
- Only one describe block uses the setup (inline is simpler)
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Over-decomposing trivial setup
|
|
setup [:assign_name]
|
|
|
|
defp assign_name(_ctx), do: {:ok, name: "Alice"}
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
setup do
|
|
%{name: "Alice"}
|
|
end
|
|
```
|
|
|
|
**Why:** Named setup functions shine when they compose independently and describe preconditions. For trivial assignments, the indirection makes the code harder to follow, not easier.
|
|
|
|
---
|
|
|
|
## 5. `on_exit` for Reversing Global Side Effects
|
|
|
|
**Source:** [lib/elixir/test/elixir/task_test.exs#L1128](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/task_test.exs#L1128), [lib/logger/test/logger_test.exs#L12](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/logger/test/logger_test.exs#L12)
|
|
|
|
**What it does:** Registers cleanup callbacks that always run, even if the test fails.
|
|
|
|
**Why:** Guarantees global state (Logger config, ETS tables, process registrations) is restored regardless of test outcome.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
setup do
|
|
translator = :logger.get_primary_config().filters[:logger_translator]
|
|
assert :ok = :logger.remove_primary_filter(:logger_translator)
|
|
on_exit(fn -> :logger.add_primary_filter(:logger_translator, translator) end)
|
|
end
|
|
```
|
|
|
|
**Key design:** `on_exit` runs in a *separate process* from the test, so it cannot interfere with test assertions.
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Your test modifies global state (application config, Logger, ETS, registered names)
|
|
- State must be restored even if the test crashes or fails
|
|
- You need cleanup that runs after the test process dies
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
test "custom log level" do
|
|
Logger.configure(level: :error)
|
|
# test logic...
|
|
Logger.configure(level: :debug) # Never runs if test fails!
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
test "custom log level" do
|
|
original = Logger.level()
|
|
on_exit(fn -> Logger.configure(level: original) end)
|
|
Logger.configure(level: :error)
|
|
# test logic — cleanup guaranteed
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- `start_supervised` handles the lifecycle (it cleans up automatically)
|
|
- The state is test-local (process dictionary, local variables)
|
|
- You're using it for process cleanup that `start_supervised` does better
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
setup do
|
|
{:ok, pid} = MyWorker.start_link()
|
|
on_exit(fn -> Process.exit(pid, :kill) end)
|
|
%{worker: pid}
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
setup do
|
|
pid = start_supervised!(MyWorker)
|
|
%{worker: pid}
|
|
end
|
|
```
|
|
|
|
**Why:** `on_exit` is for *global* side effects that `start_supervised` can't handle. For process lifecycle, `start_supervised` is more robust — it handles ordering, linking, and shutdown properly.
|
|
|
|
---
|
|
|
|
## 6. Pattern Match Assertions
|
|
|
|
**Source:** [lib/ex_unit/lib/ex_unit/assertions.ex#L145](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/ex_unit/lib/ex_unit/assertions.ex#L145)
|
|
|
|
**What it does:** Uses `assert` with `=` for structural pattern matching in assertions.
|
|
|
|
**Why:** Provides rich failure messages showing both sides. More expressive than `assert x == y` when you only care about shape.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
# Assert structure, ignore specifics
|
|
assert {:ok, %{id: id}} = create_user("alice")
|
|
assert is_integer(id)
|
|
|
|
# Pin variables in patterns
|
|
x = 5
|
|
assert {:count, ^x} = get_counter()
|
|
|
|
# match? for complex guards
|
|
assert match?([%{id: id} | _] when is_integer(id), records)
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- You care about the shape/structure but not every field
|
|
- You want to bind a value from the result for further assertions
|
|
- The response has dynamic fields (IDs, timestamps) you can't predict
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
result = create_user("alice")
|
|
assert elem(result, 0) == :ok
|
|
user = elem(result, 1)
|
|
assert Map.has_key?(user, :id)
|
|
assert is_integer(user.id)
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
assert {:ok, %{id: id}} = create_user("alice")
|
|
assert is_integer(id)
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- You need to assert exact equality (use `==`)
|
|
- The pattern is so loose it would match unintended values
|
|
- You're asserting on a boolean or simple scalar
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Pattern match that matches too broadly
|
|
assert {:ok, _} = dangerous_operation()
|
|
# This passes for {:ok, nil}, {:ok, :error_actually}, anything!
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
assert {:ok, %User{active: true}} = dangerous_operation()
|
|
# Or when you need exact value:
|
|
assert create_user("alice") == {:ok, %User{name: "alice", active: true}}
|
|
```
|
|
|
|
**Why:** Pattern match assertions are for structural validation. If your pattern is so loose it can't distinguish success from unexpected values, you're not actually testing anything meaningful.
|
|
|
|
---
|
|
|
|
## 7. `assert_receive` / `refute_receive` for Process Communication
|
|
|
|
**Source:** [lib/ex_unit/lib/ex_unit/assertions.ex#L466](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/ex_unit/lib/ex_unit/assertions.ex#L466), [lib/elixir/test/elixir/process_test.exs#L90](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/process_test.exs#L90)
|
|
|
|
**What it does:** Waits for messages matching a pattern within a timeout (default 100ms).
|
|
|
|
**Why:** Tests asynchronous process communication without `Process.sleep`. The test either receives the expected message or fails with a helpful mailbox dump.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
# Basic message assertion
|
|
test "send_after/3 sends messages once expired" do
|
|
Process.send_after(self(), :hello, 10)
|
|
assert_receive :hello
|
|
end
|
|
|
|
# Pattern matching with pins
|
|
test "monitor/2 with monitor options" do
|
|
ref_and_alias = Process.monitor(pid, alias: :explicit_unalias)
|
|
send(pid, {:ping, ref_and_alias})
|
|
assert_receive :pong
|
|
assert_receive {:DOWN, ^ref_and_alias, _, _, _}
|
|
end
|
|
|
|
# Negative assertion
|
|
test "exit(pid, :normal) does not cause the target process to exit" do
|
|
Process.exit(pid, :normal)
|
|
refute_receive {:EXIT, ^pid, :normal}, 100
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Testing async process communication (messages, monitors, links)
|
|
- You need to verify a message was sent without blocking indefinitely
|
|
- Testing pub/sub, GenServer casts, or event broadcasts
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
test "notifies subscriber" do
|
|
subscribe(self(), :topic)
|
|
publish(:topic, "hello")
|
|
Process.sleep(100) # Hope it arrived...
|
|
assert_received {:topic, "hello"} # assert_received checks mailbox NOW
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
test "notifies subscriber" do
|
|
subscribe(self(), :topic)
|
|
publish(:topic, "hello")
|
|
assert_receive {:topic, "hello"}, 500 # Waits UP TO 500ms
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The operation is synchronous (use regular `assert` with the return value)
|
|
- You're testing a GenServer.call (it already returns the result synchronously)
|
|
- The default 100ms timeout would make tests flaky — consider a longer timeout or redesigning
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Using assert_receive for a synchronous operation
|
|
test "get user" do
|
|
send(self(), {:user, get_user(1)}) # Why send to self?
|
|
assert_receive {:user, %User{id: 1}}
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
test "get user" do
|
|
assert %User{id: 1} = get_user(1)
|
|
end
|
|
```
|
|
|
|
**Why:** `assert_receive` is for genuinely async communication — messages that arrive independently of the test's control flow. For synchronous return values, just assert on the return directly.
|
|
|
|
---
|
|
|
|
## 8. Testing GenServers via Public API (No Internal State Inspection)
|
|
|
|
**Source:** [lib/elixir/test/elixir/gen_server_test.exs#L87](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/gen_server_test.exs#L87)
|
|
|
|
**What it does:** Tests GenServer behavior exclusively through `GenServer.call/cast/stop` — never peeks at internal state.
|
|
|
|
**Why:** Tests the contract, not the implementation. Internal state changes don't break tests.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
test "start_link/2, call/2 and cast/2" do
|
|
{:ok, pid} = GenServer.start_link(Stack, [:hello])
|
|
|
|
assert GenServer.call(pid, :pop) == :hello
|
|
assert GenServer.cast(pid, {:push, :world}) == :ok
|
|
assert GenServer.call(pid, :pop) == :world
|
|
assert GenServer.stop(pid) == :ok
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Testing any GenServer, Agent, or stateful process
|
|
- You want tests that survive internal refactoring
|
|
- The process has a well-defined public API
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
test "push adds to stack" do
|
|
{:ok, pid} = Stack.start_link([:hello])
|
|
GenServer.cast(pid, {:push, :world})
|
|
# Peeking at internal state — couples test to implementation
|
|
assert :sys.get_state(pid) == [:world, :hello]
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
test "push adds to stack" do
|
|
{:ok, pid} = Stack.start_link([:hello])
|
|
GenServer.cast(pid, {:push, :world})
|
|
assert GenServer.call(pid, :pop) == :world
|
|
assert GenServer.call(pid, :pop) == :hello
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- You're debugging a specific state corruption bug (temporarily peek, then fix)
|
|
- The process has no public query API and adding one just for tests would bloat the interface
|
|
- You're testing internal state transitions explicitly (state machine verification)
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Avoiding state inspection when there's no query API
|
|
test "handles concurrent updates" do
|
|
cast(pid, {:increment, 5})
|
|
cast(pid, {:increment, 3})
|
|
# No way to observe the result without :sys.get_state or adding a query call
|
|
Process.sleep(100)
|
|
# ...can't assert anything useful
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
# Add a minimal query function to the public API
|
|
test "handles concurrent updates" do
|
|
cast(pid, {:increment, 5})
|
|
cast(pid, {:increment, 3})
|
|
assert call(pid, :get_count) == 8
|
|
end
|
|
```
|
|
|
|
**Why:** The principle is "test through the API," not "never observe state." If your process lacks observability, add a read function — that's a better public API, not just a test convenience.
|
|
|
|
---
|
|
|
|
## 9. `catch_exit` for Testing Process Failures
|
|
|
|
**Source:** [lib/ex_unit/lib/ex_unit/assertions.ex#L950](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/ex_unit/lib/ex_unit/assertions.ex#L950), [lib/elixir/test/elixir/gen_server_test.exs#L118](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/gen_server_test.exs#L118)
|
|
|
|
**What it does:** Catches exit signals from linked processes for assertion, or uses `Process.flag(:trap_exit, true)` + `assert_receive {:EXIT, ...}`.
|
|
|
|
**Why:** Testing error conditions in OTP requires intercepting exit signals. The two approaches serve different needs.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
# catch_exit for synchronous exit testing
|
|
test "call/3 exit messages" do
|
|
assert catch_exit(GenServer.call(pid, :noreply, 1)) ==
|
|
{:timeout, {GenServer, :call, [pid, :noreply, 1]}}
|
|
end
|
|
|
|
# trap_exit for linked process exits
|
|
test "exits on task error" do
|
|
Process.flag(:trap_exit, true)
|
|
task = Task.async(fn -> raise "oops" end)
|
|
assert {{%RuntimeError{}, _}, {Task, :await, [^task, 5000]}} = catch_exit(Task.await(task))
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Testing that a GenServer call times out or exits with a specific reason
|
|
- Testing that linked processes propagate exit signals correctly
|
|
- Verifying OTP shutdown behavior and exit reasons
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
test "times out" do
|
|
# Just assert it raises... but exit != raise
|
|
assert_raise RuntimeError, fn ->
|
|
GenServer.call(pid, :slow, 1)
|
|
end
|
|
# This fails! Exits aren't raises.
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
test "times out" do
|
|
assert catch_exit(GenServer.call(pid, :slow, 1)) ==
|
|
{:timeout, {GenServer, :call, [pid, :slow, 1]}}
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The function raises an exception (use `assert_raise` instead)
|
|
- You can test the error through the public API's return value `{:error, reason}`
|
|
- The exit is a side effect you don't care about (just test the observable behavior)
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Using catch_exit when the API already returns error tuples
|
|
test "handles missing key" do
|
|
assert catch_exit(GenServer.call(pid, {:get, :missing}))
|
|
# The server actually returns {:error, :not_found} — no exit!
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
test "handles missing key" do
|
|
assert GenServer.call(pid, {:get, :missing}) == {:error, :not_found}
|
|
end
|
|
```
|
|
|
|
**Why:** `catch_exit` is specifically for OTP exit signals (`:timeout`, `:noproc`, `:normal`, `{:shutdown, reason}`). If the function returns error tuples rather than exiting, assert on the return value directly.
|
|
|
|
---
|
|
|
|
## 10. `@tag capture_log: true` for Suppressing Expected Log Output
|
|
|
|
**Source:** [lib/elixir/test/elixir/gen_server_test.exs#L114](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/gen_server_test.exs#L114), [lib/elixir/test/elixir/task_test.exs#L10](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/task_test.exs#L10)
|
|
|
|
**What it does:** Captures log output during the test, only printing it if the test fails.
|
|
|
|
**Why:** Tests that intentionally trigger error conditions produce noisy log output. Capturing keeps the test output clean while preserving diagnostics on failure.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
# Per-test tag
|
|
@tag capture_log: true
|
|
test "call/3 exit messages" do
|
|
# This test triggers error logs — they're captured
|
|
end
|
|
|
|
# Module-level for all tests
|
|
@moduletag :capture_log
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Tests intentionally trigger error paths that log warnings/errors
|
|
- Test output is noisy with expected error messages
|
|
- You want clean `mix test` output but need logs preserved for debugging failures
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
test "handles crash" do
|
|
# Passes, but test output shows:
|
|
# [error] GenServer #PID<0.123.0> terminating
|
|
# [error] ** (RuntimeError) intentional crash
|
|
crash_worker(pid)
|
|
assert_receive {:restarted, _}
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
@tag capture_log: true
|
|
test "handles crash" do
|
|
crash_worker(pid)
|
|
assert_receive {:restarted, _}
|
|
# Logs captured — only shown if this test FAILS
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- You want to assert on log content (use `capture_log/2` function instead)
|
|
- The logs indicate a real problem you should fix, not expected behavior
|
|
- You're hiding logs to mask flaky tests
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
@moduletag :capture_log # Blanket capture on entire module
|
|
|
|
test "creates user" do
|
|
# This test logs "[warn] duplicate email check" — is that expected?
|
|
# By capturing everything, you might miss real warnings
|
|
assert {:ok, _} = create_user(attrs)
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
# Only capture on tests that INTENTIONALLY trigger errors
|
|
test "creates user" do
|
|
assert {:ok, _} = create_user(attrs)
|
|
# If there's a warning, investigate it — don't hide it
|
|
end
|
|
|
|
@tag capture_log: true
|
|
test "rejects duplicate email" do
|
|
create_user(attrs)
|
|
assert {:error, :duplicate} = create_user(attrs)
|
|
# Expected warning about duplicate — captured
|
|
end
|
|
```
|
|
|
|
**Why:** Blanket log capture hides signals. Apply `capture_log` surgically to tests where you *expect* error output. Unexpected logs in other tests might reveal bugs.
|
|
|
|
---
|
|
|
|
## 11. `capture_log` / `capture_io` for Content Assertions
|
|
|
|
**Source:** [lib/ex_unit/lib/ex_unit/capture_log.ex#L1](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/ex_unit/lib/ex_unit/capture_log.ex#L1), [lib/elixir/test/elixir/task_test.exs#L1138](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/task_test.exs#L1138)
|
|
|
|
**What it does:** Captures log/IO output and returns it as a string for assertion.
|
|
|
|
**Why:** Tests that the right messages are logged/printed without relying on side effects.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
# capture_log for asserting log content
|
|
test "logs a terminated task" do
|
|
assert ExUnit.CaptureLog.capture_log(fn ->
|
|
ref = Process.monitor(pid)
|
|
send(pid, :go)
|
|
receive do: ({:DOWN, ^ref, _, _, _} -> :ok)
|
|
end) =~ ~r/Task .* terminating/
|
|
end
|
|
|
|
# with_io returns both result and output (since v1.13)
|
|
{result, output} = with_io(fn ->
|
|
IO.puts("a")
|
|
2 + 2
|
|
end)
|
|
assert result == 4
|
|
assert output == "a\n"
|
|
```
|
|
|
|
**Important for async tests:** Use `=~` instead of `==` for `:stderr` captures because output from other tests may interleave.
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- You need to verify specific log messages are emitted
|
|
- Testing CLI output or formatted display
|
|
- Verifying error messages are user-friendly
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
test "logs warning on retry" do
|
|
# Just call it and... hope the log is right?
|
|
retry_request(url)
|
|
# No assertion on the log content
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
test "logs warning on retry" do
|
|
log = capture_log(fn -> retry_request(url) end)
|
|
assert log =~ "retrying request"
|
|
assert log =~ url
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- You just want to suppress noisy logs (use `@tag capture_log: true`)
|
|
- The log message is an implementation detail that shouldn't be part of the contract
|
|
- Testing the behavior is sufficient — the log is incidental
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
test "creates user" do
|
|
log = capture_log(fn ->
|
|
assert {:ok, user} = create_user(attrs)
|
|
end)
|
|
assert log =~ "INSERT INTO users" # Testing SQL logs?! Fragile.
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
test "creates user" do
|
|
assert {:ok, user} = create_user(attrs)
|
|
assert user.name == "Alice"
|
|
# SQL logs are an implementation detail — don't assert on them
|
|
end
|
|
```
|
|
|
|
**Why:** Assert on logs that are *part of the contract* (user-facing warnings, audit trail). Don't assert on incidental logs that change with implementation — they make tests brittle.
|
|
|
|
---
|
|
|
|
## 12. `describe` Blocks for Logical Grouping
|
|
|
|
**Source:** `lib/elixir/test/elixir/task_test.exs:218,272,365`, [lib/elixir/test/elixir/process_test.exs#L146](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/process_test.exs#L146)
|
|
|
|
**What it does:** Groups related tests under a named describe block. Setup inside describe only applies to that group.
|
|
|
|
**Why:** Organizes tests by function/feature. Makes test output readable. Allows scoped `@describetag` and scoped setup.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
describe "await/2" do
|
|
test "exits on timeout" do
|
|
task = %Task{ref: make_ref(), owner: self(), pid: nil, mfa: {__MODULE__, :test, 1}}
|
|
assert catch_exit(Task.await(task, 0)) == {:timeout, {Task, :await, [task, 0]}}
|
|
end
|
|
|
|
test "exits on normal exit" do
|
|
task = Task.async(fn -> exit(:normal) end)
|
|
assert catch_exit(Task.await(task)) == {:normal, {Task, :await, [task, 5000]}}
|
|
end
|
|
end
|
|
```
|
|
|
|
**Constraint:** Describe blocks cannot be nested. `setup_all` cannot appear inside describe.
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- A module tests multiple public functions
|
|
- Tests share setup that's specific to one function/feature
|
|
- You want test output organized by feature (e.g., "describe await/2 — exits on timeout")
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
test "push adds element" do ... end
|
|
test "push returns :ok" do ... end
|
|
test "pop returns top element" do ... end
|
|
test "pop from empty raises" do ... end
|
|
# Flat list — hard to see which tests cover which function
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
describe "push/2" do
|
|
test "adds element to the top" do ... end
|
|
test "returns :ok" do ... end
|
|
end
|
|
|
|
describe "pop/1" do
|
|
test "returns top element" do ... end
|
|
test "raises on empty stack" do ... end
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The module tests one function (describe adds an unnecessary nesting level)
|
|
- You'd have a describe block with a single test in it
|
|
- You need nested grouping (describe can't nest — use separate modules)
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
describe "parse/1" do
|
|
test "parses input" do
|
|
assert parse("hello") == {:ok, "hello"}
|
|
end
|
|
end
|
|
# One test in a describe — just use a standalone test
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
test "parse/1 parses input" do
|
|
assert parse("hello") == {:ok, "hello"}
|
|
end
|
|
```
|
|
|
|
**Why:** `describe` provides structure when there are multiple tests per feature. A single test in a describe is over-organization — the describe name adds visual noise without grouping benefit.
|
|
|
|
---
|
|
|
|
## 13. `ExUnit.CaseTemplate` for Shared Test Infrastructure
|
|
|
|
**Source:** [lib/mix/test/test_helper.exs#L79](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/mix/test/test_helper.exs#L79), [lib/logger/test/test_helper.exs#L24](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/logger/test/test_helper.exs#L24)
|
|
|
|
**What it does:** Defines reusable test case templates with shared setup, helpers, and imports.
|
|
|
|
**Why:** Eliminates duplication across test modules. Provides domain-specific test DSLs.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
# In test_helper.exs
|
|
defmodule Logger.Case do
|
|
use ExUnit.CaseTemplate
|
|
|
|
using _ do
|
|
quote do
|
|
import Logger.Case
|
|
end
|
|
end
|
|
|
|
setup do
|
|
on_exit(fn ->
|
|
# Shared cleanup for all tests using this template
|
|
end)
|
|
:ok
|
|
end
|
|
|
|
def capture_log(level \\ :debug, fun) do
|
|
Logger.configure(level: level)
|
|
capture_io(:user, fn ->
|
|
fun.()
|
|
Logger.flush()
|
|
end)
|
|
after
|
|
Logger.configure(level: :debug)
|
|
end
|
|
end
|
|
|
|
# In test file:
|
|
defmodule LoggerTest do
|
|
use Logger.Case
|
|
# Gets all imports and setup from the template
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Multiple test modules share the same setup/teardown logic
|
|
- You're building a test DSL (e.g., `DataCase` for database tests in Phoenix)
|
|
- Shared helpers need to be imported into every test module of a certain type
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
# Repeated in every test module that uses the database
|
|
defmodule UsersTest do
|
|
use ExUnit.Case
|
|
setup do
|
|
:ok = Ecto.Adapters.SQL.Sandbox.checkout(Repo)
|
|
Ecto.Adapters.SQL.Sandbox.mode(Repo, {:shared, self()})
|
|
end
|
|
end
|
|
|
|
defmodule PostsTest do
|
|
use ExUnit.Case
|
|
setup do
|
|
:ok = Ecto.Adapters.SQL.Sandbox.checkout(Repo) # Same thing again
|
|
Ecto.Adapters.SQL.Sandbox.mode(Repo, {:shared, self()})
|
|
end
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
defmodule MyApp.DataCase do
|
|
use ExUnit.CaseTemplate
|
|
|
|
setup do
|
|
:ok = Ecto.Adapters.SQL.Sandbox.checkout(Repo)
|
|
Ecto.Adapters.SQL.Sandbox.mode(Repo, {:shared, self()})
|
|
end
|
|
end
|
|
|
|
defmodule UsersTest do
|
|
use MyApp.DataCase # All DB setup handled
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- Only one or two test modules share the setup (just use named setup functions)
|
|
- The template would have many options making it hard to understand what's actually set up
|
|
- You're hiding important test context behind abstraction
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
defmodule MyApp.SuperCase do
|
|
use ExUnit.CaseTemplate
|
|
using do
|
|
quote do
|
|
import MyApp.Factory
|
|
import MyApp.Assertions
|
|
import MyApp.Helpers
|
|
alias MyApp.{Repo, User, Post, Comment, Tag}
|
|
# 20 more aliases...
|
|
end
|
|
end
|
|
# Every test gets everything whether it needs it or not
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
# Focused templates for specific test types
|
|
defmodule MyApp.DataCase do ... end # Database tests
|
|
defmodule MyApp.ConnCase do ... end # HTTP tests
|
|
defmodule MyApp.ChannelCase do ... end # WebSocket tests
|
|
```
|
|
|
|
**Why:** A "god template" that imports everything creates implicit dependencies and makes it impossible to know what a test actually needs. Use focused templates that match distinct test categories.
|
|
|
|
---
|
|
|
|
## 14. `doctest` Integration
|
|
|
|
**Source:** [lib/ex_unit/lib/ex_unit/doc_test.ex#L1](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/ex_unit/lib/ex_unit/doc_test.ex#L1), [lib/elixir/test/elixir/agent_test.exs#L9](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/agent_test.exs#L9)
|
|
|
|
**What it does:** Generates tests from `@doc` and `@moduledoc` code examples.
|
|
|
|
**Why:** Documentation examples are always verified. Prevents docs from rotting.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
defmodule AgentTest do
|
|
use ExUnit.Case, async: true
|
|
doctest Agent
|
|
end
|
|
|
|
# Selective doctesting:
|
|
doctest Kernel, except: [===: 2, !==: 2, and: 2, or: 2]
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Your module has `iex>` examples in `@doc` or `@moduledoc`
|
|
- You want to ensure documentation examples stay correct across refactors
|
|
- The function has simple, deterministic input/output that's easy to show in docs
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
@doc """
|
|
Doubles a number.
|
|
|
|
iex> double(5)
|
|
10
|
|
"""
|
|
def double(n), do: n * 2
|
|
# No doctest — this example might rot if you rename the function
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
# In test file:
|
|
defmodule MathTest do
|
|
use ExUnit.Case, async: true
|
|
doctest MyApp.Math
|
|
# Now the iex> example is verified on every test run
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- Examples require complex setup (database, processes, external state)
|
|
- Output is non-deterministic (timestamps, random values, PIDs)
|
|
- The function's behavior is better tested with dedicated unit tests
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
@doc """
|
|
Creates a user.
|
|
|
|
iex> {:ok, user} = create_user(%{name: "Alice"})
|
|
iex> user.id
|
|
1
|
|
"""
|
|
# User ID is auto-incremented — this breaks on second run!
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
@doc """
|
|
Creates a user.
|
|
|
|
{:ok, user} = MyApp.create_user(%{name: "Alice"})
|
|
user.name
|
|
#=> "Alice"
|
|
|
|
"""
|
|
# Don't use iex> prefix — this is illustrative, not a doctest
|
|
# Test the actual behavior in a proper test with DB setup
|
|
```
|
|
|
|
**Why:** Doctests are for deterministic, setup-free examples. If the example needs a database, process, or produces non-deterministic output, write a proper test — doctests can't handle setup/teardown.
|
|
|
|
---
|
|
|
|
## 15. `Process.sleep(:infinity)` as a Process Parking Pattern
|
|
|
|
**Source:** [lib/elixir/test/elixir/task_test.exs#L417](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/task_test.exs#L417), [lib/elixir/test/elixir/registry_test.exs#L71](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/registry_test.exs#L71)
|
|
|
|
**What it does:** Spawns processes that block forever, used as test subjects that need to exist until explicitly killed.
|
|
|
|
**Why:** Creates stable process references for testing supervision, monitoring, and registry behavior. The process stays alive until the test supervisor shuts it down.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
# Process exists solely to be registered/monitored
|
|
{:ok, task} =
|
|
Task.start(fn ->
|
|
send(parent, Registry.register(registry, key, value))
|
|
Process.sleep(:infinity)
|
|
end)
|
|
|
|
# Then kill it to test cleanup:
|
|
Process.exit(task, :kill)
|
|
assert_receive {:DOWN, ^ref, _, _, _}
|
|
```
|
|
|
|
**Important distinction:** This is NOT `Process.sleep(100)` for timing — it's an intentional "park this process" pattern where the process is always explicitly terminated by the test.
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- You need a process that exists as a test subject (for monitoring, registry, supervision tests)
|
|
- The process doesn't need to do anything — just exist and be killable
|
|
- Testing cleanup/shutdown behavior when a process dies
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
test "monitors processes" do
|
|
pid = spawn(fn ->
|
|
receive do: (:stop -> :ok) # Must send :stop to clean up
|
|
end)
|
|
ref = Process.monitor(pid)
|
|
send(pid, :stop)
|
|
assert_receive {:DOWN, ^ref, _, _, :normal}
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
test "monitors processes" do
|
|
pid = spawn(fn -> Process.sleep(:infinity) end)
|
|
ref = Process.monitor(pid)
|
|
Process.exit(pid, :kill)
|
|
assert_receive {:DOWN, ^ref, _, _, :killed}
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- You need the process to actually do something (send messages, handle calls)
|
|
- You're using it as a timing mechanism (`Process.sleep(100)` for "wait a bit")
|
|
- The process should be managed by `start_supervised` (which handles cleanup)
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
test "worker processes requests" do
|
|
pid = spawn(fn ->
|
|
Process.sleep(:infinity) # But the test needs it to handle messages!
|
|
end)
|
|
send(pid, {:process, data})
|
|
assert_receive {:result, _} # Never arrives — process is sleeping!
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
test "worker processes requests" do
|
|
pid = spawn(fn ->
|
|
receive do
|
|
{:process, data} -> send(parent, {:result, transform(data)})
|
|
end
|
|
end)
|
|
send(pid, {:process, data})
|
|
assert_receive {:result, _}
|
|
end
|
|
```
|
|
|
|
**Why:** `Process.sleep(:infinity)` is for *inert* test subjects — processes that exist to be observed, not to perform work. If the test needs the process to respond, it needs a receive loop, not a sleep.
|
|
|
|
---
|
|
|
|
## 16. Helper Functions for Test-Specific Behavior
|
|
|
|
**Source:** [lib/elixir/test/elixir/task_test.exs#L12](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/task_test.exs#L12), [lib/elixir/test/elixir/supervisor_test.exs#L278](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/supervisor_test.exs#L278)
|
|
|
|
**What it does:** Defines private helper functions within test modules for common test operations.
|
|
|
|
**Why:** Keeps tests DRY without over-abstracting. Helpers like `wait_until_down`, `assert_kill`, `create_dummy_task` encapsulate recurring patterns.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
defmodule TaskTest do
|
|
use ExUnit.Case
|
|
|
|
# Helper to create a known-state task for testing edge cases
|
|
defp create_dummy_task(reason) do
|
|
{pid, ref} = spawn_monitor(Kernel, :exit, [reason])
|
|
receive do
|
|
{:DOWN, ^ref, _, _, _} ->
|
|
%Task{ref: ref, pid: pid, owner: self(), mfa: {__MODULE__, :create_dummy_task, 1}}
|
|
end
|
|
end
|
|
|
|
# Helper that properly waits for process termination
|
|
def wait_until_down(task) do
|
|
ref = Process.monitor(task.pid)
|
|
assert_receive {:DOWN, ^ref, _, _, _}
|
|
end
|
|
|
|
# Helper for asserting process kill
|
|
defp assert_kill(pid, reason) do
|
|
ref = Process.monitor(pid)
|
|
Process.exit(pid, reason)
|
|
assert_receive {:DOWN, ^ref, _, _, _}
|
|
end
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- The same 3-5 line pattern repeats across multiple tests in the module
|
|
- The helper name makes tests more readable by expressing intent
|
|
- Setup logic is complex enough to obscure the test's actual assertion
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
test "cleans up on crash 1" do
|
|
ref = Process.monitor(pid1)
|
|
Process.exit(pid1, :kill)
|
|
assert_receive {:DOWN, ^ref, _, _, _}
|
|
assert Registry.lookup(reg, :key) == []
|
|
end
|
|
|
|
test "cleans up on crash 2" do
|
|
ref = Process.monitor(pid2)
|
|
Process.exit(pid2, :kill)
|
|
assert_receive {:DOWN, ^ref, _, _, _} # Same 3 lines again
|
|
assert Registry.lookup(reg, :key) == []
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
defp kill_and_wait(pid) do
|
|
ref = Process.monitor(pid)
|
|
Process.exit(pid, :kill)
|
|
assert_receive {:DOWN, ^ref, _, _, _}
|
|
end
|
|
|
|
test "cleans up on crash 1" do
|
|
kill_and_wait(pid1)
|
|
assert Registry.lookup(reg, :key) == []
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The "helper" is used only once (inline it)
|
|
- The helper hides important test details that readers need to see
|
|
- Over-abstraction makes tests harder to understand in isolation
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
defp setup_and_assert(input, expected) do
|
|
{:ok, pid} = start_supervised({Worker, input})
|
|
result = Worker.process(pid)
|
|
assert result == expected
|
|
end
|
|
|
|
test "processes integers", do: setup_and_assert(42, {:ok, 42})
|
|
test "processes strings", do: setup_and_assert("hi", {:ok, "hi"})
|
|
# Tests are now opaque — can't see what's actually being tested
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
test "processes integers" do
|
|
pid = start_supervised!({Worker, 42})
|
|
assert Worker.process(pid) == {:ok, 42}
|
|
end
|
|
|
|
test "processes strings" do
|
|
pid = start_supervised!({Worker, "hi"})
|
|
assert Worker.process(pid) == {:ok, "hi"}
|
|
end
|
|
```
|
|
|
|
**Why:** Test helpers should extract *mechanics* (kill-and-wait, setup-server), not *logic* (the actual behavior under test). If a helper contains assertions, readers can't understand the test without reading the helper.
|
|
|
|
---
|
|
|
|
## 17. `@tag :tmp_dir` for Filesystem Tests
|
|
|
|
**Source:** [lib/ex_unit/lib/ex_unit/case.ex#L281](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/ex_unit/lib/ex_unit/case.ex#L281), [lib/elixir/test/elixir/path_test.exs#L12](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/path_test.exs#L12)
|
|
|
|
**What it does:** ExUnit automatically creates a unique temporary directory and passes its path via the test context.
|
|
|
|
**Why:** Filesystem tests need isolation. Each test gets its own directory, removed before creation to ensure a clean slate.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
@tag :tmp_dir
|
|
test "writes files", %{tmp_dir: tmp_dir} do
|
|
path = Path.join(tmp_dir, "test.txt")
|
|
File.write!(path, "hello")
|
|
assert File.read!(path) == "hello"
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Tests create, read, or modify files
|
|
- Multiple tests would conflict if they used the same paths
|
|
- You want filesystem tests to run with `async: true`
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
test "writes config file" do
|
|
path = "/tmp/test_config.json"
|
|
File.write!(path, "{}")
|
|
# Another async test writes to the same path — race condition!
|
|
assert File.read!(path) == "{}"
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
@tag :tmp_dir
|
|
test "writes config file", %{tmp_dir: dir} do
|
|
path = Path.join(dir, "config.json")
|
|
File.write!(path, "{}")
|
|
assert File.read!(path) == "{}"
|
|
# Unique directory per test — safe for async
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- Tests only read files (no isolation needed for read-only access)
|
|
- You're testing in-memory operations that happen to involve path strings
|
|
- The filesystem interaction is mocked/stubbed
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
@tag :tmp_dir
|
|
test "parses path components", %{tmp_dir: _dir} do
|
|
# Doesn't actually touch the filesystem!
|
|
assert Path.basename("/foo/bar/baz.txt") == "baz.txt"
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
test "parses path components" do
|
|
assert Path.basename("/foo/bar/baz.txt") == "baz.txt"
|
|
end
|
|
```
|
|
|
|
**Why:** `@tag :tmp_dir` creates actual directories on disk — it's overhead. Only use it when the test genuinely needs filesystem isolation. Pure path manipulation doesn't touch the filesystem.
|
|
|
|
---
|
|
|
|
## 18. `assert_raise` with Message Matching
|
|
|
|
**Source:** [lib/ex_unit/lib/ex_unit/assertions.ex#L815](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/ex_unit/lib/ex_unit/assertions.ex#L815)
|
|
|
|
**What it does:** Asserts both the exception type AND the message content (string or regex).
|
|
|
|
**Why:** Verifying the exception type alone is insufficient — the message tells users what went wrong. Testing it ensures error UX.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
# Exact message match
|
|
assert_raise ArgumentError, ~r"expected :name option to be one of the following:", fn ->
|
|
GenServer.start_link(Stack, [:hello], name: "my_gen_server_name")
|
|
end
|
|
|
|
# Regex for dynamic content
|
|
assert_raise RuntimeError, ~r/^today's lucky number is 0\.\d+!$/, fn ->
|
|
raise "today's lucky number is #{:rand.uniform()}!"
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Testing error messages that users will see
|
|
- Validating that errors provide actionable information
|
|
- The error type alone isn't specific enough (many places raise `ArgumentError`)
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
test "rejects bad input" do
|
|
assert_raise ArgumentError, fn ->
|
|
parse("not_a_number")
|
|
end
|
|
# Passes even if the message is wrong or unhelpful
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
test "rejects bad input" do
|
|
assert_raise ArgumentError, ~r/cannot parse .* as integer/, fn ->
|
|
parse("not_a_number")
|
|
end
|
|
# Verifies the error message is helpful
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The exact message text is an implementation detail that changes often
|
|
- You're testing a third-party library's error messages (they might change)
|
|
- The exception type alone is sufficient to distinguish the error case
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
test "file not found" do
|
|
assert_raise File.Error, "could not read file \"/no/such/file\": no such file or directory", fn ->
|
|
File.read!("/no/such/file")
|
|
end
|
|
# Exact message match breaks across OS versions / locales
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
test "file not found" do
|
|
assert_raise File.Error, fn ->
|
|
File.read!("/no/such/file")
|
|
end
|
|
# Or with regex for the stable part:
|
|
assert_raise File.Error, ~r/could not read file/, fn ->
|
|
File.read!("/no/such/file")
|
|
end
|
|
end
|
|
```
|
|
|
|
**Why:** Match the *stable, meaningful* part of error messages. OS-specific paths, locale-dependent strings, and implementation details make exact matches brittle across environments.
|
|
|
|
---
|
|
|
|
## 19. `@moduletag` / `@describetag` for Cross-Cutting Configuration
|
|
|
|
**Source:** `lib/elixir/test/elixir/system_test.exs:104,163`, [lib/elixir/test/elixir/task_test.exs#L10](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/task_test.exs#L10)
|
|
|
|
**What it does:** Sets tags that apply to all tests in a module or describe block, used for filtering and configuration.
|
|
|
|
**Why:** Enables running subsets of tests (`mix test --include unix`) and applying configuration (like `:capture_log`) without repeating it on every test.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
defmodule SystemTest do
|
|
use ExUnit.Case, async: true
|
|
|
|
describe "Windows" do
|
|
@describetag :windows
|
|
# All tests here tagged :windows
|
|
end
|
|
|
|
describe "Unix" do
|
|
@describetag :unix
|
|
# All tests here tagged :unix
|
|
end
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Tests only run on certain platforms (OS, architecture)
|
|
- You want to run a subset of tests via `--include` / `--exclude`
|
|
- A whole module or describe block shares configuration (capture_log, tmp_dir)
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
# Manually skipping in each test
|
|
test "symlinks" do
|
|
if :os.type() != {:unix, :linux}, do: ExUnit.skip("unix only")
|
|
# ...
|
|
end
|
|
|
|
test "permissions" do
|
|
if :os.type() != {:unix, :linux}, do: ExUnit.skip("unix only")
|
|
# ...
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
describe "Unix filesystem" do
|
|
@describetag :unix
|
|
|
|
test "symlinks" do ... end
|
|
test "permissions" do ... end
|
|
end
|
|
|
|
# In test_helper.exs:
|
|
ExUnit.configure(exclude: [:unix], include: [])
|
|
# Run with: mix test --include unix
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The tag applies to a single test (use `@tag` instead)
|
|
- You're using tags as a substitute for proper test organization
|
|
- The tag doesn't enable filtering or configuration — it's just metadata no one reads
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
@moduletag :unit
|
|
@moduletag :fast
|
|
@moduletag :users
|
|
@moduletag :important
|
|
# Tags no one filters on — just noise in the module header
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
# Only tag what you actually filter on
|
|
@moduletag :capture_log # Used by ExUnit
|
|
# If you never run `--include unit`, don't tag it
|
|
```
|
|
|
|
**Why:** Tags have cost — they clutter the module header and create the illusion of organization. Only add tags that drive behavior (ExUnit configuration) or that you actually filter on in CI/development.
|
|
|
|
---
|
|
|
|
## 20. Context Pattern Matching in Test Signatures
|
|
|
|
**Source:** [lib/ex_unit/lib/ex_unit/case.ex#L57](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/ex_unit/lib/ex_unit/case.ex#L57), [lib/elixir/test/elixir/gen_server_test.exs#L166](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/test/elixir/gen_server_test.exs#L166)
|
|
|
|
**What it does:** Destructures the test context directly in the test function signature.
|
|
|
|
**Why:** Makes dependencies explicit. You see exactly what each test needs from setup.
|
|
|
|
**Pattern:**
|
|
```elixir
|
|
test "abcast/3", %{test: name} do
|
|
{:ok, _} = GenServer.start_link(Stack, [], name: name)
|
|
assert GenServer.abcast(name, {:push, :hello}) == :abcast
|
|
end
|
|
|
|
# Using test name for unique naming — prevents collision in async tests
|
|
```
|
|
|
|
The `%{test: name}` pattern is ubiquitous — the test name is unique per module, making it perfect for naming registered processes in async tests.
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Your test needs values from `setup` (server pids, registry names, tmp dirs)
|
|
- You want to make test dependencies visible in the test signature
|
|
- Using `%{test: name}` for unique process registration in async tests
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
setup do
|
|
{:ok, pid} = start_supervised(MyServer)
|
|
%{server: pid, name: "test_user"}
|
|
end
|
|
|
|
test "queries server" do
|
|
# Where does 'server' come from? Must read setup.
|
|
server = ???
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
setup do
|
|
{:ok, pid} = start_supervised(MyServer)
|
|
%{server: pid, name: "test_user"}
|
|
end
|
|
|
|
test "queries server", %{server: pid, name: name} do
|
|
assert MyServer.get(pid, name) == :ok
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The test doesn't use any setup context
|
|
- You're destructuring the entire context when you only need one field
|
|
- The test is standalone and self-contained
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
test "basic math", %{test: _test} do
|
|
# Destructuring context for a test that doesn't need it
|
|
assert 2 + 2 == 4
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
test "basic math" do
|
|
assert 2 + 2 == 4
|
|
end
|
|
```
|
|
|
|
**Why:** Context destructuring signals "this test depends on external setup." If the test is self-contained, the pattern match is misleading — readers will look for setup that doesn't exist or isn't needed.
|
|
|
|
## Decision Tree
|
|
|
|
- If you are creating a new test module and need to decide on concurrency → [Module-Level Async Declaration](#1-module-level-async-declaration)
|
|
- If the same logic must work across multiple configurations or backends → [Parameterized Tests](#2-parameterized-tests)
|
|
- If your test needs a running process with guaranteed cleanup → [Setup with `start_supervised/2`](#3-setup-with-start_supervised2)
|
|
- If setup has multiple independent steps that different describe blocks reuse → [Named Setup Functions](#4-named-setup-functions-composable-pipelines)
|
|
- If your test modifies global state that must be restored regardless of outcome → [`on_exit` for Reversing Global Side Effects](#5-on_exit-for-reversing-global-side-effects)
|
|
- If you care about the shape/structure of a result but not every field → [Pattern Match Assertions](#6-pattern-match-assertions)
|
|
- If you need to test asynchronous message delivery between processes → [`assert_receive` / `refute_receive`](#7-assert_receive--refute_receive-for-process-communication)
|
|
- If you are testing a GenServer and want tests that survive refactoring → [Testing GenServers via Public API](#8-testing-genservers-via-public-api-no-internal-state-inspection)
|
|
- If you need to assert on OTP exit signals (timeouts, noproc, shutdown) → [`catch_exit`](#9-catch_exit-for-testing-process-failures)
|
|
- If tests intentionally trigger error paths that produce noisy log output → [`@tag capture_log: true`](#10-tag-capture_log-true-for-suppressing-expected-log-output)
|
|
- If you need to verify specific log or IO content was emitted → [`capture_log` / `capture_io`](#11-capture_log--capture_io-for-content-assertions)
|
|
- If a module tests multiple public functions and needs logical organization → [`describe` Blocks](#12-describe-blocks-for-logical-grouping)
|
|
- If multiple test modules share the same setup/teardown infrastructure → [`ExUnit.CaseTemplate`](#13-exunitcasetemplate-for-shared-test-infrastructure)
|
|
- If your module has `iex>` examples that should be verified automatically → [`doctest` Integration](#14-doctest-integration)
|
|
- If you need an inert process that exists only to be observed or killed → [`Process.sleep(:infinity)`](#15-processsleepinfinity-as-a-process-parking-pattern)
|
|
- If the same 3-5 line test pattern repeats across multiple tests → [Helper Functions](#16-helper-functions-for-test-specific-behavior)
|
|
- If tests create or modify files and need filesystem isolation → [`@tag :tmp_dir`](#17-tag-tmp_dir-for-filesystem-tests)
|
|
- If you need to verify both the exception type and the user-facing message → [`assert_raise` with Message Matching](#18-assert_raise-with-message-matching)
|
|
- If tests only run on certain platforms or you want to filter subsets → [`@moduletag` / `@describetag`](#19-moduletag--describetag-for-cross-cutting-configuration)
|
|
- If you want to make test dependencies on setup context explicit → [Context Pattern Matching](#20-context-pattern-matching-in-test-signatures)
|
|
|
|
<!-- PATTERN_COMPLETE -->
|