Compare commits

..

18 Commits

Author SHA1 Message Date
Rodin e75a97707d docs: add Logger.warn deprecation as common mistake 2026-05-10 21:24:24 -07:00
Rodin df9c856d96 patterns(testing): add async test filtering pattern (#21)
When testing telemetry, pub/sub, or any broadcast mechanism with
async: true, events from concurrent tests can leak into mailbox.

Fix: Pin-match on unique identifier to filter.

Two-part pattern:
1. Filter at source — only forward events matching test's identifier
2. Pin in assertion — ^variable rejects mismatches that slip through

Applies to telemetry handlers, PubSub, GenStage/Broadway consumers,
any shared message bus.

Triggered by PR #710 flaky test fix in gargoyle.
2026-05-09 18:27:05 -07:00
Rodin b833d05410 patterns: add telemetry emission patterns (dedicated submodule, naming, measurements vs metadata) 2026-05-07 19:28:48 -07:00
Rodin 6400d626dc docs: rewrite usage section as agent instructions
Frame the README for someone setting up a subagent that uses
this repo as a knowledge base. Three prompt templates:
solving problems, reviewing code, evaluating patterns.
2026-05-07 18:06:54 -07:00
Rodin d4c919f8df docs: add usage prompts for writing, reviewing, and evaluating 2026-05-07 18:06:07 -07:00
Rodin b4e3cf2824 docs: define patterns vs conventions in README
Patterns are prescriptive — follow them.
Conventions are descriptive — study for ideas.

Clarifies repo purpose, directory structure, and how to
use patterns during development and review.
2026-05-07 18:04:50 -07:00
Rodin 74101b513c chore: merge elixir-conventions and oban-conventions into sources/
Absorbed content from rodin/elixir-conventions and rodin/oban-conventions
into a sources/ directory. These are reference material — descriptive,
not prescriptive. Patterns that prove broadly applicable get promoted
into patterns/.

Part of taxonomy cleanup (issue #4):
- Pattern = prescriptive, follow these
- Convention/Source = reference, study for ideas

The original repos can now be archived.
2026-05-07 18:01:42 -07:00
OpenClaw f595b91030 fix: update drifted citation line numbers
- option_parser.ex: String.to_existing_atom/1 is at line 859, not 855
  (line 855 is the String.to_atom clause for allow_nonexistent_atoms)
- logger test_helper.exs: capture_log after clause spans lines 57-65,
  not 57-62 (the 'after' keyword is at line 64, restore at line 65)
2026-05-06 17:17:10 -07:00
Rodin 40f024b477 fix: update drifted source citations to match current upstream
Verified all 17 file:line citations against elixir-lang/elixir HEAD.
Fixed 10 citations where line numbers had shifted due to upstream changes:

- patterns/genserver.md: agent.ex:246 → 279 (start_link spec)
- patterns/process-design.md: task.ex:282 → 327 (child_spec)
- smells/anti-patterns.md: registry_test.exs:28 → 29, gen_server_test.exs:166 → 164,
  test_helper.exs:98 → 99
- smells/common-mistakes.md: registry_test.exs:28 → 29, callbacks.ex:423 → 433,
  task_test.exs:297,305,315,330 → 300,308,316,327,
  supervisor_test.exs:278 → 289, callbacks.ex:277 → 520
2026-05-06 16:33:21 -07:00
rodin e989536bfb Merge pull request 'docs: add rule for when @impl functions earn their own @doc' (#2) from docs/impl-doc-override-rule into master 2026-05-02 17:18:15 +00:00
aweiker d3adeaac25 fix: standardize cross-reference link text for Pattern 10 subsection 2026-05-02 10:13:56 -07:00
aweiker edef02ed0f docs: add rule for when @impl functions earn their own @doc
Pattern 10 (Callback Documentation Convention) now owns the full rule
for callback documentation — both the behaviour side (@callback docs)
and the implementation side (when to override @doc false on @impl
functions). Patterns 2 and 5 cross-reference Pattern 10 instead of
making their own partial statements.

The test: "Would this statement be true of any implementation?" If yes,
it belongs on the @callback. If no, the implementation earns its own
@doc.
2026-05-02 10:03:56 -07:00
rodin 8e77a5e321 Merge pull request 'feat: add Ecto patterns extracted from elixir-ecto/ecto source' (#1) from feat/ecto-patterns into master 2026-05-02 05:48:16 +00:00
aweiker 10218813d3 docs: backfill TOC + decision trees, fix review findings
- Add ## Contents and ## Decision Tree to all 10 existing pattern files
- Fix embed_as/1 semantics inversion in types.md (:self → :dump)
- Fix fabricated __meta__.changes reference in changesets.md
- Fix default primary key type (:integer → :id) in schemas.md
- Combine @impl subsections into single "Minimal Callback Annotation"
2026-05-01 22:13:35 -07:00
aweiker b33accf37c docs: extend @impl pattern with multi-clause and @spec guidance 2026-05-01 21:29:47 -07:00
aweiker 49a006dd18 docs: add table of contents to all Ecto pattern files 2026-05-01 20:55:15 -07:00
aweiker d28b9c8844 fix: use Repo.transaction instead of non-existent Repo.transact 2026-05-01 20:41:06 -07:00
aweiker 7b38ac9b2a feat: add Ecto patterns extracted from elixir-ecto/ecto source
Adds 5 new pattern files covering the core Ecto APIs, each with real
source citations at commit fd2ec52b, before/after examples, anti-patterns,
and decision trees.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-01 19:22:39 -07:00
23 changed files with 5982 additions and 20 deletions
+50 -7
View File
@@ -1,15 +1,58 @@
# Elixir Patterns
Idiomatic Elixir patterns extracted from the [Elixir source code](https://github.com/elixir-lang/elixir) with verified file:line citations.
**Prescriptive.** Follow these when writing Elixir code.
A pattern is a reusable solution to a recurring problem. Each one has:
- **When to use** — the problem it solves
- **When NOT to use** — where it causes harm
- **Why** — the reasoning, not just the rule
- **Source citations** — verified file:line from real codebases
These are derived from what mature Elixir codebases *actually do*, not opinions or blog posts.
## Structure
- `patterns/`Core patterns (GenServer, error handling, data transforms, processes, testing, docs, typespecs, macros, behaviours, modules)
- `smells/`Anti-patterns and common mistakes the Elixir team avoids
- `changelog/` — Daily digest of merged Elixir PRs with discussion summaries
- `patterns/`what to do (behaviours, GenServer, error handling, testing, typespecs, etc.)
- `smells/`what NOT to do (anti-patterns, common mistakes)
- `sources/` — reference material from specific projects (Oban, elixir-lang). Study for ideas, don't copy blindly.
## Philosophy
## How to use
These rules are derived from what the Elixir source code *actually does*, not opinions or blog posts. Every pattern cites specific files and line numbers.
Give your agent these instructions depending on the task:
When unsure how to do something in Elixir, look at how Elixir core does it. This is how we define what "idiomatic" actually means.
### Solving a problem
> You have access to a patterns repo containing proven solutions to recurring Elixir problems. When I describe a problem:
>
> 1. Identify which pattern files are relevant (read them)
> 2. Check if my problem matches a "When to use" case
> 3. Check if it matches a "When NOT to use" case
> 4. If a pattern fits: suggest the approach, cite the pattern, explain why it applies here
> 5. If no pattern fits: say so, and suggest an approach grounded in the principles you see across the patterns
> 6. If my problem matches a smell: warn me before I make the mistake
>
> Never suggest something that contradicts a documented pattern without explicitly calling out the deviation and justifying it.
### Reviewing code
> You have access to a patterns repo that defines how Elixir code should be written. For each file in the diff:
>
> 1. Read the relevant pattern files
> 2. Verify the code follows the documented patterns
> 3. If it deviates: flag it with a reference to the specific pattern, section, and why it matters
> 4. If it matches a smell: flag it as a known anti-pattern
> 5. A deviation without justification is a finding
>
> Don't invent rules. Only flag what the patterns document.
### Evaluating a pattern
> Read the pattern file. Compare against how the following projects handle the same problem: [list projects]. Does the pattern hold? Are there cases where it breaks down? Should it be updated, split, or retired? File your findings as an issue.
## Patterns vs Conventions
**Pattern** = prescriptive. "When you face X, do Y." Language-scoped. Follow these.
**Convention** = descriptive. "Project Z does it this way." Context-specific. Study for ideas — applying another project's conventions to yours without understanding their constraints causes harm.
The `sources/` directory is convention material absorbed from thin repos. The `patterns/` directory is what you actually follow.
+23
View File
@@ -2,6 +2,18 @@
How behaviours are designed, implemented, and used in Elixir core and Phoenix.
## Contents
1. [Behaviour Definition with `@callback`](#1-behaviour-definition-with-callback)
2. [`@optional_callbacks` for Extensibility](#2-optional_callbacks-for-extensibility)
3. [`@behaviour` Declaration in `__using__`](#3-behaviour-declaration-in-__using__)
4. [Default Implementations via `defoverridable`](#4-default-implementations-via-defoverridable)
5. [Phoenix Channel: Behaviour + Process + Protocol](#5-phoenix-channel-behaviour--process--protocol)
6. [Callback Documentation Pattern](#6-callback-documentation-pattern)
7. [Phoenix.Endpoint: Behaviour as Interface Contract](#7-phoenixendpoint-behaviour-as-interface-contract)
---
## 1. Behaviour Definition with `@callback`
**Source:** [lib/elixir/lib/gen_server.ex#L577](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/lib/gen_server.ex#L577) (all callback definitions)
@@ -678,4 +690,15 @@ end
**Why:** The more code a `use` macro generates, the harder it is to debug. If users regularly need to read the generated code to understand failures, the abstraction is leaking. Reserve heavy `use` macros for well-established patterns (GenServer, Endpoint, Channel) where the community has internalized the mental model.
## Decision Tree
- If you need a contract that multiple modules will implement differently → define a behaviour with `@callback` (Pattern 1)
- If most implementors will use a default for some callbacks → mark those `@optional_callbacks` (Pattern 2)
- If your behaviour requires boilerplate setup (module attributes, compile hooks) → inject `@behaviour` inside `__using__` (Pattern 3)
- If 90% of implementors want the same default for a callback → provide a `defoverridable` implementation (Pattern 4)
- If the behaviour involves a running process with lifecycle configuration → combine behaviour + process + module attributes (Pattern 5)
- If callback semantics are non-obvious (multiple return shapes, triggering conditions) → write comprehensive `@doc` with examples on each `@callback` (Pattern 6)
- If the behaviour requires significant generated boilerplate (plugs, routing, supervision wiring) → use the `use` macro as the full interface contract (Pattern 7)
- If there is only one implementation and no plans for more → skip the behaviour, use a plain module
<!-- PATTERN_COMPLETE -->
File diff suppressed because it is too large Load Diff
+28
View File
@@ -2,6 +2,21 @@
Patterns extracted from Elixir's standard library source code.
## Contents
1. [List-Specialized Clause Before Protocol Dispatch](#1-list-specialized-clause-before-protocol-dispatch)
2. [Build-Then-Reverse (Cons-Cell Accumulation)](#2-build-then-reverse-cons-cell-accumulation)
3. [Pipeline for Linear Transformations, Bare Calls for Control Flow](#3-pipeline-for-linear-transformations-bare-calls-for-control-flow)
4. [Pipeline Ending with `|> elem(1)` (Protocol Reduce Unwrap)](#4-pipeline-ending-with--elem1-protocol-reduce-unwrap)
5. [Private Helper Decomposition: Recursive Workers with Guards](#5-private-helper-decomposition-recursive-workers-with-guards)
6. [Enum vs Stream Decision Pattern](#6-enum-vs-stream-decision-pattern)
7. [Map.update vs Map.put Decision Pattern](#7-mapupdate-vs-mapput-decision-pattern)
8. [Pattern Matching on Map Structure for Dispatch](#8-pattern-matching-on-map-structure-for-dispatch)
9. [Delegating to Erlang BIFs with `defdelegate`](#9-delegating-to-erlang-bifs-with-defdelegate)
10. [Reduce as the Universal Primitive](#10-reduce-as-the-universal-primitive)
11. [Keyword Multi-Clause Guard Dispatch (String.split pattern)](#11-keyword-multi-clause-guard-dispatch-stringsplit-pattern)
12. [Lazy Private Helpers with `defp parts_to_index`](#12-lazy-private-helpers-with-defp-parts_to_index)
---
## 1. List-Specialized Clause Before Protocol Dispatch
@@ -1010,4 +1025,17 @@ def log(msg) when is_atom(msg), do: IO.puts(Atom.to_string(msg))
**Why:** When a conversion is used exactly once and the calling function already dispatches on clauses, folding the conversion into the caller's clauses reduces indirection. Named helpers shine when reused or when they name a non-obvious transformation.
## Decision Tree
- If you accept "any enumerable" but lists are the common case → add a `when is_list` clause before protocol dispatch (Pattern 1)
- If you are building a result list element-by-element and order matters → prepend with `[x | acc]` then reverse at the end (Pattern 2)
- If data flows through 2+ sequential transformations → use the pipe operator (Pattern 3)
- If you call `Enumerable.reduce/3` directly and always want the accumulated value → unwrap with `|> elem(1)` (Pattern 4)
- If you need a recursive function with multiple termination conditions → decompose into public entry + private multi-clause worker (Pattern 5)
- If the collection is large/infinite or you chain 3+ transforms → use Stream; otherwise use Enum (Pattern 6)
- If the new value depends on the old value (increment, append) → use `Map.update/4`; if replacing unconditionally → use `Map.put/3` (Pattern 7)
- If you need to branch on whether a key exists and extract the value → pattern-match with `%{^key => value}` in a `case` (Pattern 8)
- If an Erlang function has identical semantics and argument order → use `defdelegate` (Pattern 9)
- If you are implementing a custom iterable data structure → implement `Enumerable.reduce/3` to get the full Enum API (Pattern 10)
<!-- PATTERN_COMPLETE -->
+92 -1
View File
@@ -2,6 +2,20 @@
Patterns extracted from the Elixir standard library source code.
## Contents
1. [@moduledoc with Structured Sections](#1-moduledoc-with-structured-sections)
2. [@doc with Sections and Examples](#2-doc-with-sections-and-examples)
3. [@doc since: Version Annotation](#3-doc-since-version-annotation)
4. [@doc guard: true Metadata](#4-doc-guard-true-metadata)
5. [@doc false — Hiding from Documentation](#5-doc-false--hiding-from-documentation)
6. [@moduledoc false — Hiding Modules](#6-moduledoc-false--hiding-modules)
7. [Mermaid Diagrams in Documentation](#7-mermaid-diagrams-in-documentation)
8. [Admonition Blocks in Documentation](#8-admonition-blocks-in-documentation)
9. [@doc deprecated: Soft Deprecation](#9-doc-deprecated-soft-deprecation)
10. [Callback Documentation Convention](#10-callback-documentation-convention)
11. [Documentation with Link References (c: and t: prefixes)](#11-documentation-with-link-references-c-and-t-prefixes)
---
## 1. @moduledoc with Structured Sections
@@ -201,7 +215,7 @@ def chunk(list, size), do: ...
**Don't use this when:**
- The function is private (use `# comments` for private function notes)
- The function name + typespec are completely self-explanatory (e.g., `@spec pid() :: pid()`)
- You're implementing a behaviour callback and want it hidden (`@impl true` sets `@doc false` automatically)
- You're implementing a behaviour callback `@impl true` sets `@doc false` automatically. Override only when the implementation has semantics the behaviour can't speak to (see [Pattern 10 — implementation-side docs](#when-to-override-doc-false-on-impl-functions))
**Over-application example:**
```elixir
@@ -442,6 +456,7 @@ def __struct__(fields), do: ...
- The function IS part of the public API (even if you think it's "obvious")
- You want to discourage use but still document it (use `@doc deprecated:` instead)
- You're hiding functions because you're too lazy to document them
- The `@impl` function has implementation-specific semantics that callers need to know (see [Pattern 10 — implementation-side docs](#when-to-override-doc-false-on-impl-functions))
**Over-application example:**
```elixir
@@ -467,6 +482,8 @@ end
**Why:** `@doc false` means "this function is not part of the public API." If users are expected to call it, it needs documentation. Hiding public API behind `@doc false` is a maintenance hazard — users will call undocumented functions and break on upgrades.
> **Note:** `@impl true` auto-sets `@doc false`, but some implementations earn their own `@doc`. See [Pattern 10 — implementation-side docs](#when-to-override-doc-false-on-impl-functions) for the rule.
---
## 6. @moduledoc false — Hiding Modules
@@ -927,6 +944,67 @@ Called to format the value.
**Why:** A callback with one parameter and one return type doesn't need a full reference manual. Match documentation depth to complexity — a one-liner with good naming is better than padded sections that add no information.
### When to override `@doc false` on `@impl` functions
`@impl true` sets `@doc false` by default — the assumption is that the behaviour's `@callback` doc covers it. This is correct when the implementation is unremarkable. Override it with an explicit `@doc` when the implementation has semantics that would not be true of every conforming implementation.
**The test:** "Would this statement be true of *any* implementation of this callback?" If yes → the doc belongs on the `@callback` in the behaviour, not here. If no → the implementation earns its own `@doc`.
**Override when:**
- The implementation makes a guarantee the behaviour doesn't promise (e.g., "always returns immediately", "never buffers", "fires at most once")
- There's a race condition, ordering constraint, or subtle failure mode specific to this implementation
- The implementation's behavior under edge cases differs from what the behaviour's generic contract implies
**Don't override when:**
- The doc would just restate the `@callback` doc in different words
- The doc describes what the function does rather than what's surprising about this implementation
- The function name + behaviour doc are sufficient for an implementor or caller to understand it
**Example — unnecessary override (just restates the contract):**
```elixir
defmodule JsonSerializer do
@behaviour Serializer
@doc """
Encodes the given term to a binary.
"""
@impl true
def encode(term), do: Jason.encode!(term)
end
```
**Example — justified override (implementation-specific guarantee):**
```elixir
defmodule Immediate do
@behaviour Aggregation
@doc """
Always returns `{:ready, [signal]}` — immediate mode fires on first signal
without buffering or waiting for a timer.
"""
@impl true
def check(%Signal{} = signal), do: {:ready, [signal]}
end
```
**Example — justified override (race condition warning):**
```elixir
defmodule AlpacaAdapter do
@behaviour BrokerAdapter
@doc """
Cancels an open order by broker ID. Returns `:ok` on success.
The order may still receive a final fill between the cancel request
and confirmation — callers must handle the `partially_filled` → `cancelled` race.
"""
@impl true
def cancel(credential, broker_order_id), do: ...
end
```
**Why:** The behaviour's `@callback` doc is the generic contract — "what any implementation must do." An implementation's `@doc` is the specific contract — "what callers of *this* implementation can rely on." When those differ meaningfully, silence (`@doc false`) hides information that would prevent bugs.
---
## 11. Documentation with Link References (c: and t: prefixes)
@@ -1000,4 +1078,17 @@ Returns `true` if the calling process is the owner of this resource.
**Why:** Link references should aid navigation, not turn documentation into hypertext soup. Link types and callbacks that users might need to look up; don't link primitive types or universally known functions.
## Decision Tree
- If the module is a primary entry point with 4+ public functions → use structured `@moduledoc` with sections (Pattern 1)
- If a function has non-obvious behavior or edge cases → add `@doc` with sections and `## Examples` doctests (Pattern 2)
- If adding a new public function to a versioned library → annotate with `@doc since: "X.Y.Z"` (Pattern 3)
- If the function/macro is valid in guard clauses → add `@doc guard: true` metadata (Pattern 4)
- If a function must be public for technical reasons but is not user-facing → use `@doc false` (Pattern 5)
- If an entire module is purely internal implementation → use `@moduledoc false` (Pattern 6)
- If documenting multi-component architecture (client-server, pipelines) → embed a Mermaid diagram (Pattern 7)
- If critical information must stand out (security, breaking changes, `use` behavior) → use an admonition block (Pattern 8)
- If a function still works but a better alternative exists → use `@doc deprecated:` for soft deprecation (Pattern 9)
- If defining a behaviour callback with multiple return shapes → write comprehensive callback docs with trigger, params, returns, and example (Pattern 10)
<!-- PATTERN_COMPLETE -->
+32
View File
@@ -2,6 +2,23 @@
Patterns extracted from Elixir's standard library source code.
## Contents
1. [The `with` Macro — Normalized Error Clauses](#1-the-with-macro--normalized-error-clauses)
2. [Real-World `with` — Multi-Step Fallible Operations](#2-real-world-with--multi-step-fallible-operations)
3. [Another `with` — Error Info Extraction](#3-another-with--error-info-extraction)
4. [`{:ok, value}` / `:error` Convention (Map.fetch)](#4-ok-value--error-convention-mapfetch)
5. [Bang Functions: Raise on Error (`fetch!` vs `fetch`)](#5-bang-functions-raise-on-error-fetch-vs-fetch)
6. [Exception Structure: `defexception` Fields](#6-exception-structure-defexception-fields)
7. [Custom `exception/1` Callback for Ergonomic Raising](#7-custom-exception1-callback-for-ergonomic-raising)
8. [`raise` Macro Internals: Compile-Time Type Resolution](#8-raise-macro-internals-compile-time-type-resolution)
9. [Error Normalization: Erlang → Elixir Exception Translation](#9-error-normalization-erlang--elixir-exception-translation)
10. [`blame/2` Callback: Enriching Exceptions After the Fact](#10-blame2-callback-enriching-exceptions-after-the-fact)
11. [Guards for Type Dispatch in Error Handling](#11-guards-for-type-dispatch-in-error-handling)
12. [The `:error` / `{:error, reason}` Convention Split](#12-the-error--error-reason-convention-split)
13. [`reduce_while` — Early Exit Without Exceptions](#13-reduce_while--early-exit-without-exceptions)
14. [Three-Tier Error Strategy in Map Operations](#14-three-tier-error-strategy-in-map-operations)
---
## 1. The `with` Macro — Normalized Error Clauses
@@ -1400,4 +1417,19 @@ end
**Why:** The three-tier pattern only makes sense when failure is a real possibility and different callers genuinely need different responses to that failure. Don't cargo-cult it onto functions that always succeed or have a single calling context.
## Decision Tree
- If you have 2+ sequential steps that each return a value to pattern-match → use `with` with normalized error shapes (Pattern 1)
- If the caller only cares success vs failure (not which step failed) → use `with` + `else _ -> :error` catch-all (Pattern 2)
- If extracting nested data from loosely-structured inputs (stacktraces, metadata) → chain pattern matching in `with` (Pattern 3)
- If a function has exactly one failure mode obvious from context → return bare `:error` (Pattern 4)
- If failure means a bug (preconditions guarantee success) → provide a bang variant that raises (Pattern 5)
- If callers need to programmatically inspect error context → use `defexception` with structured fields (Pattern 6)
- If the exception message is computed from multiple fields or requires validation → override `exception/1` (Pattern 7)
- If wrapping an Erlang library that returns raw error atoms/tuples → normalize to Elixir exceptions at the boundary (Pattern 9)
- If you can provide expensive but helpful context (did-you-mean suggestions) → implement `blame/2` (Pattern 10)
- If multiple distinct failure modes exist → use `{:error, reason}` tuples; if only one → use bare `:error` (Pattern 12)
- If you need early exit from iteration without exceptions → use `reduce_while` with `{:cont, acc}` / `{:halt, acc}` (Pattern 13)
- If designing a module with lookup operations for different caller needs → provide three tiers: get/fetch/fetch! (Pattern 14)
<!-- PATTERN_COMPLETE -->
+96 -1
View File
@@ -2,6 +2,21 @@
Analysis of `lib/elixir/lib/gen_server.ex`, `lib/elixir/lib/agent.ex`, and related modules.
## Contents
1. [Pattern 1: Client/Server API Separation](#pattern-1-clientserver-api-separation)
2. [Pattern 2: `@impl true` Annotations on All Callbacks](#pattern-2-impl-true-annotations-on-all-callbacks)
3. [Pattern 3: Guard-Protected `start_link`](#pattern-3-guard-protected-start_link)
4. [Pattern 4: `handle_continue` for Post-Init Work](#pattern-4-handle_continue-for-post-init-work)
5. [Pattern 5: Timeout-Based Idle Shutdown](#pattern-5-timeout-based-idle-shutdown)
6. [Pattern 6: Periodic Work via `Process.send_after`](#pattern-6-periodic-work-via-processsend_after)
7. [Pattern 7: Call vs Cast Decision (Synchronous vs Asynchronous)](#pattern-7-call-vs-cast-decision-synchronous-vs-asynchronous)
8. [Pattern 8: Default Callback Implementations with Clear Error Messages](#pattern-8-default-callback-implementations-with-clear-error-messages)
9. [Pattern 9: `child_spec/1` Generation and Customization via `use` Options](#pattern-9-child_spec1-generation-and-customization-via-use-options)
10. [Pattern 10: Agent as Minimal State Wrapper (GenServer Under the Hood)](#pattern-10-agent-as-minimal-state-wrapper-genserver-under-the-hood)
11. [Pattern 11: Name Registration via `:via` Tuple](#pattern-11-name-registration-via-via-tuple)
12. [Pattern 12: GenServer as Anti-Pattern — Don't Use Processes for Code Organization](#pattern-12-genserver-as-anti-pattern--dont-use-processes-for-code-organization)
---
## Pattern 1: Client/Server API Separation
@@ -189,6 +204,72 @@ end
**Why:** `@impl true` only makes sense in the context of a declared behaviour. Using it without one causes a compiler warning, not a benefit.
### Minimal Callback Annotation
**Source:** [lib/elixir/lib/module.ex#L72](https://github.com/elixir-lang/elixir/blob/f4e1b34617ef92052b65781f18eae5b88a490098/lib/elixir/lib/module.ex#L72) (`@impl` documentation)
**What it does:** Once `@impl true` is on a callback, two things become redundant:
1. **Repeating `@impl true` on subsequent clauses** — the annotation applies to the function as a whole, not individual clauses. All clauses inherit it from the first.
2. **Adding `@spec`** — the behaviour's `@callback` already defines the type contract. Dialyzer uses the callback spec to check implementations, so a redundant `@spec` creates a second source of truth that can drift.
**Why:** The behaviour owns the contract. Adding `@spec init(term()) :: {:ok, map()}` on a GenServer callback just restates `@callback init(init_arg :: term) :: {:ok, state} | ...` with less information. Repeating `@impl true` on every clause is noise that misleads readers into thinking each clause is a separate function. The minimal annotation communicates "this is a callback, the behaviour defines the contract."
**Anti-pattern:**
```elixir
# BAD — redundant @spec and @impl on every clause
@spec handle_call(term(), GenServer.from(), map()) :: {:reply, term(), map()}
@impl true
def handle_call({:get, key}, _from, state) do
{:reply, Map.get(state, key), state}
end
@impl true
def handle_call({:keys}, _from, state) do
{:reply, Map.keys(state), state}
end
@impl true
def handle_call({:size}, _from, state) do
{:reply, map_size(state), state}
end
```
**Example — after:**
```elixir
@impl true
def handle_call({:get, key}, _from, state) do
{:reply, Map.get(state, key), state}
end
def handle_call({:keys}, _from, state) do
{:reply, Map.keys(state), state}
end
def handle_call({:size}, _from, state) do
{:reply, map_size(state), state}
end
```
**When NOT to apply this:**
- **Non-consecutive clauses:** If clauses of the same callback are separated by other functions, the compiler may not associate them. Keep multi-clause callbacks grouped together.
- **Intentionally narrower types:** When the implementation accepts a narrower type than the callback declares, a more specific `@spec` documents that constraint:
```elixir
# Justified — documents that this init/1 only accepts keyword lists,
# not arbitrary term() as the callback permits
@spec init(keyword()) :: {:ok, Config.t()}
@impl true
def init(opts) when is_list(opts) do
{:ok, Config.new!(opts)}
end
```
If the `@spec` would be identical to (or wider than) the `@callback`, omit it. If it's meaningfully narrower, keep it.
---
## Pattern 3: Guard-Protected `start_link`
@@ -208,7 +289,7 @@ def start_link(default) when is_binary(default) do
GenServer.start_link(__MODULE__, default)
end
# From agent.ex:246
# From agent.ex:279
@spec start_link((-> term), GenServer.options()) :: on_start
def start_link(fun, options \\ []) when is_function(fun, 0) do
GenServer.start_link(Agent.Server, fun, options)
@@ -1109,4 +1190,18 @@ end
**Why:** The pattern cuts both ways. Over-using GenServer creates bottlenecks. Under-using it means reinventing state management poorly. The litmus test: does the state need to survive between function calls? Does access need serialization? If yes, you need a process.
## Decision Tree
- If other modules will interact with your GenServer → define a client API wrapping call/cast (Pattern 1)
- If implementing any behaviour callback → annotate with `@impl true` (Pattern 2)
- If `start_link` accepts arguments with a specific expected shape → add guards for fail-fast validation (Pattern 3)
- If `init/1` does expensive work (DB, network, cache warming) → split into fast init + `handle_continue` (Pattern 4)
- If the process is ephemeral (per-user, per-session) and should clean up when idle → use timeout-based idle shutdown (Pattern 5)
- If you need work at regular intervals regardless of message traffic → use `Process.send_after` self-scheduling loop (Pattern 6)
- If the caller needs confirmation or backpressure → use `call`; only use `cast` for genuine fire-and-forget (Pattern 7)
- If the process needs non-default restart/shutdown behavior → customize via `use GenServer` options (Pattern 9)
- If the process is purely about state (no custom messages, no timers) → use Agent instead of GenServer (Pattern 10)
- If spawning processes dynamically with unbounded names → use `{:via, Registry, ...}` to avoid atom leaks (Pattern 11)
- If the operation is stateless pure computation → don't use a GenServer at all, use a plain function (Pattern 12)
<!-- PATTERN_COMPLETE -->
+29
View File
@@ -2,6 +2,21 @@
Patterns extracted from the Elixir standard library source code.
## Contents
1. [Context-Aware Macros (__CALLER__.context)](#1-context-aware-macros-__caller__context)
2. [defguard — Macro for Guard-Safe Expressions](#2-defguard--macro-for-guard-safe-expressions)
3. [quote + unquote for Code Generation](#3-quote--unquote-for-code-generation)
4. [var! for Breaking Hygiene](#4-var-for-breaking-hygiene)
5. [Macro Expanding with Macro.expand](#5-macro-expanding-with-macroexpand)
6. [assert_no_match_or_guard_scope Pattern](#6-assert_no_match_or_guard_scope-pattern)
7. [Protocol Definition as a Macro (defprotocol)](#7-protocol-definition-as-a-macro-defprotocol)
8. [@fallback_to_any in Protocols](#8-fallback_to_any-in-protocols)
9. [use/2 as Macro Injection Point](#9-use2-as-macro-injection-point)
10. [Sigil Macros (Pattern for DSL Literals)](#10-sigil-macros-pattern-for-dsl-literals)
11. [Pipe Operator as a Macro](#11-pipe-operator-as-a-macro)
12. [Macro.generate_unique_arguments for Hygiene](#12-macrogenerate_unique_arguments-for-hygiene)
---
## 1. Context-Aware Macros (__CALLER__.context)
@@ -1105,4 +1120,18 @@ end
**Why:** Variables created in `quote` are already hygienic by default — they can't clash with caller variables. `generate_unique_arguments` is needed when you're generating *multiple* variables dynamically (e.g., function parameters for a generated clause) where you need distinct names that also interoperate correctly.
## Decision Tree
- If a macro must behave differently in guards vs normal code → check `__CALLER__.context` (Pattern 1)
- If you need a reusable, compile-time-validated guard expression → use `defguard` (Pattern 2)
- If a macro argument might have side effects or be expensive → use `quote bind_quoted:` to evaluate once (Pattern 3)
- If a macro must reference a variable in the caller's scope → use `var!` sparingly (Pattern 4)
- If the macro receives input that could be an alias or module attribute → expand with `Macro.expand` before branching (Pattern 5)
- If your macro defines module-level constructs and should never appear in guards → assert context at the top (Pattern 6)
- If you need open-ended type dispatch that external code can extend → use `defprotocol` (Pattern 7)
- If a protocol should handle any value rather than raising on unknown types → use `@fallback_to_any true` (Pattern 8)
- If a module needs injected behaviours, attributes, or compile hooks → use the `use/2` + `__using__/1` pattern (Pattern 9)
- If you have compile-time-known literals that benefit from validation → define a sigil macro (Pattern 10)
- If you need a zero-cost syntactic transformation (argument rewriting) → implement as a macro like `|>` (Pattern 11)
<!-- PATTERN_COMPLETE -->
+22
View File
@@ -2,6 +2,18 @@
How modules are structured, named, and organized in Elixir core and Phoenix.
## Contents
1. [One Module per Concept, Nested for Sub-Concepts](#1-one-module-per-concept-nested-for-sub-concepts)
2. [Public API at the Top, Private Functions at the Bottom](#2-public-api-at-the-top-private-functions-at-the-bottom)
3. [`@moduledoc false` for Internal Modules](#3-moduledoc-false-for-internal-modules)
4. [Struct Definition Conventions](#4-struct-definition-conventions)
5. [Selective Imports in `__using__`](#5-selective-imports-in-__using__)
6. [Alias at Module Scope for Readability](#6-alias-at-module-scope-for-readability)
7. [Boolean-Suffixed Fields in Structs](#7-boolean-suffixed-fields-in-structs)
---
## 1. One Module per Concept, Nested for Sub-Concepts
**Source:** `lib/elixir/lib/` directory structure
@@ -590,4 +602,14 @@ defstruct [:user, :admin?, :count]
**Why:** The `?` suffix should only mark genuine booleans. Using it on non-boolean fields creates confusion about the field's type and breaks the convention's usefulness as a type signal.
## Decision Tree
- If your module has grown beyond 300 lines with distinct sub-responsibilities → [One Module per Concept, Nested for Sub-Concepts](#1-one-module-per-concept-nested-for-sub-concepts)
- If you need to decide function ordering within a module → [Public API at the Top, Private Functions at the Bottom](#2-public-api-at-the-top-private-functions-at-the-bottom)
- If a module exists purely for internal code organization and should not appear in docs → [`@moduledoc false` for Internal Modules](#3-moduledoc-false-for-internal-modules)
- If you need to define a struct and decide which fields are mandatory → [Struct Definition Conventions](#4-struct-definition-conventions)
- If your `use` macro needs to set up the caller's namespace with specific functions → [Selective Imports in `__using__`](#5-selective-imports-in-__using__)
- If multiple modules from the same parent namespace are used repeatedly → [Alias at Module Scope for Readability](#6-alias-at-module-scope-for-readability)
- If a struct field stores a boolean value and you want self-documenting naming → [Boolean-Suffixed Fields in Structs](#7-boolean-suffixed-fields-in-structs)
<!-- PATTERN_COMPLETE -->
+712
View File
@@ -0,0 +1,712 @@
# Ecto.Multi Patterns
Patterns extracted from Ecto's `Ecto.Multi` source code.
## Contents
1. [`Multi.new() |> Multi.insert/update/delete` — Named Operation Pipeline](#1-multinew--multiinsertupdatedelete--named-operation-pipeline)
2. [`Multi.run/3` — Arbitrary Code in a Transaction](#2-multirun3--arbitrary-code-in-a-transaction)
3. [Dependent Operations with Function Variants](#3-dependent-operations-with-function-variants)
4. [`Multi.merge/2` — Dynamic Transaction Composition](#4-multimerge2--dynamic-transaction-composition)
5. [`Multi.append/2` / `Multi.prepend/2` — Static Multi Composition](#5-multiappend2--multiprepend2--static-multi-composition)
6. [Tuple Keys — Dynamic Collections of Operations](#6-tuple-keys--dynamic-collections-of-operations)
7. [`Multi.to_list/1` — Testing Without a Database](#7-multito_list1--testing-without-a-database)
---
## 1. `Multi.new() |> Multi.insert/update/delete` — Named Operation Pipeline
**Source:** [lib/ecto/multi.ex#L58](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/multi.ex#L58)
```elixir
def reset(account, params) do
Multi.new()
|> Multi.update(:account, Account.password_reset_changeset(account, params))
|> Multi.insert(:log, Log.password_reset_changeset(account, params))
|> Multi.delete_all(:sessions, Ecto.assoc(account, :sessions))
end
# Execute:
case Repo.transaction(PasswordManager.reset(account, params)) do
{:ok, %{account: account, log: log}} -> # success
{:error, :account, changeset, _} -> # account step failed
end
```
**Why:** Each operation is named. On success, `Repo.transaction` returns `{:ok, results_map}` where each key is the name given to that operation. On failure, it returns `{:error, failed_name, failed_value, changes_so_far}`, making it immediately clear which step aborted the transaction and why. This is more precise than a bare transaction function where you'd have to inspect the return value to guess which step failed.
**Anti-pattern:** Using an anonymous function with bare `case` statements inside a transaction, where failure attribution is implicit:
```elixir
# BAD — no way to know which operation failed from the return value alone
Repo.transaction(fn ->
case Repo.update(Account.password_reset_changeset(account, params)) do
{:ok, account} ->
case Repo.insert(Log.password_reset_changeset(account, params)) do
{:ok, log} -> {:ok, %{account: account, log: log}}
{:error, changeset} -> Repo.rollback(changeset)
end
{:error, changeset} -> Repo.rollback(changeset)
end
end)
```
### When to Use
**Triggers:**
- You have 2+ database operations that must all succeed or all roll back
- The set of operations is known at compile time (not dynamically generated)
- The caller needs to know which specific operation failed
**Example — before:**
```elixir
def create_user_with_profile(params) do
Repo.transaction(fn ->
case Repo.insert(User.changeset(params)) do
{:ok, user} ->
case Repo.insert(Profile.changeset(user, params)) do
{:ok, profile} -> {:ok, {user, profile}}
{:error, cs} -> Repo.rollback(cs)
end
{:error, cs} -> Repo.rollback(cs)
end
end)
end
```
**Example — after:**
```elixir
def create_user_with_profile(params) do
Multi.new()
|> Multi.insert(:user, User.changeset(params))
|> Multi.insert(:profile, fn %{user: user} ->
Profile.changeset(user, params)
end)
|> Repo.transaction()
end
# Caller:
case create_user_with_profile(params) do
{:ok, %{user: user, profile: profile}} -> {:ok, user}
{:error, :user, changeset, _} -> {:error, changeset}
{:error, :profile, changeset, _} -> {:error, changeset}
end
```
### When NOT to Use
**Don't use this when:**
- You have a single database operation (just call `Repo.insert/update/delete` directly)
- Operations are simple and sequential with no branching (a plain `Repo.transaction(fn -> ... end)` is more readable)
- The overhead of building a Multi struct is not justified by the number of operations
**Over-application example:**
```elixir
# Overkill for a single operation
Multi.new()
|> Multi.insert(:user, User.changeset(params))
|> Repo.transaction()
```
**Better alternative:**
```elixir
Repo.insert(User.changeset(params))
```
**Why:** `Ecto.Multi` introduces indirection. For simple cases, calling Repo functions directly or using `Repo.transaction(fn -> ... end)` is clearer. The named-pipeline form pays off when 3+ operations are involved and failure attribution matters.
---
## 2. `Multi.run/3` — Arbitrary Code in a Transaction
**Source:** [lib/ecto/multi.ex#L39](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/multi.ex#L39)
```elixir
Multi.new()
|> Multi.insert(:user, user_changeset)
|> Multi.run(:welcome_email, fn repo, %{user: user} ->
case Mailer.send_welcome(user) do
:ok -> {:ok, :sent}
{:error, reason} -> {:error, reason}
end
end)
```
**Why:** `Multi.run` is the escape hatch for operations that don't fit the standard insert/update/delete API. The callback receives `(repo, changes_so_far)` — the repo argument means you can issue raw queries using the same transaction connection. Returning `{:error, value}` from the function aborts the whole transaction and surfaces the value in the `{:error, :welcome_email, value, _}` result. Returning `{:ok, value}` stores the value under `:welcome_email` in the success map.
**Anti-pattern:** Embedding operations in `Multi.run` that don't need transaction context or that don't return the required `{:ok, value}` / `{:error, value}` shape:
```elixir
# BAD — run used just to transform data, no transaction context needed
Multi.new()
|> Multi.insert(:user, user_changeset)
|> Multi.run(:formatted_name, fn _repo, %{user: user} ->
# This has no side effects and doesn't need a transaction
{:ok, String.upcase(user.name)}
end)
```
### When to Use
**Triggers:**
- You need to perform an operation that isn't a standard Ecto schema action (e.g., calling an external service, running a raw query, computing a value that might fail)
- The operation must participate in the transaction rollback on failure
- The result of the operation is needed by a subsequent step in the Multi
**Example — before:**
```elixir
def register_user(params) do
Repo.transaction(fn ->
{:ok, user} = Repo.insert(User.changeset(params))
# This runs outside Multi — if it fails, user was already inserted
case ExternalService.provision(user.id) do
{:ok, token} -> {:ok, %{user: user, token: token}}
{:error, reason} -> Repo.rollback(reason)
end
end)
end
```
**Example — after:**
```elixir
def register_user(params) do
Multi.new()
|> Multi.insert(:user, User.changeset(params))
|> Multi.run(:provision, fn _repo, %{user: user} ->
ExternalService.provision(user.id)
end)
|> Repo.transaction()
end
```
### When NOT to Use
**Don't use this when:**
- The operation is a standard Ecto schema action — use `Multi.insert`, `Multi.update`, `Multi.delete`, or `Multi.delete_all` instead
- The callback only transforms data without any side effects or failure modes
- You need to insert/update using a changeset that depends on prior results — use the function variant of `Multi.insert/update` (see Pattern 3) instead
**Over-application example:**
```elixir
# Multi.run adds overhead when Multi.insert's function variant covers this
Multi.new()
|> Multi.insert(:post, post_changeset)
|> Multi.run(:comment, fn _repo, %{post: post} ->
Repo.insert(Ecto.build_assoc(post, :comments, body: "first"))
end)
```
**Better alternative:**
```elixir
Multi.new()
|> Multi.insert(:post, post_changeset)
|> Multi.insert(:comment, fn %{post: post} ->
Ecto.build_assoc(post, :comments, body: "first")
end)
```
**Why:** `Multi.run` requires you to return a tagged `{:ok, value}` / `{:error, value}` tuple. When the operation is a plain Ecto changeset, the function variants of `Multi.insert/update/delete` accept a function that returns a changeset and handle the wrapping internally. Reserve `Multi.run` for operations with their own failure semantics.
---
## 3. Dependent Operations with Function Variants
**Source:** [lib/ecto/multi.ex#L298](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/multi.ex#L298)
```elixir
Multi.new()
|> Multi.insert(:post, %Post{title: "first"})
|> Multi.insert(:comment, fn %{post: post} ->
Ecto.build_assoc(post, :comments, body: "first comment")
end)
```
**Why:** Each of the standard Multi operations (`insert`, `update`, `delete`, `delete_all`, `insert_or_update`) accepts either a static changeset/struct or a one-argument function `fn changes -> changeset end`. When an operation depends on the result of a previous step, pass the function form. The function receives the accumulated changes map up to that point, giving you access to all prior results. This keeps operations in a single pipeline without introducing a separate `Multi.run` call.
**Anti-pattern:** Using `Multi.run` when a function variant of `Multi.insert/update` suffices:
```elixir
# BAD — Multi.run is more verbose and requires explicit {:ok, value} wrapping
Multi.new()
|> Multi.insert(:post, %Post{title: "first"})
|> Multi.run(:comment, fn _repo, %{post: post} ->
Repo.insert(Ecto.build_assoc(post, :comments, body: "first comment"))
end)
```
### When to Use
**Triggers:**
- An operation's changeset or struct must be constructed using the result of a prior operation
- You need an association ID, a foreign key, or any field that isn't available until a prior step runs
- You want to keep the pipeline as `Multi.insert/update/delete` calls without dropping into `Multi.run`
**Example — before:**
```elixir
def create_order_with_items(cart, user) do
Multi.new()
|> Multi.insert(:order, Order.changeset(cart, user))
|> Multi.run(:items, fn _repo, %{order: order} ->
items = Enum.map(cart.items, &Repo.insert(OrderItem.changeset(&1, order)))
errors = Enum.filter(items, &match?({:error, _}, &1))
if errors == [], do: {:ok, items}, else: {:error, errors}
end)
end
```
**Example — after:**
```elixir
def create_order_with_items(cart, user) do
cart.items
|> Enum.with_index()
|> Enum.reduce(Multi.new() |> Multi.insert(:order, Order.changeset(cart, user)),
fn {item, idx}, multi ->
Multi.insert(multi, {:item, idx}, fn %{order: order} ->
OrderItem.changeset(item, order)
end)
end)
end
```
### When NOT to Use
**Don't use this when:**
- The changeset does not depend on any prior operation — pass it directly as a static value
- The dependent operation has complex failure logic that requires returning `{:ok, value}` / `{:error, value}` — use `Multi.run` in that case
- The function would need to perform multiple Repo calls — the function variant only accepts a changeset/struct return, not arbitrary DB operations
**Over-application example:**
```elixir
# Function variant used when the changeset doesn't depend on prior results
Multi.new()
|> Multi.insert(:user, fn _changes ->
User.changeset(%User{}, params) # No dependency on changes — pass directly
end)
```
**Better alternative:**
```elixir
Multi.new()
|> Multi.insert(:user, User.changeset(%User{}, params))
```
**Why:** The function variant defers changeset construction to transaction execution time, which means it can't be inspected with `Multi.to_list/1` until the Multi runs. For static changesets, passing the value directly keeps the Multi inspectable and testable without executing the transaction.
---
## 4. `Multi.merge/2` — Dynamic Transaction Composition
**Source:** [lib/ecto/multi.ex#L239](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/multi.ex#L239)
```elixir
Multi.new()
|> Multi.insert(:post, post_changeset)
|> Multi.merge(fn %{post: post} ->
if post.requires_approval do
Multi.new()
|> Multi.insert(:approval_request, ApprovalRequest.changeset(post))
else
Multi.new()
end
end)
```
**Why:** `Multi.merge/2` accepts a function that receives the changes accumulated so far and must return another `Ecto.Multi`. The returned Multi's operations are appended to the pipeline at execution time. This is the correct tool when the *set of operations to add* — not just the changesets — depends on prior results. A function variant of `Multi.insert` can swap in a different changeset; `merge` can add or remove entire operations.
**Anti-pattern:** Using `merge` when `append` or a function variant of `insert/update` would suffice — `merge` defers the entire sub-pipeline to runtime, making it harder to inspect:
```elixir
# BAD — merge used just to pass a different changeset, not to change which operations run
Multi.new()
|> Multi.insert(:post, post_changeset)
|> Multi.merge(fn %{post: post} ->
Multi.new()
|> Multi.update(:post_meta, PostMeta.changeset(post))
end)
```
### When to Use
**Triggers:**
- Whether certain operations should be included at all depends on runtime data
- A prior step's result determines which of several alternative sub-pipelines to execute
- You have a reusable function that takes results and returns a configured `Ecto.Multi`
**Example — before:**
```elixir
def publish_article(article, user) do
multi = Multi.new()
|> Multi.update(:article, Article.publish_changeset(article))
# Tacking on conditonal operations awkwardly outside the pipeline
multi =
if article.notify_subscribers do
Multi.run(multi, :notifications, fn _repo, %{article: article} ->
send_notifications(article, user)
end)
else
multi
end
multi
end
```
**Example — after:**
```elixir
def publish_article(article, user) do
Multi.new()
|> Multi.update(:article, Article.publish_changeset(article))
|> Multi.merge(fn %{article: published} ->
if published.notify_subscribers do
Multi.new()
|> Multi.run(:notifications, fn _repo, _changes ->
send_notifications(published, user)
end)
else
Multi.new()
end
end)
end
```
### When NOT to Use
**Don't use this when:**
- The set of operations is fixed — use `Multi.append/2` to combine two pre-built Multis instead
- Only the changeset values vary between operations — use the function variant of `Multi.insert/update`
- The merge function is trivially always the same Multi (just use `Multi.append`)
**Over-application example:**
```elixir
# merge used when append would do — the sub-multi never varies
Multi.new()
|> Multi.insert(:user, user_changeset)
|> Multi.merge(fn _changes ->
Multi.new() |> Multi.insert(:audit_log, AuditLog.create())
end)
```
**Better alternative:**
```elixir
audit_multi = Multi.new() |> Multi.insert(:audit_log, AuditLog.create())
Multi.new()
|> Multi.insert(:user, user_changeset)
|> Multi.append(audit_multi)
```
**Why:** `Multi.merge` with a constant-returning function is just `Multi.append` with extra indirection. Use `merge` only when the function body actually branches on the `changes` argument.
---
## 5. `Multi.append/2` / `Multi.prepend/2` — Static Multi Composition
**Source:** [lib/ecto/multi.ex#L183](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/multi.ex#L183)
```elixir
def audit_multi(entity, user) do
Multi.new()
|> Multi.insert(:audit_log, AuditLog.changeset(entity, user))
end
def create_post(params, user) do
post_multi = Multi.new() |> Multi.insert(:post, Post.changeset(params))
Multi.append(post_multi, audit_multi(:post, user))
end
```
**Why:** `append` and `prepend` combine two fully-built `Ecto.Multi` structs into one. All operation names across both Multis must be unique — a conflict raises at composition time, not at execution time. This makes transaction fragments reusable: `audit_multi/2` can be appended to any business operation without duplicating the audit logic. `prepend` puts the second Multi's operations first; `append` adds them at the end.
**Anti-pattern:** Re-defining the same operations in every Multi instead of extracting reusable fragments:
```elixir
# BAD — audit logic duplicated across every operation
def create_post(params, user) do
Multi.new()
|> Multi.insert(:post, Post.changeset(params))
|> Multi.insert(:audit_log, AuditLog.changeset(:post, user))
end
def update_post(post, params, user) do
Multi.new()
|> Multi.update(:post, Post.changeset(post, params))
|> Multi.insert(:audit_log, AuditLog.changeset(:post, user)) # copy-paste
end
```
### When to Use
**Triggers:**
- You have a recurring group of operations (auditing, notifications, cleanup) that should attach to multiple different transactions
- Two independently-defined Multis must run in the same transaction
- You want to compose transaction fragments from different modules without either module knowing about the other's internals
**Example — before:**
```elixir
def transfer_funds(from, to, amount) do
Multi.new()
|> Multi.update(:debit, Account.debit_changeset(from, amount))
|> Multi.update(:credit, Account.credit_changeset(to, amount))
|> Multi.insert(:ledger_entry, LedgerEntry.changeset(from, to, amount))
|> Multi.insert(:audit_log, AuditLog.changeset(:transfer, %{from: from, to: to}))
end
```
**Example — after:**
```elixir
def audit_multi(action, context) do
Multi.new()
|> Multi.insert(:audit_log, AuditLog.changeset(action, context))
end
def transfer_funds(from, to, amount) do
transfer =
Multi.new()
|> Multi.update(:debit, Account.debit_changeset(from, amount))
|> Multi.update(:credit, Account.credit_changeset(to, amount))
|> Multi.insert(:ledger_entry, LedgerEntry.changeset(from, to, amount))
Multi.append(transfer, audit_multi(:transfer, %{from: from, to: to}))
end
```
### When NOT to Use
**Don't use this when:**
- Which operations to add depends on the results of earlier operations — use `Multi.merge/2` instead
- The two Multis share operation names — this will raise at composition time and requires renaming
- The combination is only used once — just build a single Multi inline
**Over-application example:**
```elixir
# Splitting a single logical operation into two Multis for no benefit
user_part = Multi.new() |> Multi.insert(:user, user_cs)
profile_part = Multi.new() |> Multi.insert(:profile, profile_cs)
Multi.append(user_part, profile_part)
```
**Better alternative:**
```elixir
Multi.new()
|> Multi.insert(:user, user_cs)
|> Multi.insert(:profile, profile_cs)
```
**Why:** `append` and `prepend` are tools for *reuse*. When operations are only ever combined in one place, defining them as a single pipeline is simpler. Extract fragments only when the same group of operations genuinely recurs across different transactions.
---
## 6. Tuple Keys — Dynamic Collections of Operations
**Source:** [lib/ecto/multi.ex#L109](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/multi.ex#L109)
```elixir
Enum.reduce(accounts, Multi.new(), fn account, multi ->
Multi.update(
multi,
{:account, account.id},
Account.password_reset_changeset(account, params)
)
end)
# Error pattern-matching:
case Repo.transaction(multi) do
{:ok, results} -> Map.keys(results) # [{:account, 1}, {:account, 2}, ...]
{:error, {:account, id}, changeset, _} -> "account #{id} failed"
end
```
**Why:** Multi operation names can be any term, not just atoms. Tuple keys like `{:account, account.id}` give each operation in a dynamically-generated collection a unique, structured name. On failure, the name in `{:error, name, value, _}` tells you exactly which item failed — without this, all accounts would compete for the same atom name, which would raise a duplicate name error.
**Anti-pattern:** Using a bare atom for all iterations, which raises at composition time:
```elixir
# BAD — all operations have the same name :account — raises DuplicateNameError
Enum.reduce(accounts, Multi.new(), fn account, multi ->
Multi.update(multi, :account, Account.password_reset_changeset(account, params))
end)
```
### When to Use
**Triggers:**
- You're building a Multi by reducing over a collection and need each item to be a separate named operation
- You need to identify which specific item in a collection caused the transaction to fail
- The result map keys must be inspectable to determine which items succeeded
**Example — before:**
```elixir
# Forced to use Multi.run and handle errors manually
def reset_passwords(accounts, params) do
Multi.new()
|> Multi.run(:all_accounts, fn _repo, _changes ->
results = Enum.map(accounts, fn account ->
Repo.update(Account.password_reset_changeset(account, params))
end)
errors = Enum.filter(results, &match?({:error, _}, &1))
if errors == [], do: {:ok, results}, else: {:error, hd(errors)}
end)
end
```
**Example — after:**
```elixir
def reset_passwords(accounts, params) do
Enum.reduce(accounts, Multi.new(), fn account, multi ->
Multi.update(
multi,
{:account, account.id},
Account.password_reset_changeset(account, params)
)
end)
end
# Caller knows exactly which account failed:
case Repo.transaction(reset_passwords(accounts, params)) do
{:ok, _} -> :ok
{:error, {:account, id}, changeset, _} ->
Logger.error("Failed to reset account #{id}: #{inspect(changeset.errors)}")
end
```
### When NOT to Use
**Don't use this when:**
- The collection has a fixed, known size — just name each operation with a distinct atom instead
- You don't need per-item failure attribution — a `Multi.run` that processes all items may be simpler
- All items should be processed regardless of individual failures — use a `Multi.run` that collects partial results
**Over-application example:**
```elixir
# Tuple keys on a fixed two-item "collection"
[:primary, :secondary]
|> Enum.reduce(Multi.new(), fn role, multi ->
Multi.insert(multi, {:membership, role}, Membership.changeset(user, role))
end)
```
**Better alternative:**
```elixir
Multi.new()
|> Multi.insert(:primary_membership, Membership.changeset(user, :primary))
|> Multi.insert(:secondary_membership, Membership.changeset(user, :secondary))
```
**Why:** Tuple keys shine for truly dynamic collections where the size is not known at compile time. For small, fixed sets, distinct atom names are more readable and produce clearer error messages.
---
## 7. `Multi.to_list/1` — Testing Without a Database
**Source:** [lib/ecto/multi.ex#L88](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/multi.ex#L88)
```elixir
test "dry run password reset" do
account = %Account{password: "letmein"}
multi = PasswordManager.reset(account, params)
assert [
{:account, {:update, account_changeset, []}},
{:log, {:insert, log_changeset, []}},
{:sessions, {:delete_all, query, []}}
] = Ecto.Multi.to_list(multi)
assert account_changeset.valid?
end
```
**Why:** `Multi.to_list/1` returns the list of operations in the Multi as `{name, {operation_type, changeset_or_query, opts}}` tuples. This allows you to assert on changeset validity, query structure, and operation order without running the transaction against a database. Tests that previously required database setup and teardown can become pure unit tests — faster and fully isolated.
**Anti-pattern:** Always running the transaction in tests even when only changeset validity is being checked:
```elixir
# BAD — hits the database just to validate a changeset
test "password reset changeset is valid" do
account = %Account{password: "letmein"}
{:ok, %{account: updated}} =
PasswordManager.reset(account, valid_params)
|> Repo.transaction()
# The changeset validity was the whole point, not the DB state
assert updated.password != account.password
end
```
### When to Use
**Triggers:**
- You want to unit-test changeset validity or query construction without a database connection
- Your test environment cannot or should not touch the database for a given test
- You want to assert on operation names, order, or count in a Multi pipeline
**Example — before:**
```elixir
# Tests require database + ExUnit.DataCase + fixtures
@tag :integration
test "creates user with profile" do
{:ok, %{user: user, profile: profile}} =
UserRegistration.multi(valid_params())
|> Repo.transaction()
assert user.email == "test@example.com"
assert profile.user_id == user.id
end
```
**Example — after:**
```elixir
# Pure unit test — no database, no fixtures, no DataCase
test "multi contains user insert with valid changeset" do
multi = UserRegistration.multi(valid_params())
assert [
{:user, {:insert, user_changeset, []}},
{:profile, _}
] = Ecto.Multi.to_list(multi)
assert user_changeset.valid?
assert Ecto.Changeset.get_change(user_changeset, :email) == "test@example.com"
end
```
### When NOT to Use
**Don't use this when:**
- Operations use the function variant (e.g., `Multi.insert(:comment, fn %{post: post} -> ... end)`) — those deferred functions are opaque in `to_list` output until the transaction runs
- You need to test that the operations actually succeed against a real database (constraint checks, triggers, concurrent writes)
- The Multi uses `Multi.run` with side effects — `to_list` cannot execute those callbacks
**Over-application example:**
```elixir
# to_list can't help here — the changeset is hidden inside a function
multi =
Multi.new()
|> Multi.insert(:post, post_changeset)
|> Multi.insert(:comment, fn %{post: post} ->
Comment.changeset(post, params) # This fn is opaque to to_list
end)
# This assertion will fail or be meaningless
[{:comment, {:insert, cs, _}}] = Enum.drop(Ecto.Multi.to_list(multi), 1)
assert cs.valid? # cs is a function, not a changeset
```
**Better alternative:**
```elixir
# Test the function-variant changesets in isolation
test "comment changeset is valid given a post" do
post = %Post{id: 1, title: "first"}
changeset = Comment.changeset(post, valid_comment_params())
assert changeset.valid?
end
```
**Why:** `Multi.to_list/1` reflects the state of the pipeline at build time. Deferred values (function variants, `Multi.run` callbacks, `Multi.merge` functions) are not evaluated until `Repo.transaction` runs them. Test those deferred pieces independently rather than trying to inspect them through `to_list`.
---
## Decision Tree
- If operations are static and ordered → `Multi.new() |> Multi.insert/update/delete`
- If an operation depends on a prior operation's result → use function variant `fn changes -> changeset end`
- If later operations depend on runtime data to decide what to include → `Multi.merge/2` with anonymous function
- If you have reusable Multi fragments to combine → `Multi.append/2` or `Multi.prepend/2`
- If you're updating a dynamic collection → tuple keys `{:operation, id}`
- If you want to validate changesets without hitting the DB → `Multi.to_list/1` in tests
- If operations are simple and static (no dynamic branching) → consider `Repo.transaction(fn -> ... end)` instead
<!-- PATTERN_COMPLETE -->
+43 -1
View File
@@ -2,6 +2,27 @@
Analysis of `lib/elixir/lib/supervisor.ex`, `lib/elixir/lib/dynamic_supervisor.ex`, `lib/elixir/lib/task.ex`, `lib/elixir/lib/task/supervisor.ex`, `lib/elixir/lib/process.ex`, and `lib/elixir/lib/registry.ex`.
## Contents
1. [Pattern 1: Static vs Dynamic Supervision — Choose the Right Tool](#pattern-1-static-vs-dynamic-supervision--choose-the-right-tool)
2. [Pattern 2: PartitionSupervisor for Scalability](#pattern-2-partitionsupervisor-for-scalability)
3. [Pattern 3: Supervision Strategies — Choosing the Right Restart Behavior](#pattern-3-supervision-strategies--choosing-the-right-restart-behavior)
4. [Pattern 4: Restart Intensity (`max_restarts` / `max_seconds`)](#pattern-4-restart-intensity-max_restarts--max_seconds)
5. [Pattern 5: Restart Values — `:permanent` vs `:transient` vs `:temporary`](#pattern-5-restart-values--permanent-vs-transient-vs-temporary)
6. [Pattern 6: Automatic Shutdown for Pipeline Supervisors](#pattern-6-automatic-shutdown-for-pipeline-supervisors)
7. [Pattern 7: Task.async/await for Concurrent Value Computation](#pattern-7-taskasyncawait-for-concurrent-value-computation)
8. [Pattern 8: Task.Supervisor.async_nolink for Fault-Tolerant Task Execution](#pattern-8-tasksupervisorasync_nolink-for-fault-tolerant-task-execution)
9. [Pattern 9: Task Supervisor as DynamicSupervisor Specialization](#pattern-9-task-supervisor-as-dynamicsupervisor-specialization)
10. [Pattern 10: Registry for Dynamic Process Naming and PubSub](#pattern-10-registry-for-dynamic-process-naming-and-pubsub)
11. [Pattern 11: Shutdown Semantics — Graceful Termination](#pattern-11-shutdown-semantics--graceful-termination)
12. [Pattern 12: DynamicSupervisor Internal State — Struct with Restart Tracking](#pattern-12-dynamicsupervisor-internal-state--struct-with-restart-tracking)
13. [Pattern 13: Restart Logic with Exponential Backoff via `:try_again`](#pattern-13-restart-logic-with-exponential-backoff-via-try_again)
14. [Pattern 14: `$ancestors` and `$callers` — Process Lineage Tracking](#pattern-14-ancestors-and-callers--process-lineage-tracking)
15. [Pattern 15: GenServer.reply/2 for Deferred Responses](#pattern-15-genserverreply2-for-deferred-responses)
16. [Pattern 16: Process.alias for Safe Request/Response](#pattern-16-processalias-for-safe-requestresponse)
17. [Pattern 17: Registry Partitioning Strategies](#pattern-17-registry-partitioning-strategies)
18. [Pattern 18: `init/1` Return Values — The Full Spectrum](#pattern-18-init1-return-values--the-full-spectrum)
---
## Pattern 1: Static vs Dynamic Supervision — Choose the Right Tool
@@ -436,7 +457,7 @@ Supervisor.init(children,
**Code example from source:**
```elixir
# Task defaults to :temporary — intentional one-shot work
# (from task.ex:282)
# (from task.ex:327)
def child_spec(arg) do
%{
id: Task,
@@ -1911,4 +1932,25 @@ end
**Why:** `:ignore` means "this child intentionally should not run right now." `{:stop, reason}` means "this child tried to start and failed." Conflating the two hides real failures from your supervision tree.
## Decision Tree
- If you have children known at compile time with ordering dependencies → [Pattern 1: Static vs Dynamic Supervision](#pattern-1-static-vs-dynamic-supervision--choose-the-right-tool)
- If a single DynamicSupervisor or Task.Supervisor is a bottleneck under high spawn load → [Pattern 2: PartitionSupervisor for Scalability](#pattern-2-partitionsupervisor-for-scalability)
- If you need to decide how a supervisor reacts when children share state or have dependencies → [Pattern 3: Supervision Strategies](#pattern-3-supervision-strategies--choosing-the-right-restart-behavior)
- If you want to tune how many restarts are tolerated before escalation → [Pattern 4: Restart Intensity](#pattern-4-restart-intensity-max_restarts--max_seconds)
- If different processes have different lifecycle expectations (one-shot vs permanent) → [Pattern 5: Restart Values](#pattern-5-restart-values--permanent-vs-transient-vs-temporary)
- If a supervisor should self-terminate when its children finish their work → [Pattern 6: Automatic Shutdown](#pattern-6-automatic-shutdown-for-pipeline-supervisors)
- If you need to compute values concurrently and the caller should crash on failure → [Pattern 7: Task.async/await](#pattern-7-taskasyncawait-for-concurrent-value-computation)
- If a GenServer needs to spawn work that might fail without taking down the server → [Pattern 8: Task.Supervisor.async_nolink](#pattern-8-tasksupervisorasync_nolink-for-fault-tolerant-task-execution)
- If you need supervised tasks with caller tracking, async_nolink, and streaming → [Pattern 9: Task Supervisor](#pattern-9-task-supervisor-as-dynamicsupervisor-specialization)
- If you need to look up processes by a dynamic key without atom leaks → [Pattern 10: Registry](#pattern-10-registry-for-dynamic-process-naming-and-pubsub)
- If processes hold external resources that need cleanup on shutdown → [Pattern 11: Shutdown Semantics](#pattern-11-shutdown-semantics--graceful-termination)
- If you are building a custom supervisor-like process and need efficient child tracking → [Pattern 12: DynamicSupervisor Internal State](#pattern-12-dynamicsupervisor-internal-state--struct-with-restart-tracking)
- If a child fails to start due to transient conditions and you want non-blocking retry → [Pattern 13: Restart Logic with Backoff](#pattern-13-restart-logic-with-exponential-backoff-via-try_again)
- If you need to trace which process initiated spawned work for debugging → [Pattern 14: Process Lineage Tracking](#pattern-14-ancestors-and-callers--process-lineage-tracking)
- If a GenServer needs to do async work before replying to a caller → [Pattern 15: GenServer.reply/2](#pattern-15-genserverreply2-for-deferred-responses)
- If you build a custom request/response protocol with timeouts and need to prevent late replies → [Pattern 16: Process.alias](#pattern-16-processalias-for-safe-requestresponse)
- If your Registry dispatch is slow because of wrong partitioning strategy → [Pattern 17: Registry Partitioning](#pattern-17-registry-partitioning-strategies)
- If you need to communicate "don't start this child" or split init into fast/slow phases → [Pattern 18: init/1 Return Values](#pattern-18-init1-return-values--the-full-spectrum)
<!-- PATTERN_COMPLETE -->
+922
View File
@@ -0,0 +1,922 @@
# Ecto Query Patterns
Patterns extracted from Ecto's query layer source code.
## Contents
1. [Named Query Functions — Composable Query Building](#1-named-query-functions--composable-query-building)
2. [Query Piping — Schema to Query Pipeline](#2-query-piping--schema-to-query-pipeline)
3. [Named Bindings — Position-Independent Composition](#3-named-bindings--position-independent-composition)
4. [`dynamic/2` — Runtime-Constructed Predicates](#4-dynamic2--runtime-constructed-predicates)
5. [`subquery/1` — Correlated Subqueries](#5-subquery1--correlated-subqueries)
6. [`exclude/2` — Strip Clauses for Reuse](#6-exclude2--strip-clauses-for-reuse)
7. [Bindingless Queries — Data-Driven Clauses](#7-bindingless-queries--data-driven-clauses)
8. [`select_merge/3` — Augmenting Selects Dynamically](#8-select_merge3--augmenting-selects-dynamically)
9. [`fragment/1` and `type/2` — Escape Hatches for DB-Specific Expressions](#9-fragment1-and-type2--escape-hatches-for-db-specific-expressions)
---
## 1. Named Query Functions — Composable Query Building
**Source:** [lib/ecto/query.ex#L1112](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/query.ex#L1112)
**What it does:** Define named functions that accept a query and return a refined query. The query itself is the accumulator; each function layers one concern.
```elixir
# From lib/ecto/query.ex lines 1112-1134
def paginate(query, page, size) do
from query,
limit: ^size,
offset: ^((page-1) * size)
end
def published(query) do
from p in query, where: not(is_nil(p.published_at))
end
```
These functions compose naturally at the call site:
```elixir
User |> active() |> published() |> paginate(1, 20)
```
**Why:** Each function encodes exactly one policy decision. The composed result is a single query that the database executes once. Because `from query` appends rather than replaces, the caller chooses which policies to apply and in what order — without any one function needing to know about the others.
**Anti-pattern:** One monolithic query that mixes pagination, filtering, and ordering:
```elixir
# Hard to reuse parts independently
def list_published_users(page, size) do
from u in User,
where: u.active == true and not is_nil(u.published_at),
order_by: [desc: u.inserted_at],
limit: ^size,
offset: ^((page - 1) * size)
end
```
### When to Use
**Triggers:**
- The same filter, ordering, or limit appears in multiple query contexts
- You need to mix and match clauses — some queries paginate, some don't
- A policy (e.g. "only active records") should be enforced consistently without copy-pasting conditions
**Example — before:**
```elixir
def list_recent_posts(page, size) do
from p in Post,
where: not is_nil(p.published_at),
order_by: [desc: p.published_at],
limit: ^size,
offset: ^((page - 1) * size)
end
def count_published_posts do
from p in Post,
where: not is_nil(p.published_at),
select: count()
end
```
**Example — after:**
```elixir
def published(query), do: from p in query, where: not is_nil(p.published_at)
def by_newest(query), do: from p in query, order_by: [desc: p.published_at]
def paginate(query, page, size) do
from query, limit: ^size, offset: ^((page - 1) * size)
end
def list_recent_posts(page, size) do
Post |> published() |> by_newest() |> paginate(page, size)
end
def count_published_posts do
Post |> published() |> select([p], count())
end
```
### When NOT to Use
**Don't use this when:**
- The query is used exactly once and decomposing it adds names with no reuse value
- The clauses are tightly coupled and meaningless in isolation (e.g. a join whose `on` condition references a specific sibling join)
**Over-application example:**
```elixir
# Not worth extracting — used once, no meaningful reuse
def with_user_and_org_and_permissions(query) do
from [u, o, p] in query,
where: u.org_id == o.id and p.user_id == u.id and p.role == "admin"
end
```
**Better alternative:**
```elixir
from u in User,
join: o in Org, on: u.org_id == o.id,
join: p in Permission, on: p.user_id == u.id,
where: p.role == "admin"
```
**Why:** Extraction is worth it when the function has a name that communicates intent reusably. When a query is one-off and the extracted name just paraphrases the code, keep it inline.
---
## 2. Query Piping — Schema to Query Pipeline
**Source:** [lib/ecto/query.ex#L310](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/query.ex#L310)
**What it does:** Ecto implements the `Ecto.Queryable` protocol for schemas, strings, and query structs, so any of them can be the starting point of a pipeline. The pipe operator chains named query functions:
```elixir
# From the macro API docs (query.ex line 319-324)
"users"
|> where([u], u.age > 18)
|> select([u], u.name)
```
Starting from a schema module name is idiomatic:
```elixir
User
|> where([u], u.active == true)
|> order_by([u], u.name)
|> limit(10)
```
**Why:** The pipe operator makes query construction read left-to-right, mirroring how SQL clauses are mentally composed. The `Ecto.Queryable` protocol means `from Schema` and `Schema |> where(...)` are equivalent, so the choice of `from`/`|>` is stylistic — but pipe form scales better when each step is a named function.
**Anti-pattern:** Building one monolithic keyword query instead of small composable pipes:
```elixir
# Cannot reuse paginate, active, or order_by separately
from u in User,
where: u.active == true,
order_by: [asc: u.name],
limit: ^limit,
offset: ^offset
```
### When to Use
**Triggers:**
- You have 3+ clauses that each correspond to an independently reusable policy
- The query is assembled conditionally based on runtime inputs
- You want the query construction steps to be readable as an English sentence
**Example — before:**
```elixir
from p in Post,
where: p.author_id == ^author_id and not is_nil(p.published_at),
order_by: [desc: p.published_at],
limit: 10
```
**Example — after:**
```elixir
Post
|> by_author(author_id)
|> published()
|> by_newest()
|> limit(10)
```
### When NOT to Use
**Don't use this when:**
- The query has two or fewer clauses and a single `from` is more concise
- You're assembling a complex join where positional bindings require a single `from` for clarity
**Over-application example:**
```elixir
# Excessive piping for a trivial lookup
User
|> where([u], u.id == ^id)
|> limit(1)
|> Repo.one()
```
**Better alternative:**
```elixir
Repo.get(User, id)
```
**Why:** `Repo.get/2` and `Repo.get_by/2` exist precisely for simple lookups. Piping adds ceremony without benefit when the standard API already expresses the intent.
---
## 3. Named Bindings — Position-Independent Composition
**Source:** [lib/ecto/query.ex#L211](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/query.ex#L211)
**What it does:** Assign stable names to `from` and `join` sources using `as:`. Reference those names in any function without knowing or caring about join order.
```elixir
# Name the join at definition time (query.ex line 218-219)
posts_with_comments =
from p in Post,
join: c in Comment, as: :comment, on: c.post_id == p.id
# Reference by name instead of position (line 223)
from [p, comment: c] in posts_with_comments, select: {p.title, c.body}
```
Generic sort function that works on any named binding (line 254-256):
```elixir
def sort(query, as, field) do
from [{^as, x}] in query, order_by: field(x, ^field)
end
```
The spread `...` syntax lets you reference first and last bindings without caring about the middle (line 206):
```elixir
from [p, ..., c] in posts_with_comments, select: {p.title, c.body}
```
**Why:** Positional bindings break when a new join is inserted earlier in the pipeline. Named bindings are stable: adding a join between `Post` and `Comment` does not change how `:comment` is referenced. This is essential for composable query libraries where join order is not known at authorship time.
**Anti-pattern:** Relying on position when queries are built across functions:
```elixir
# Breaks if anyone inserts a join before Comment
def with_comment_body(query) do
from [p, c] in query, select: {p.title, c.body}
end
```
### When to Use
**Triggers:**
- A join is referenced from a function that didn't define it
- You're writing generic helpers (sorting, filtering) that work on any named source
- Multiple joins make positional counting error-prone
**Example — before:**
```elixir
def filter_by_org(query) do
from [u, o] in query, where: o.active == true # Breaks if join order changes
end
```
**Example — after:**
```elixir
# Define the join with a name
from u in User,
join: o in Org, as: :org, on: u.org_id == o.id
# Reference by name — order-independent
def filter_by_org(query) do
from [org: o] in query, where: o.active == true
end
```
### When NOT to Use
**Don't use this when:**
- The query lives entirely in one function and is never extended
- You have a single join that will never be reordered — positional is fine and shorter
- You're using `...` spread and don't need to reference intermediate sources by name
**Over-application example:**
```elixir
# Naming everything in a one-off query adds noise
from u in User, as: :user,
join: p in Post, as: :post, on: p.user_id == u.id,
where: p.published == true,
select: u
```
**Better alternative:**
```elixir
from u in User,
join: p in Post, on: p.user_id == u.id,
where: p.published == true,
select: u
```
**Why:** Named bindings pay off at composition boundaries. In a self-contained query, they add verbosity without stability benefit.
---
## 4. `dynamic/2` — Runtime-Constructed Predicates
**Source:** [lib/ecto/query.ex#L770](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/query.ex#L770)
**What it does:** `dynamic/2` builds query expressions at runtime without executing a query. The resulting value can be composed with `and`/`or` and interpolated into `where`, `having`, `on`, `order_by`, and `select_merge`.
```elixir
# From the dynamic/2 docs (query.ex lines 568-585)
conditions = false
conditions =
if params["is_public"] do
dynamic([p], p.is_public or ^conditions)
else
conditions
end
conditions =
if params["allow_reviewers"] do
dynamic([p, a], a.reviewer == true or ^conditions)
else
conditions
end
from query, where: ^conditions
```
The canonical reduce pattern for multi-field search forms:
```elixir
def filter(params) do
Enum.reduce(params, dynamic(true), fn
{:name, name}, dynamic ->
dynamic([p], ^dynamic and p.name == ^name)
{:age, age}, dynamic ->
dynamic([p], ^dynamic and p.age > ^age)
_, dynamic ->
dynamic
end)
end
from p in Post, where: ^filter(params)
```
**Why:** Without `dynamic/2`, building conditional filters requires runtime `if` guards that build different query structs, or string interpolation (SQL injection risk). `dynamic/2` keeps filtering logic in Ecto's type-safe DSL while composing predicates conditionally. The resulting expression is validated and cast before the query runs.
**Anti-pattern:** Building filter strings with interpolation, or separate query branches per condition:
```elixir
# SQL injection risk
where_clause = "name = '#{params["name"]}'"
Repo.query("SELECT * FROM posts WHERE #{where_clause}")
# Brittle — duplicates the query structure N times
query =
if params["name"] do
from p in Post, where: p.name == ^params["name"]
else
from p in Post
end
```
### When to Use
**Triggers:**
- You're building a search or filter form where 0..N conditions apply based on user input
- Conditions need to be composed with `and`/`or` across different code paths
- You want conditional filtering without forking the entire query
**Example — before:**
```elixir
def search(params) do
query = from p in Post
query =
if params[:title] do
from p in query, where: ilike(p.title, ^"%#{params[:title]}%")
else
query
end
query =
if params[:category] do
from p in query, where: p.category == ^params[:category]
else
query
end
query
end
```
**Example — after:**
```elixir
def search(params) do
filters =
Enum.reduce(params, dynamic(true), fn
{:title, title}, d -> dynamic([p], ^d and ilike(p.title, ^"%#{title}%"))
{:category, cat}, d -> dynamic([p], ^d and p.category == ^cat)
_, d -> d
end)
from p in Post, where: ^filters
end
```
### When NOT to Use
**Don't use this when:**
- The conditions are always applied — static `where` clauses in a named function are simpler
- You only have one conditional — a simple `if` that builds two query variants is clearer
- The condition references a join binding that may not exist — use named bindings and verify first
**Over-application example:**
```elixir
# dynamic() for a condition that's always present
filters = dynamic([p], p.active == true)
from p in Post, where: ^filters
```
**Better alternative:**
```elixir
from p in Post, where: p.active == true
```
**Why:** `dynamic/2` introduces a layer of indirection. When the condition is unconditional, a plain `where` clause in the `from` expression communicates intent more directly.
---
## 5. `subquery/1` — Correlated Subqueries
**Source:** [lib/ecto/query.ex#L897](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/query.ex#L897)
**What it does:** `subquery/1` wraps an `Ecto.Query` for use as a source inside another query — in joins, `where` conditions, or directly in `select`. The canonical use case is batched `update_all` without loading rows into memory.
```elixir
# From subquery/1 docs (query.ex lines 869-878)
subset = from(p in Post,
where: p.synced == false and
(is_nil(p.sync_started_at) or p.sync_started_at < ^min_sync_started_at),
limit: ^batch_size
)
Repo.update_all(
from(p in Post, join: s in subquery(subset), on: s.id == p.id),
set: [sync_started_at: NaiveDateTime.utc_now()]
)
```
Correlated subquery in `select` using `parent_as` (lines 894-895):
```elixir
comments_count = from(c in Comment, where: c.post_id == parent_as(:post).id, select: count())
from(p in Post, as: :post, select: %{id: p.id, comments: subquery(comments_count)})
```
**Why:** Batched updates via a subquery join let the database enforce the limit at the SQL level — no rows are fetched into Elixir. `parent_as` correlates a subquery to the outer query's binding, computing aggregates per row without an explicit `GROUP BY` in the outer query.
**Anti-pattern:** Loading rows into memory to get their IDs, then issuing a second query:
```elixir
# Fetches all IDs into memory before updating
ids =
Post
|> where([p], p.synced == false)
|> limit(^batch_size)
|> select([p], p.id)
|> Repo.all()
Repo.update_all(from(p in Post, where: p.id in ^ids),
set: [sync_started_at: NaiveDateTime.utc_now()])
```
### When to Use
**Triggers:**
- You need to batch-update records matching a subselect without loading them
- You need a per-row aggregate (count, sum) in a `select` without adding it as a join
- The subquery filter depends on the parent row's value (`parent_as`)
**Example — before:**
```elixir
# N+1 pattern — one query per post to count comments
posts = Repo.all(Post)
Enum.map(posts, fn post ->
count = Repo.aggregate(from(c in Comment, where: c.post_id == ^post.id), :count)
Map.put(post, :comment_count, count)
end)
```
**Example — after:**
```elixir
comments_count =
from c in Comment,
where: c.post_id == parent_as(:post).id,
select: count()
Repo.all(from p in Post, as: :post,
select: %{id: p.id, title: p.title, comment_count: subquery(comments_count)})
```
### When NOT to Use
**Don't use this when:**
- A join + `group_by` expresses the aggregation more clearly and performs comparably
- The subquery is not correlated — a preload or separate query may be more readable
- The query is simple enough that `Repo.all` + in-memory grouping is fast enough and clearer
**Over-application example:**
```elixir
# Subquery where a simple preload is idiomatic
comments_query = from(c in Comment, where: c.post_id == parent_as(:post).id)
from(p in Post, as: :post, select: %{id: p.id, comments: subquery(comments_query)})
# Returns raw maps, not structs — preloads are often better for associations
```
**Better alternative:**
```elixir
Post |> Repo.all() |> Repo.preload(:comments)
```
**Why:** `subquery/1` is best suited to aggregates and batched writes. For loading associated structs, `preload` is idiomatic and returns properly typed structs that Ecto's association machinery can use.
---
## 6. `exclude/2` — Strip Clauses for Reuse
**Source:** [lib/ecto/query.ex#L989](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/query.ex#L989)
**What it does:** Removes one or more previously set clauses from a query. Enables deriving a variant of a base query — most commonly stripping `select`, `order_by`, and `preload` to build a count query.
```elixir
# From exclude/2 docs (query.ex lines 946-958)
Ecto.Query.exclude(query, :join)
Ecto.Query.exclude(query, :where)
Ecto.Query.exclude(query, :order_by)
Ecto.Query.exclude(query, :select)
Ecto.Query.exclude(query, :preload)
Ecto.Query.exclude(query, :limit)
Ecto.Query.exclude(query, :offset)
# Remove a list at once (line 964)
Ecto.Query.exclude(query, [:limit, :offset])
```
The count query pattern:
```elixir
def count_query(query) do
query
|> exclude(:select)
|> exclude(:order_by)
|> exclude(:preload)
|> select([x], count(x.id))
end
```
**Why:** Without `exclude/2`, you must maintain two parallel query paths — one for data, one for counts — that can drift out of sync. Deriving the count query from the data query guarantees they share all `where` and `join` clauses: adding a filter in one place automatically applies to both.
**Anti-pattern:** Two independent query definitions that must be kept in sync manually:
```elixir
# Any filter added to data_query must also be added to count_query
def data_query(params) do
from p in Post, where: p.active == true, order_by: [desc: p.inserted_at]
end
def count_query(params) do
from p in Post, where: p.active == true, select: count() # easy to forget
end
```
### When to Use
**Triggers:**
- You need both a data query and a count query from the same base (pagination)
- A query includes `order_by` or `limit` that must be absent for counting or aggregation
- You need to reuse a query's `where` clauses in an update or delete without its `select`
**Example — before:**
```elixir
defmodule MyApp.Posts do
def list_posts(filters) do
base = build_base_query(filters)
data = Repo.all(base)
count = Repo.aggregate(build_count_query(filters), :count)
{data, count}
end
defp build_base_query(filters), do: ...
defp build_count_query(filters), do: ... # must track build_base_query manually
end
```
**Example — after:**
```elixir
defmodule MyApp.Posts do
def list_posts(filters) do
base = build_base_query(filters)
data = Repo.all(base)
count = base |> exclude(:select) |> exclude(:order_by) |> Repo.aggregate(:count)
{data, count}
end
defp build_base_query(filters), do: ... # one source of truth
end
```
### When NOT to Use
**Don't use this when:**
- The base query uses a `join` that is only needed for sorting, not filtering — excluding `order_by` still keeps the join, which may produce duplicates in the count
- The clauses to exclude would leave the query in an invalid state (e.g. excluding `select` from a query with `select_merge` built on it)
- The count query differs structurally enough that a shared base would be forced
**Over-application example:**
```elixir
# exclude doesn't remove joins — count may be inflated by join duplicates
def count(query) do
query
|> exclude(:order_by)
|> exclude(:select)
|> select([x], count(x.id))
|> Repo.one()
# If query has a left_join, this overcounts
end
```
**Better alternative:**
```elixir
def count(query) do
query
|> exclude(:order_by)
|> exclude(:select)
|> select([x], count(x.id, :distinct)) # or exclude joins and recount
|> Repo.one()
end
```
**Why:** `exclude/2` removes clause expressions but not join sources. If the base query joins tables that multiply rows (one-to-many), counting without `DISTINCT` overstates results. Know your join cardinality before deriving count queries this way.
---
## 7. Bindingless Queries — Data-Driven Clauses
**Source:** [lib/ecto/query.ex#L264](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/query.ex#L264)
**What it does:** When a query has only one source and clauses use simple field equality or fixed expressions, bindings can be omitted entirely. Clauses accept keyword lists and atom field names.
```elixir
# From the bindingless docs (query.ex lines 265-268)
from Post,
where: [category: "fresh and new"],
order_by: [desc: :published_at],
select: [:id, :title, :body]
```
This is equivalent to the binding form:
```elixir
from p in Post,
where: p.category == "fresh and new",
order_by: [desc: p.published_at],
select: struct(p, [:id, :title, :body])
```
Bindingless syntax is fully dynamic (line 283-287):
```elixir
where = [category: "fresh and new"]
order_by = [desc: :published_at]
select = [:id, :title, :body]
from Post, where: ^where, order_by: ^order_by, select: ^select
```
**Why:** Bindings exist to name sources so they can be referenced in expressions. When you're only filtering by equality on fields of the single source, bindings add syntax without adding capability. The bindingless form is shorter, more data-driven, and maps cleanly to keyword lists built at runtime.
**Anti-pattern:** Always using binding syntax even for simple equality filters:
```elixir
# More verbose than necessary for simple filters
from p in Post,
where: p.category == ^category and p.status == ^status,
select: [:id, :title]
```
### When to Use
**Triggers:**
- The query has exactly one source (no joins)
- All `where` conditions are field equality checks against interpolated values
- You're building query clauses dynamically from a map or keyword list (web search forms, CLIs)
**Example — before:**
```elixir
def search(filters) do
from p in Post,
where: p.category == ^filters[:category],
where: p.status == ^filters[:status],
select: [:id, :title, :body]
end
```
**Example — after:**
```elixir
def search(filters) do
where = Keyword.take(filters, [:category, :status])
from Post, where: ^where, select: [:id, :title, :body]
end
```
### When NOT to Use
**Don't use this when:**
- The query includes a `join` — bindings are required to reference joined sources
- A `where` condition uses operators other than equality (`>`, `<`, `like`, `fragment`)
- You need to pass the source binding to a function like `field/2` or `type/2`
**Over-application example:**
```elixir
# Bindingless can't express non-equality conditions
from Post,
where: [inserted_at: ^date] # Works only for exact equality — not a range
```
**Better alternative:**
```elixir
from p in Post, where: p.inserted_at >= ^start_date and p.inserted_at < ^end_date
```
**Why:** Bindingless keyword syntax maps to equality (`==`). Any non-equality comparison, function call, or multi-table reference requires a named binding. Use bindingless for pure equality filters; reach for bindings the moment expressions get richer.
---
## 8. `select_merge/3` — Augmenting Selects Dynamically
**Source:** [lib/ecto/query.ex#L693](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/query.ex#L693)
**What it does:** Merges additional fields into an existing `select` without replacing it. Especially useful with `dynamic/2` to add computed columns conditionally.
```elixir
# From the dynamic docs (query.ex lines 693-695)
metric = dynamic([p], p.distance)
from query, select: [:period, :metric], select_merge: ^%{metric: metric}
```
With aliasing and dynamic ordering (lines 700-707):
```elixir
fields = %{
period: dynamic([p], selected_as(p.month, :month)),
metric: dynamic([p], p.distance)
}
order = dynamic(selected_as(:month))
from query, select: ^fields, order_by: ^order
```
**Why:** `select_merge` lets base queries define the fixed fields and separate concerns add computed or conditional fields. Without it, adding a field requires rewriting the entire `select` clause — or maintaining multiple `select` variants. Combined with `dynamic/2`, it enables data-driven projections where which columns appear depends on runtime configuration.
**Anti-pattern:** Rewriting the entire `select` whenever a computed column is needed:
```elixir
# The base select must be duplicated in every variant
def with_distance(query) do
from p in query, select: %{id: p.id, name: p.name, distance: p.distance}
end
def without_distance(query) do
from p in query, select: %{id: p.id, name: p.name}
end
```
### When to Use
**Triggers:**
- A computed column should be added conditionally depending on caller context
- You're building a reporting query where which aggregates appear is configured at runtime
- A base query provides the structural select and feature-specific code augments it
**Example — before:**
```elixir
def list_metrics(include_distance?) do
if include_distance? do
from p in Post, select: %{period: p.period, metric: p.views, distance: p.distance}
else
from p in Post, select: %{period: p.period, metric: p.views}
end
end
```
**Example — after:**
```elixir
def list_metrics(opts) do
base = from p in Post, select: %{period: p.period, metric: p.views}
if opts[:include_distance] do
from p in base, select_merge: %{distance: p.distance}
else
base
end
end
```
### When NOT to Use
**Don't use this when:**
- The `select` is a single value or tuple rather than a map — `select_merge` requires a map shape
- The added field requires a binding not present in the base query
- You're replacing, not augmenting — use a plain `select` or `exclude(:select)` first
**Over-application example:**
```elixir
# select_merge on a non-map select causes a runtime error
from p in Post,
select: p.name,
select_merge: %{email: p.email} # error: base select is not a map
```
**Better alternative:**
```elixir
from p in Post, select: %{name: p.name, email: p.email}
```
**Why:** `select_merge` merges into the existing map-shaped select. If the base select returns a struct or a scalar, there is no map to merge into. Ensure the base `select` produces a map before using `select_merge`.
---
## 9. `fragment/1` and `type/2` — Escape Hatches for DB-Specific Expressions
**Source:** [lib/ecto/query.ex#L291](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/query.ex#L291)
**What it does:** `fragment/1` passes a raw SQL expression to the database engine for functions or operators the Ecto DSL cannot express. `type/2` coerces an Elixir value to a specific Ecto type for comparison when schema type information is not available.
```elixir
# From the fragments docs (query.ex lines 301-303)
from p in Post,
where: is_nil(p.published_at) and
fragment("lower(?)", p.title) == ^title
```
`type/2` for schemaless queries where Ecto cannot infer the cast type:
```elixir
# Coerce to :integer when no schema field exists to infer from
from u in "users",
where: u.age > type(^age, :integer)
```
**Why:** Databases have functions (full-text search, JSON operators, trigram similarity, window functions) that the Ecto DSL cannot enumerate. `fragment/1` is the intentional escape hatch: interpolations with `?` placeholders are still parameterized, so SQL injection is not a risk. `type/2` is necessary for schemaless queries where Ecto cannot cast a bound parameter to the correct DB type automatically.
**Anti-pattern:** Using fragment for everything, bypassing Ecto's type safety and composability:
```elixir
# Loses all type inference; fragment output is opaque to Ecto
from u in User,
where: fragment("? = ? AND ? > ?", u.name, ^name, u.age, ^age)
```
Or using string interpolation inside fragments:
```elixir
# SQL injection — never interpolate directly into fragment strings
from u in User,
where: fragment("lower(email) = '#{email}'")
```
### When to Use
**Triggers:**
- The database function has no Ecto DSL equivalent (e.g. `lower()`, `similarity()`, `jsonb_array_elements()`)
- You're writing a schemaless query (`from u in "users"`) and need to cast a bound parameter
- A DB-specific operator or syntax is required for a performance-critical path
**Example — before:**
```elixir
# Can't express case-insensitive equality in pure Ecto DSL
from u in User, where: u.email == ^String.downcase(email)
# Compares raw stored value; doesn't work if DB stores mixed case
```
**Example — after:**
```elixir
from u in User,
where: fragment("lower(?)", u.email) == ^String.downcase(email)
```
### When NOT to Use
**Don't use this when:**
- The Ecto DSL or a library like `EctoCommons` already provides the operation
- You want type-cast values in a regular schema query — Ecto infers the type from the field
- You're tempted to fragment an entire `WHERE` clause — named functions with `dynamic/2` compose better
**Over-application example:**
```elixir
# Fragments for standard operations Ecto handles natively
from p in Post,
where: fragment("? = ?", p.status, ^:published),
order_by: fragment("? DESC", p.inserted_at)
```
**Better alternative:**
```elixir
from p in Post,
where: p.status == :published,
order_by: [desc: p.inserted_at]
```
**Why:** Fragment output is opaque to Ecto's type system: no cast validation, no composability with `dynamic/2` type inference, and no portability across adapters. Reserve `fragment/1` for genuine gaps in the DSL; prefer native Ecto expressions for everything the DSL can express.
---
## Decision Tree
- If you need to filter conditionally at runtime → `dynamic/2` (Pattern 4)
- If you need to join or sort across function composition boundaries → named bindings with `as:` (Pattern 3)
- If you need a count or aggregate from the same base as a data query → `exclude/2` (Pattern 6)
- If you need a DB-side correlated count or aggregate per row → `subquery/1` with `parent_as` (Pattern 5)
- If the query has one source and all filters are equality checks → bindingless keyword syntax (Pattern 7)
- If you need to add computed columns without rewriting the select → `select_merge/3` (Pattern 8)
- If the DB function has no Ecto DSL equivalent → `fragment/1` as last resort (Pattern 9)
- For all queries: define small named functions that take a query and return a query (Patterns 1 and 2)
<!-- PATTERN_COMPLETE -->
+893
View File
@@ -0,0 +1,893 @@
# Ecto Schema Patterns
Patterns extracted from `lib/ecto/schema.ex` in the Ecto source.
## Contents
1. [Base Schema Module — App-Wide Schema Defaults](#1-base-schema-module--app-wide-schema-defaults)
2. [`@primary_key false` — Composite or No Primary Key](#2-primary_key-false--composite-or-no-primary-key)
3. [Virtual Fields — In-Memory-Only Data](#3-virtual-fields--in-memory-only-data)
4. [`embedded_schema/1` — Schemaless Validation Structs](#4-embedded_schema1--schemaless-validation-structs)
5. [`@timestamps_opts` — Consistent Timestamp Types](#5-timestamps_opts--consistent-timestamp-types)
6. [Field `:source` Option — Column Name Mapping](#6-field-source-option--column-name-mapping)
7. [`redact: true` — Protecting Sensitive Fields](#7-redact-true--protecting-sensitive-fields)
8. [`__schema__/1` Reflection — Runtime Schema Introspection](#8-__schema__1-reflection--runtime-schema-introspection)
---
## 1. Base Schema Module — App-Wide Schema Defaults
**Source:** [lib/ecto/schema.ex#L194](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L194)
```elixir
defmodule MyApp.Schema do
defmacro __using__(_) do
quote do
use Ecto.Schema
@primary_key {:id, :binary_id, autogenerate: true}
@foreign_key_type :binary_id
end
end
end
defmodule MyApp.Comment do
use MyApp.Schema
schema "comments" do
belongs_to :post, MyApp.Post
end
end
```
**Why:** `@primary_key` and `@foreign_key_type` are module attributes that must be set before each `schema/2` block. Without a shared base module, every schema in the application must repeat these two lines. One forgotten schema silently gets integer `:id` columns and integer foreign keys — a type mismatch that only surfaces at runtime when joins or associations break.
**Anti-pattern:** Setting `@primary_key` in every schema individually:
```elixir
# BAD — duplicated in every schema, easy to miss in a new module
defmodule MyApp.Comment do
use Ecto.Schema
@primary_key {:id, :binary_id, autogenerate: true}
@foreign_key_type :binary_id
schema "comments" do
belongs_to :post, MyApp.Post
end
end
defmodule MyApp.Post do
use Ecto.Schema
# Forgot to set @primary_key — now uses integer :id
schema "posts" do
has_many :comments, MyApp.Comment # Association type mismatch
end
end
```
### When to Use
**Triggers:**
- Application uses UUID (`:binary_id`) primary keys
- Application has more than one schema module
- You want all future schemas to inherit the same defaults without manual setup
**Example — before:**
```elixir
defmodule MyApp.User do
use Ecto.Schema
@primary_key {:id, :binary_id, autogenerate: true}
@foreign_key_type :binary_id
schema "users" do
field :name, :string
end
end
defmodule MyApp.Post do
use Ecto.Schema
@primary_key {:id, :binary_id, autogenerate: true}
@foreign_key_type :binary_id
schema "posts" do
belongs_to :user, MyApp.User
end
end
```
**Example — after:**
```elixir
# lib/my_app/schema.ex — define once
defmodule MyApp.Schema do
defmacro __using__(_) do
quote do
use Ecto.Schema
@primary_key {:id, :binary_id, autogenerate: true}
@foreign_key_type :binary_id
end
end
end
# All schemas use the base module
defmodule MyApp.User do
use MyApp.Schema
schema "users" do
field :name, :string
end
end
defmodule MyApp.Post do
use MyApp.Schema
schema "posts" do
belongs_to :user, MyApp.User
end
end
```
### When NOT to Use
**Don't use this when:**
- The application uses integer primary keys (Ecto's default) — no base module needed
- Different schemas intentionally use different primary key types
- You only have one schema (the base module adds indirection for no gain)
**Over-application example:**
```elixir
# Unnecessary when using integer primary keys (the Ecto default)
defmodule MyApp.Schema do
defmacro __using__(_) do
quote do
use Ecto.Schema
@primary_key {:id, :id, autogenerate: true} # Same as the default (:id resolves to integer)
@foreign_key_type :id
end
end
end
```
**Better alternative:**
```elixir
# Just use Ecto.Schema directly when the defaults are fine
defmodule MyApp.Post do
use Ecto.Schema
schema "posts" do
field :title, :string
end
end
```
**Why:** The base module pattern exists to override Ecto defaults, not to mirror them. When integer primary keys are acceptable, the base module adds a layer of indirection without adding any value.
---
## 2. `@primary_key false` — Composite or No Primary Key
**Source:** [lib/ecto/schema.ex#L1525](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L1525)
```elixir
defmodule PostTag do
use Ecto.Schema
@primary_key false
schema "posts_tags" do
belongs_to :post, Post
belongs_to :tag, Tag
end
end
```
**Why:** By default, Ecto injects an `:id` field as the first field in every schema. For join tables with composite primary keys — or tables that intentionally have no primary key — this auto-injected field causes problems: migrations add a spurious column, queries include it, and Ecto's identity tracking operates on the wrong key. Setting `@primary_key false` disables the auto-injection entirely, giving the schema full control over which fields exist.
**Anti-pattern:** Defining a join table schema with the default `:id` field:
```elixir
# BAD — Ecto injects :id, which doesn't exist in posts_tags
defmodule PostTag do
use Ecto.Schema
schema "posts_tags" do
belongs_to :post, Post
belongs_to :tag, Tag
end
end
# Migration adds an :id column that the DB or application doesn't need.
# Ecto load/insert operations will reference a non-meaningful primary key.
```
### When to Use
**Triggers:**
- The table is a join/association table with a composite primary key (e.g., `post_id + tag_id`)
- The table has no meaningful single-column primary key at all
- You're mapping a legacy table that lacks a primary key column
**Example — before:**
```elixir
# posts_tags table has only (post_id, tag_id) — no separate :id column
defmodule PostTag do
use Ecto.Schema
schema "posts_tags" do
belongs_to :post, Post # generates post_id
belongs_to :tag, Tag # generates tag_id
timestamps()
end
end
# Ecto injects :id — but the DB column doesn't exist, causing query errors.
```
**Example — after:**
```elixir
defmodule PostTag do
use Ecto.Schema
@primary_key false
schema "posts_tags" do
belongs_to :post, Post
belongs_to :tag, Tag
timestamps()
end
end
```
### When NOT to Use
**Don't use this when:**
- The table has a legitimate single-column primary key (use the default or `@primary_key {:id, :binary_id, autogenerate: true}`)
- You want a composite primary key but still need Ecto's `Repo.get/2` and identity-based operations — those rely on a single primary key field
- The schema is used with `Repo.get/2` or `Repo.update/2`, which require a primary key
**Over-application example:**
```elixir
# Disabling primary key on a regular entity schema
defmodule MyApp.User do
use Ecto.Schema
@primary_key false # Users need a primary key for Repo.get/2
schema "users" do
field :email, :string
field :name, :string
end
end
```
**Better alternative:**
```elixir
defmodule MyApp.User do
use Ecto.Schema
schema "users" do # Default :id primary key is correct here
field :email, :string
field :name, :string
end
end
```
**Why:** `@primary_key false` removes Ecto's ability to uniquely identify rows. `Repo.get/2`, `Repo.update/2`, and association loading all depend on a primary key. Use it only for tables that are genuinely identified by composite keys or have no identity concept.
---
## 3. Virtual Fields — In-Memory-Only Data
**Source:** [lib/ecto/schema.ex#L332](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L332)
```elixir
schema "users" do
field :password_hash, :string
field :password, :string, virtual: true
field :delete, :boolean, virtual: true
end
```
**Why:** Virtual fields live on the struct and participate in changesets — they can be `cast`, `validate_required`, and read in changeset functions — but Ecto never attempts to load or persist them. This is exactly right for data that only exists for the duration of an operation: a plaintext password for hashing, a confirmation field, a checkbox that signals intent to delete, or a computed display value assembled in application code.
**Anti-pattern:** Storing transient data in a map or separate variable passed alongside the changeset:
```elixir
# BAD — passing raw password alongside changeset is error-prone
def register_user(params) do
changeset = User.changeset(%User{}, params)
password = Map.get(params, "password") # Not validated, not in changeset pipeline
if changeset.valid?, do: hash_and_insert(changeset, password)
end
```
### When to Use
**Triggers:**
- A field is needed in changeset validation but must never be persisted (passwords, confirmation fields)
- A value is computed in application code and attached to the struct for rendering but has no DB column
- A form sends a signal (e.g., a "delete" checkbox) that controls behavior rather than being stored
**Example — before:**
```elixir
# Confirmation field handled outside the changeset pipeline
def create_account(params) do
password = params["password"]
confirm = params["password_confirmation"]
if password != confirm do
{:error, "passwords do not match"}
else
User.changeset(%User{}, params) |> Repo.insert()
end
end
```
**Example — after:**
```elixir
defmodule User do
use Ecto.Schema
import Ecto.Changeset
schema "users" do
field :email, :string
field :password_hash, :string
field :password, :string, virtual: true
field :password_confirmation, :string, virtual: true
end
def registration_changeset(user, params) do
user
|> cast(params, [:email, :password, :password_confirmation])
|> validate_required([:email, :password])
|> validate_confirmation(:password)
|> hash_password()
end
defp hash_password(%{valid?: true, changes: %{password: pw}} = changeset) do
put_change(changeset, :password_hash, Bcrypt.hash_pwd_salt(pw))
end
defp hash_password(changeset), do: changeset
end
```
### When NOT to Use
**Don't use this when:**
- The value should be persisted and loaded from the DB (use a regular field)
- The value needs to participate in Ecto queries (`where`, `order_by`) — virtual fields cannot be queried
- The field has a `:default` that should be the DB column default — virtual field defaults only affect the struct, not the database
**Over-application example:**
```elixir
# Using virtual for a value that should live in the DB
schema "articles" do
field :title, :string
field :slug, :string, virtual: true # Wrong — slug needs to be persisted and queried
field :body, :string
end
```
**Better alternative:**
```elixir
schema "articles" do
field :title, :string
field :slug, :string # Regular field — persisted and queryable
field :body, :string
end
```
**Why:** Virtual fields are intentionally invisible to the database layer. Any field that needs to survive a page reload, be searched, sorted, or returned in a list query must be a real column.
---
## 4. `embedded_schema/1` — Schemaless Validation Structs
**Source:** [lib/ecto/schema.ex#L110](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L110)
```elixir
defmodule Search do
use Ecto.Schema
import Ecto.Changeset
embedded_schema do
field :query, :string
field :page, :integer, default: 1
field :per_page, :integer, default: 20
end
def changeset(search, params) do
search
|> cast(params, [:query, :page, :per_page])
|> validate_required([:query])
|> validate_number(:page, greater_than: 0)
end
end
```
**Why:** `embedded_schema` defines a schema module with no backing DB table. The struct, changeset pipeline, and all validations work identically to a regular schema, but there is no `Repo` involved. This makes it the right tool for validating and casting data that doesn't map to a table: search/filter forms, API request bodies, command parameters, multi-step wizard state. A `Search.changeset/2` call returns a changeset with errors just like `User.changeset/2` does — the same controller and form helper code works for both.
**Anti-pattern:** Validating formless data with raw `Map` operations and ad-hoc checks:
```elixir
# BAD — manual validation without changeset pipeline
def search(params) do
query = Map.get(params, "query")
page = Map.get(params, "page", "1") |> String.to_integer()
if is_nil(query) or query == "" do
{:error, "query is required"}
else
{:ok, %{query: query, page: page}}
end
# No type coercion, no error accumulation, no `valid?` flag
end
```
### When to Use
**Triggers:**
- A form or API endpoint needs validated, typed input but no persistence
- You want `Ecto.Changeset` error formatting and `valid?` semantics without a Repo
- Multi-step wizard where intermediate state needs validation before DB write
- An external API receives a complex payload that needs casting and validation
**Example — before:**
```elixir
# Validating filter params without Ecto — verbose and fragile
def filter_posts(params) do
status = Map.get(params, "status")
valid_statuses = ["draft", "published", "archived"]
cond do
status not in valid_statuses ->
{:error, "status must be one of: #{Enum.join(valid_statuses, ", ")}"}
true ->
Post |> where(status: ^status) |> Repo.all() |> then(&{:ok, &1})
end
end
```
**Example — after:**
```elixir
defmodule PostFilter do
use Ecto.Schema
import Ecto.Changeset
embedded_schema do
field :status, Ecto.Enum, values: [:draft, :published, :archived]
field :page, :integer, default: 1
end
def changeset(filter \\ %PostFilter{}, params) do
filter
|> cast(params, [:status, :page])
|> validate_required([:status])
|> validate_number(:page, greater_than: 0)
end
end
def filter_posts(params) do
with %{valid?: true} = cs <- PostFilter.changeset(params),
filter <- Ecto.Changeset.apply_changes(cs) do
Post |> where(status: ^filter.status) |> Repo.all() |> then(&{:ok, &1})
else
cs -> {:error, cs}
end
end
```
### When NOT to Use
**Don't use this when:**
- The data needs to be persisted — use a regular `schema/2` with a migration
- The validation logic is trivial (one or two fields) — a plain `cast` + `validate_required` in a context module may be sufficient
- You need DB-level constraints or uniqueness checks — `embedded_schema` has no Repo access
**Over-application example:**
```elixir
# Overkill for a single required string field
defmodule SearchQuery do
use Ecto.Schema
embedded_schema do
field :q, :string
end
def changeset(s, p), do: s |> cast(p, [:q]) |> validate_required([:q])
end
```
**Better alternative:**
```elixir
# Simple inline validation is clearer for trivial cases
def search(params) do
case Map.fetch(params, "q") do
{:ok, q} when q != "" -> {:ok, q}
_ -> {:error, :missing_query}
end
end
```
**Why:** `embedded_schema` earns its keep when you have multiple fields, type coercion, or multiple validators — the changeset pipeline pays for itself. For a single field with one constraint, the ceremony outweighs the benefit.
---
## 5. `@timestamps_opts` — Consistent Timestamp Types
**Source:** [lib/ecto/schema.ex#L170](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L170)
```elixir
defmodule MyApp.Schema do
defmacro __using__(_) do
quote do
use Ecto.Schema
@timestamps_opts [type: :utc_datetime_usec]
end
end
end
```
**Why:** The `timestamps()` macro inserts `inserted_at` and `updated_at` fields whose type is controlled by `@timestamps_opts`. The default type is `:naive_datetime`, which stores timestamps with no timezone information. Applications that store or compare timestamps across timezones need `:utc_datetime` or `:utc_datetime_usec`. Microsecond precision (`:utc_datetime_usec`) avoids silent truncation when high-resolution timestamps are generated in Elixir and stored in PostgreSQL (which supports microsecond precision). Setting this in the base module ensures every schema uses the same timestamp type.
**Anti-pattern:** Using the default `naive_datetime` timestamps when UTC compliance or precision is required:
```elixir
# BAD — naive_datetime drops timezone context
defmodule MyApp.Post do
use Ecto.Schema
schema "posts" do
field :title, :string
timestamps() # inserts inserted_at/updated_at as :naive_datetime
end
end
# Timestamps stored as "2024-01-15 10:30:00" — no UTC indicator.
# Comparisons and serialization can silently produce wrong results.
```
### When to Use
**Triggers:**
- Application stores or compares timestamps across timezones (almost always true for web apps)
- Timestamps need to be serialized to ISO 8601 / JSON with timezone information
- PostgreSQL or another DB with microsecond precision should not lose sub-second data
- Multiple schemas share the same timestamp behavior
**Example — before:**
```elixir
# Each schema repeats the option
defmodule MyApp.User do
use Ecto.Schema
schema "users" do
timestamps(type: :utc_datetime_usec)
end
end
defmodule MyApp.Post do
use Ecto.Schema
schema "posts" do
timestamps(type: :utc_datetime_usec)
end
end
```
**Example — after:**
```elixir
# Base module sets the default once
defmodule MyApp.Schema do
defmacro __using__(_) do
quote do
use Ecto.Schema
@primary_key {:id, :binary_id, autogenerate: true}
@foreign_key_type :binary_id
@timestamps_opts [type: :utc_datetime_usec]
end
end
end
defmodule MyApp.User do
use MyApp.Schema
schema "users" do
timestamps() # uses :utc_datetime_usec from base module
end
end
```
### When NOT to Use
**Don't use this when:**
- The application intentionally avoids timezones (internal tooling, batch jobs)
- The DB column type is `timestamp without time zone` and you want to preserve that semantic
- Only one or two schemas use timestamps and the option is set inline
**Over-application example:**
```elixir
# Setting timestamps_opts for a schema that has no timestamps() call
defmodule PostTag do
use Ecto.Schema
@primary_key false
@timestamps_opts [type: :utc_datetime_usec] # Never used — no timestamps() call
schema "posts_tags" do
belongs_to :post, Post
belongs_to :tag, Tag
end
end
```
**Better alternative:**
```elixir
# The option is harmless but unnecessary — omit it when there are no timestamps
defmodule PostTag do
use MyApp.Schema # Base module sets it — no need to repeat
@primary_key false
schema "posts_tags" do
belongs_to :post, Post
belongs_to :tag, Tag
end
end
```
**Why:** `@timestamps_opts` only takes effect when `timestamps()` is called inside a `schema` block. Setting it on schemas that call `timestamps()` is correct; setting it on schemas that don't is dead code.
---
## 6. Field `:source` Option — Column Name Mapping
**Source:** [lib/ecto/schema.ex](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex)
```elixir
schema "legacy_users" do
field :email_address, :string, source: :emailaddress
field :created_at, :utc_datetime, source: :creation_timestamp
end
```
**Why:** The `:source` option on `field/3` maps an Elixir field name to a different database column name. This is the escape hatch for working with databases that don't follow Elixir naming conventions — legacy systems with camelCase or abbreviated column names, databases shared with other applications, or tables generated by external tools. The Elixir struct uses the field name (`:email_address`); SQL uses the source name (`emailaddress`). Ecto translates transparently in both directions — `select`, `where`, and `insert` all use the column name.
**Anti-pattern:** Mapping column names by wrapping all query fragments in raw SQL or using `fragment/1` everywhere:
```elixir
# BAD — forced to use raw column names in every query
from u in "legacy_users",
where: fragment("emailaddress = ?", ^email),
select: %{email: u.emailaddress, created_at: u.creation_timestamp}
# Loses type casting, association loading, and changeset integration
```
### When to Use
**Triggers:**
- The database column name cannot be changed (shared DB, legacy system, migration risk)
- The column name violates Elixir conventions (camelCase, abbreviations, reserved words)
- You want Ecto's type casting and query DSL to work with idiomatic Elixir field names
**Example — before:**
```elixir
# Schema mirrors the ugly DB column names directly
schema "legacy_users" do
field :emailaddress, :string
field :creation_timestamp, :utc_datetime
field :lastlogindt, :utc_datetime
end
# Every callsite uses the ugly names
from u in User, where: u.emailaddress == ^email
%{emailaddress: user.emailaddress, created: user.creation_timestamp}
```
**Example — after:**
```elixir
schema "legacy_users" do
field :email, :string, source: :emailaddress
field :inserted_at, :utc_datetime, source: :creation_timestamp
field :last_login_at, :utc_datetime, source: :lastlogindt
end
# Callsites use clean Elixir names
from u in User, where: u.email == ^email
%{email: user.email, created: user.inserted_at}
```
### When NOT to Use
**Don't use this when:**
- You control the database schema — use a migration to rename the column instead
- The column name follows Elixir conventions already — `:source` adds no value
- You're creating a new table — design the column names correctly from the start
**Over-application example:**
```elixir
# Renaming columns that are already idiomatic
schema "users" do
field :user_name, :string, source: :username # username is fine as-is
field :created_at_time, :utc_datetime, source: :inserted_at # just use inserted_at
end
```
**Better alternative:**
```elixir
schema "users" do
field :username, :string # Keep the DB name if it's already clear
timestamps() # inserted_at / updated_at are already conventional
end
```
**Why:** `:source` is a mapping layer between two names that creates a permanent translation cost every time someone reads the schema. When you control the DB, eliminate the mismatch at the migration level rather than carrying it forever in the schema.
---
## 7. `redact: true` — Protecting Sensitive Fields
**Source:** [lib/ecto/schema.ex#L128](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L128)
```elixir
schema "users" do
field :email, :string
field :password_hash, :string, redact: true
field :api_token, :string, redact: true
end
```
**Why:** Ecto derives an `Inspect` implementation for every schema struct. Without `redact: true`, inspecting a changeset or struct in logs, IEx, or crash reports prints every field value in plain text. Marking a field with `redact: true` replaces its value with `**redacted**` in `inspect/2` output and in `changeset.changes` inspection. This is a passive, always-on protection that requires no additional code at each log site — once set on the field, the field is protected everywhere the struct is printed.
**Anti-pattern:** Relying on callers to manually omit sensitive fields before logging:
```elixir
# BAD — easy to forget, must be repeated at every log site
def create_user(params) do
changeset = User.changeset(%User{}, params)
# Must remember to drop :password_hash before logging
safe = Map.drop(changeset.changes, [:password_hash, :api_token])
Logger.info("Creating user: #{inspect(safe)}")
Repo.insert(changeset)
end
```
### When to Use
**Triggers:**
- A field stores a secret, credential, or token (passwords, API keys, session tokens)
- A field stores PII that should not appear in logs or error reports (SSN, credit card data)
- A changeset for the schema might be logged, inspected in IEx, or appear in crash reports
- You use `Logger.info/debug` calls that could inadvertently print struct values
**Example — before:**
```elixir
# Sensitive values appear in plain text in logs and crash dumps
defmodule User do
use Ecto.Schema
schema "users" do
field :email, :string
field :password_hash, :string # Logged as "password_hash: \"$2b$12$...\""
field :reset_token, :string # Logged as "reset_token: \"abc123secret\""
end
end
```
**Example — after:**
```elixir
defmodule User do
use Ecto.Schema
schema "users" do
field :email, :string
field :password_hash, :string, redact: true
field :reset_token, :string, redact: true
end
end
# inspect(user) => #User<email: "alice@example.com", password_hash: **redacted**, ...>
```
### When NOT to Use
**Don't use this when:**
- The field contains non-sensitive data — redaction adds visual noise to inspect output
- You need to see the field value during development debugging (temporarily remove `redact: true` locally, or use pattern matching to extract the value directly)
- The struct is used exclusively in contexts where it is never printed (pure computation)
**Over-application example:**
```elixir
# Redacting non-sensitive fields makes debugging needlessly difficult
schema "products" do
field :name, :string, redact: true # Why is a product name sensitive?
field :price, :decimal, redact: true # Price data isn't a secret
field :sku, :string, redact: true
end
```
**Better alternative:**
```elixir
schema "products" do
field :name, :string
field :price, :decimal
field :sku, :string
end
```
**Why:** `redact: true` makes fields invisible to standard Elixir tooling. Overusing it on non-sensitive fields makes crash reports and debug output harder to read without adding any security benefit. Apply it precisely to fields whose values would constitute a security or privacy exposure if logged.
---
## 8. `__schema__/1` Reflection — Runtime Schema Introspection
**Source:** [lib/ecto/schema.ex#L450](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L450)
```elixir
# Get all field names
User.__schema__(:fields) #=> [:id, :name, :email, :age]
User.__schema__(:associations) #=> [:posts, :comments]
User.__schema__(:type, :email) #=> :string
User.__schema__(:virtual_fields) #=> [:password]
```
**Why:** Every module that calls `use Ecto.Schema` gets `__schema__/1` generated at compile time. These reflection callbacks expose the schema's structure at runtime without requiring the caller to know the schema's field list at compile time. Generic code — CSV exporters, audit loggers, API serializers, test factories — can introspect any schema and work with all its fields automatically. When a new field is added to the schema, the generic code picks it up without modification.
**Anti-pattern:** Hardcoding field names in generic helpers:
```elixir
# BAD — must be updated every time any schema gains or loses a field
def audit_changes(changeset) do
fields = [:name, :email, :role] # Hardcoded — User-specific, not generic
Enum.each(fields, fn field ->
if Map.has_key?(changeset.changes, field) do
AuditLog.record(field, changeset.changes[field])
end
end)
end
```
### When to Use
**Triggers:**
- Writing a helper that must work across multiple schema modules without knowing their fields
- Building a generic CSV exporter, JSON serializer, or audit logger
- Generating test factories or seed data for any schema
- Writing a linter or validation tool that checks schema conventions
**Example — before:**
```elixir
# Must manually maintain field lists for each schema
defmodule CsvExporter do
def export_users(records) do
headers = ["id", "name", "email", "age"]
rows = Enum.map(records, fn u -> [u.id, u.name, u.email, u.age] end)
[headers | rows]
end
def export_posts(records) do
headers = ["id", "title", "body", "user_id"]
rows = Enum.map(records, fn p -> [p.id, p.title, p.body, p.user_id] end)
[headers | rows]
end
end
```
**Example — after:**
```elixir
defmodule CsvExporter do
def export(schema, records) do
fields = schema.__schema__(:fields)
headers = Enum.map(fields, &to_string/1)
rows = Enum.map(records, fn record ->
Enum.map(fields, &Map.get(record, &1))
end)
[headers | rows]
end
end
# Works for any schema without modification
CsvExporter.export(User, users)
CsvExporter.export(Post, posts)
```
### When NOT to Use
**Don't use this when:**
- You're writing code for one specific schema and the fields are known and stable — just use them directly
- You need to control exactly which fields are exposed (e.g., public API — use an explicit allowlist)
- The schema has virtual fields or sensitive fields that should not be included — reflection returns all fields regardless of `virtual: true` or `redact: true`
**Over-application example:**
```elixir
# Using reflection for a single known schema adds indirection with no benefit
def display_user(user) do
User.__schema__(:fields)
|> Enum.map(fn field -> "#{field}: #{Map.get(user, field)}" end)
|> Enum.join(", ")
# Dumps :password_hash, :reset_token, and every internal field
end
```
**Better alternative:**
```elixir
# For a known schema, use explicit fields — safer and clearer intent
def display_user(user) do
"#{user.name} <#{user.email}>"
end
```
**Why:** `__schema__(:fields)` returns all persisted fields including internal ones (`inserted_at`, `updated_at`, foreign keys). For display or serialization of a known schema, an explicit field list documents intent and prevents sensitive data from leaking. Use reflection for genuinely generic code where the alternative is a hardcoded list that diverges from the schema over time.
---
## Decision Tree
- If the app uses UUID primary keys across all schemas → create a base schema module with `@primary_key {:id, :binary_id, autogenerate: true}` and `@foreign_key_type :binary_id`
- If a table has no single primary key (join table, composite key) → `@primary_key false`
- If a field holds runtime-only data (password input, confirmation field, computed display value) → `virtual: true`
- If you need changeset validation without a DB table (form objects, API request bodies) → `embedded_schema`
- If timestamps need UTC compliance or microsecond precision → set `@timestamps_opts [type: :utc_datetime_usec]` in the base schema module
- If a DB column name doesn't match Elixir conventions and can't be migrated → `:source` option on `field/3`
- If a field holds sensitive data (passwords, tokens, PII) that must not appear in logs → `redact: true`
- If writing generic code that works across schema modules without knowing their fields → `__schema__/1` reflection
<!-- PATTERN_COMPLETE -->
+116
View File
@@ -0,0 +1,116 @@
# Telemetry Patterns
## Pattern 1: Dedicated Telemetry Submodule
**When to use:** Any module that emits 2+ telemetry events, especially GenServers and pipeline stages.
**What it does:** Extracts all `:telemetry.execute/3` calls into a sibling `Telemetry` module. The parent module calls into the telemetry module; the telemetry module owns event names, measurements shape, and metadata contracts.
**Structure:**
```
lib/my_app/quote_feed.ex # GenServer — state + logic
lib/my_app/quote_feed/telemetry.ex # Telemetry — event emission + docs
```
**Example:**
```elixir
defmodule MyApp.QuoteFeed.Telemetry do
@moduledoc """
Telemetry events for QuoteFeed.
## Events
* `[:my_app, :quote_feed, :connected]` — WebSocket connected.
Measurements: `%{system_time: integer()}`.
Metadata: `%{url: String.t()}`.
* `[:my_app, :quote_feed, :tick, :received]` — Price tick received.
Measurements: `%{latency_ms: integer()}`.
Metadata: `%{symbol: String.t(), price: Decimal.t()}`.
"""
@doc "Emit a connection event."
def connected(url) do
:telemetry.execute(
[:my_app, :quote_feed, :connected],
%{system_time: System.system_time()},
%{url: url}
)
end
@doc "Emit a tick received event."
def tick_received(symbol, price, latency_ms) do
:telemetry.execute(
[:my_app, :quote_feed, :tick, :received],
%{latency_ms: latency_ms},
%{symbol: symbol, price: price}
)
end
end
```
**Caller side:**
```elixir
defmodule MyApp.QuoteFeed do
alias MyApp.QuoteFeed.Telemetry
def handle_info({:connected, url}, state) do
Telemetry.connected(url)
{:noreply, %{state | connected: true}}
end
end
```
**Why this over inline:**
- **Discoverability:** One place lists all events a component emits. Handlers/dashboards have a single source of truth.
- **Cohesion:** GenServer focuses on state management. Telemetry module focuses on observability contracts.
- **Documentation:** `@moduledoc` on the telemetry module becomes the event catalog. No hunting through callbacks.
- **Testability:** You can assert on telemetry calls without coupling to GenServer internals.
**When NOT to use:** A module with exactly one telemetry call (e.g., a simple function wrapping a single `:telemetry.execute`). Inline is fine there — the dedicated module adds ceremony without benefit.
---
## Pattern 2: Event Naming Convention
**What it does:** Event names follow a hierarchical path: `[app, context, noun, verb_past_tense]`.
**Examples:**
```elixir
[:gargoyle, :engine, :aggregation, :group, :completed]
[:gargoyle, :market_data, :quote_feed, :connected]
[:gargoyle, :daily_pnl, :snapshot, :backfilled]
```
**Rules:**
- App prefix first (namespace isolation)
- Bounded context second (matches code structure)
- Noun before verb (what happened to what)
- Past tense verbs (events are facts — they already happened)
- Never use generic names like `[:my_app, :event]` or `[:my_app, :metric]`
**Why:** Consistent naming lets you attach handlers by prefix (`[:gargoyle, :engine | _]`) and build dashboards without memorizing arbitrary event names.
---
## Pattern 3: Measurements vs Metadata Separation
**What it does:** Measurements are numeric values you aggregate (sum, avg, p99). Metadata is context for filtering/grouping.
```elixir
# Good: clear separation
:telemetry.execute(
[:my_app, :request, :completed],
%{duration_ms: 42, response_bytes: 1024}, # measurements: numbers
%{method: :get, path: "/api/v1/users", status: 200} # metadata: dimensions
)
# Bad: mixing concerns
:telemetry.execute(
[:my_app, :request, :completed],
%{duration_ms: 42, method: :get}, # method isn't a measurement
%{response_bytes: 1024, path: "/api"} # bytes should be a measurement
)
```
**Rule of thumb:** If you'd put it on a Y-axis in a graph → measurement. If you'd use it as a filter/group-by → metadata.
+119
View File
@@ -2,6 +2,29 @@
Patterns extracted from the Elixir standard library source code — how the core team writes and organizes tests.
## Contents
1. [Module-Level Async Declaration](#1-module-level-async-declaration)
2. [Parameterized Tests](#2-parameterized-tests)
3. [Setup with `start_supervised/2`](#3-setup-with-start_supervised2)
4. [Named Setup Functions (Composable Pipelines)](#4-named-setup-functions-composable-pipelines)
5. [`on_exit` for Reversing Global Side Effects](#5-on_exit-for-reversing-global-side-effects)
6. [Pattern Match Assertions](#6-pattern-match-assertions)
7. [`assert_receive` / `refute_receive` for Process Communication](#7-assert_receive--refute_receive-for-process-communication)
8. [Testing GenServers via Public API (No Internal State Inspection)](#8-testing-genservers-via-public-api-no-internal-state-inspection)
9. [`catch_exit` for Testing Process Failures](#9-catch_exit-for-testing-process-failures)
10. [`@tag capture_log: true` for Suppressing Expected Log Output](#10-tag-capture_log-true-for-suppressing-expected-log-output)
11. [`capture_log` / `capture_io` for Content Assertions](#11-capture_log--capture_io-for-content-assertions)
12. [`describe` Blocks for Logical Grouping](#12-describe-blocks-for-logical-grouping)
13. [`ExUnit.CaseTemplate` for Shared Test Infrastructure](#13-exunitcasetemplate-for-shared-test-infrastructure)
14. [`doctest` Integration](#14-doctest-integration)
15. [`Process.sleep(:infinity)` as a Process Parking Pattern](#15-processsleepinfinity-as-a-process-parking-pattern)
16. [Helper Functions for Test-Specific Behavior](#16-helper-functions-for-test-specific-behavior)
17. [`@tag :tmp_dir` for Filesystem Tests](#17-tag-tmp_dir-for-filesystem-tests)
18. [`assert_raise` with Message Matching](#18-assert_raise-with-message-matching)
19. [`@moduletag` / `@describetag` for Cross-Cutting Configuration](#19-moduletag--describetag-for-cross-cutting-configuration)
20. [Context Pattern Matching in Test Signatures](#20-context-pattern-matching-in-test-signatures)
---
## 1. Module-Level Async Declaration
@@ -1725,4 +1748,100 @@ end
**Why:** Context destructuring signals "this test depends on external setup." If the test is self-contained, the pattern match is misleading — readers will look for setup that doesn't exist or isn't needed.
## Decision Tree
- If you are creating a new test module and need to decide on concurrency → [Module-Level Async Declaration](#1-module-level-async-declaration)
- If the same logic must work across multiple configurations or backends → [Parameterized Tests](#2-parameterized-tests)
- If your test needs a running process with guaranteed cleanup → [Setup with `start_supervised/2`](#3-setup-with-start_supervised2)
- If setup has multiple independent steps that different describe blocks reuse → [Named Setup Functions](#4-named-setup-functions-composable-pipelines)
- If your test modifies global state that must be restored regardless of outcome → [`on_exit` for Reversing Global Side Effects](#5-on_exit-for-reversing-global-side-effects)
- If you care about the shape/structure of a result but not every field → [Pattern Match Assertions](#6-pattern-match-assertions)
- If you need to test asynchronous message delivery between processes → [`assert_receive` / `refute_receive`](#7-assert_receive--refute_receive-for-process-communication)
- If you are testing a GenServer and want tests that survive refactoring → [Testing GenServers via Public API](#8-testing-genservers-via-public-api-no-internal-state-inspection)
- If you need to assert on OTP exit signals (timeouts, noproc, shutdown) → [`catch_exit`](#9-catch_exit-for-testing-process-failures)
- If tests intentionally trigger error paths that produce noisy log output → [`@tag capture_log: true`](#10-tag-capture_log-true-for-suppressing-expected-log-output)
- If you need to verify specific log or IO content was emitted → [`capture_log` / `capture_io`](#11-capture_log--capture_io-for-content-assertions)
- If a module tests multiple public functions and needs logical organization → [`describe` Blocks](#12-describe-blocks-for-logical-grouping)
- If multiple test modules share the same setup/teardown infrastructure → [`ExUnit.CaseTemplate`](#13-exunitcasetemplate-for-shared-test-infrastructure)
- If your module has `iex>` examples that should be verified automatically → [`doctest` Integration](#14-doctest-integration)
- If you need an inert process that exists only to be observed or killed → [`Process.sleep(:infinity)`](#15-processsleepinfinity-as-a-process-parking-pattern)
- If the same 3-5 line test pattern repeats across multiple tests → [Helper Functions](#16-helper-functions-for-test-specific-behavior)
- If tests create or modify files and need filesystem isolation → [`@tag :tmp_dir`](#17-tag-tmp_dir-for-filesystem-tests)
- If you need to verify both the exception type and the user-facing message → [`assert_raise` with Message Matching](#18-assert_raise-with-message-matching)
- If tests only run on certain platforms or you want to filter subsets → [`@moduletag` / `@describetag`](#19-moduletag--describetag-for-cross-cutting-configuration)
- If you want to make test dependencies on setup context explicit → [Context Pattern Matching](#20-context-pattern-matching-in-test-signatures)
<!-- PATTERN_COMPLETE -->
## 21. Filtering Events in Async Tests
**Source:** Gargoyle PR #710 (flaky telemetry test fix)
**What it does:** Pin-matches on a unique identifier to filter events from concurrent tests.
**Why:** When testing telemetry, pub/sub, or any broadcast mechanism with `async: true`, events from other tests can leak into your mailbox. Without filtering, tests pass in isolation but fail randomly when run in parallel.
**Pattern:**
Wrong — receives events from other tests:
```elixir
test "emits telemetry on create", %{user: user} do
:telemetry.attach("test", [:user, :created], &send_to_test/4, self())
create_post(user)
assert_receive {:telemetry, %{user_id: uid}}
assert uid == user.id # Might match event from another test!
end
```
Right — pin filters to only your test's events:
```elixir
test "emits telemetry on create", %{user: user} do
test_pid = self()
expected_uid = user.id
:telemetry.attach("test-#{inspect(test_pid)}", [:user, :created], fn _, _, meta, pid ->
if meta.user_id == expected_uid, do: send(pid, {:telemetry, meta})
end, test_pid)
create_post(user)
assert_receive {:telemetry, %{user_id: ^expected_uid}}
end
```
### Key Insight
The fix happens in two places:
1. **Filter at the source** — only send messages that match your test's unique identifier
2. **Pin in the assertion** — use `^variable` to reject mismatches that slip through
### When to Use
**Triggers:**
- Telemetry handlers (filter by user_id, request_id, or test pid)
- Phoenix.PubSub subscriptions (use unique topic per test)
- GenStage/Broadway consumers (tag events with test pid)
- Any shared message bus in async tests
- Tests pass alone but fail with `--seed` or in CI
### When NOT to Use
**Don't use this when:**
- Tests run with `async: false` (no concurrent tests to leak events)
- You control the event source and can make it test-aware by design
- The event contains a natural unique key you already have (just pin on it)
**Over-application example:**
```elixir
# Overkill when you already have a unique key
test "user update", %{user: user} do
# user.id is already unique — just pin it directly
assert_receive {:updated, %{id: ^user.id}}
end
```
### Decision Tree Addition
- If async tests randomly fail due to events from other tests → [Filtering Events in Async Tests](#21-filtering-events-in-async-tests)
<!-- PATTERN_COMPLETE -->
+723
View File
@@ -0,0 +1,723 @@
# Ecto Type Patterns
Patterns extracted from Ecto's type system source code.
## Contents
1. [`use Ecto.Type` — The Four-Callback Custom Type](#1-use-ectotype--the-four-callback-custom-type)
2. [`embed_as/1` — Controlling Embedded Serialization](#2-embed_as1--controlling-embedded-serialization)
3. [`equal?/2` — Custom Equality for Change Detection](#3-equal2--custom-equality-for-change-detection)
4. [`Ecto.Enum` — Constrained Atom Fields](#4-ectoenum--constrained-atom-fields)
5. [`Ecto.ParameterizedType` — Types with Options](#5-ectoparameterizedtype--types-with-options)
6. [Schemaless Types — `{data, types}` Changesets](#6-schemaless-types--data-types-changesets)
---
## 1. `use Ecto.Type` — The Four-Callback Custom Type
**Source:** [lib/ecto/type.ex#L57-L89](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/type.ex#L57)
The canonical custom type implements exactly four callbacks. Each has a distinct role in the data flow:
- `type/0` — declares the underlying database column type (`:string`, `:map`, `:integer`, etc.)
- `cast/1` — converts external or user-supplied values into your Elixir type; called by `Ecto.Changeset.cast/4`
- `load/1` — converts a raw database value into your Elixir type; called when reading rows
- `dump/1` — converts your Elixir type into a database-compatible value; called when writing
The state diagram is: external --[cast]--> internal --[dump]--> database --[load]--> internal
```elixir
defmodule EctoURI do
use Ecto.Type
def type, do: :map
def cast(uri) when is_binary(uri), do: {:ok, URI.parse(uri)}
def cast(%URI{} = uri), do: {:ok, uri}
def cast(_), do: :error
def load(data) when is_map(data) do
data = for {key, val} <- data, do: {String.to_existing_atom(key), val}
{:ok, struct!(URI, data)}
end
def dump(%URI{} = uri), do: {:ok, Map.from_struct(uri)}
def dump(_), do: :error
end
```
**Why:** Ecto calls each callback at a different point in the lifecycle. Without `cast/1`, user input passes through as raw strings. Without `load/1`, the value comes out of the DB as a plain map. Without `dump/1`, Ecto cannot serialize the struct for storage. Implementing all four closes every gap in the round-trip.
**Anti-pattern:** Relying on the database to store your Elixir struct as-is, or using `:map` with manual pre/post processing scattered across contexts:
```elixir
# BAD — manual conversion at every call site instead of centralizing in a type
%Post{} = Repo.get(Post, id)
uri = post.url |> Map.from_struct() |> then(&URI.parse/1) # repeated everywhere
Repo.update(Post.changeset(post, %{url: Map.from_struct(new_uri)}))
```
### When to Use
**Triggers:**
- You want to store an Elixir struct (URI, Decimal, custom struct) in a single database column
- Cast/load/dump logic would otherwise be duplicated across changesets and contexts
- You want Ecto changesets to accept raw user input and produce the correct Elixir type automatically
**Example — before:**
```elixir
# Manual conversion in every changeset
def changeset(post, params) do
post
|> cast(params, [:url])
|> validate_change(:url, fn :url, val ->
case URI.parse(val) do
%URI{host: nil} -> [url: "must be a valid URI"]
_ -> []
end
end)
end
# Manual conversion when reading
def get_post(id) do
post = Repo.get!(Post, id)
%{post | url: struct!(URI, post.url)}
end
```
**Example — after:**
```elixir
# schema declaration — one line
field :url, EctoURI
# changeset just casts; EctoURI.cast/1 validates shape
def changeset(post, params), do: cast(post, params, [:url])
# reading returns %URI{} automatically via EctoURI.load/1
def get_post(id), do: Repo.get!(Post, id)
```
### When NOT to Use
**Don't use this when:**
- The transformation is trivial (storing a plain string that needs no parsing)
- The field contains highly variable or polymorphic data better handled by embedded schemas
- You need association semantics (use Ecto associations, not custom types)
**Over-application example:**
```elixir
# Overkill — a custom type just to upcase strings
defmodule UppercaseString do
use Ecto.Type
def type, do: :string
def cast(v) when is_binary(v), do: {:ok, String.upcase(v)}
def cast(_), do: :error
def load(v), do: {:ok, v}
def dump(v), do: {:ok, v}
end
```
**Better alternative:**
```elixir
# Changeset validation handles this cleanly without a new type module
def changeset(record, params) do
record
|> cast(params, [:code])
|> update_change(:code, &String.upcase/1)
end
```
**Why:** Custom types are for encapsulating a serialization contract that must be enforced consistently at every read and write. For one-off transformations or validations, changeset functions are simpler and more obvious to future readers.
---
## 2. `embed_as/1` — Controlling Embedded Serialization
**Source:** [lib/ecto/type.ex#L104-L109](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/type.ex#L104)
When a custom type is used inside an `embeds_one` or `embeds_many` field, Ecto calls `embed_as/1` to decide whether to pass the value through `dump/1` or treat it as its own serialized form. The callback receives the embed format (`:json` by default) and returns either `:self` or `:dump`.
- `:self` — the in-memory value is used as-is without calling `dump/1`; appropriate when the runtime representation is already JSON-compatible (scalars, plain maps)
- `:dump``dump/1` is called to serialize the value before encoding; needed when the runtime representation (e.g., a struct) is not directly JSON-serializable
`use Ecto.Type` provides a default implementation that returns `:self`. Override it to return `:dump` when your type holds an Elixir struct or other value that cannot be directly encoded to JSON.
```elixir
defmodule EctoURI do
use Ecto.Type
# Override: %URI{} is not JSON-serializable, so run dump/1 in embedded contexts
def embed_as(_format), do: :dump
end
```
**Why:** When Ecto builds embedded documents for export (e.g., storing a JSON blob), it needs to know whether to trust the in-memory value or to re-serialize it. If your type holds state in an Elixir struct that cannot be stored directly (like `%URI{}`), the default `:self` passes the raw struct to the JSON encoder — which either raises or produces garbage like `%{__struct__: "Elixir.URI", host: ..., ...}`. Returning `:dump` ensures `dump/1` converts it to a clean map first.
**Anti-pattern:** Assuming the default `:self` is correct for a type whose in-memory and storable representations differ, then debugging mysterious embedded schema corruption:
```elixir
# BAD — %URI{} stored as a raw struct map because embed_as/1 was never considered
embeds_one :metadata, Metadata do
field :canonical_url, EctoURI # stored as %{__struct__: "Elixir.URI", host: ..., ...}
end
```
### When to Use
**Triggers:**
- Your custom type is used inside `embeds_one` or `embeds_many` fields
- The Elixir representation and the storable representation differ (struct vs plain map, etc.)
- You observe that embedded fields are persisted with unexpected shapes
**Example — before:**
```elixir
# embed_as/1 not considered; embedded URL stored as raw struct map
defmodule EctoURI do
use Ecto.Type
def type, do: :map
def cast(uri) when is_binary(uri), do: {:ok, URI.parse(uri)}
def cast(%URI{} = uri), do: {:ok, uri}
def cast(_), do: :error
def load(data) when is_map(data) do
data = for {key, val} <- data, do: {String.to_existing_atom(key), val}
{:ok, struct!(URI, data)}
end
def dump(%URI{} = uri), do: {:ok, Map.from_struct(uri)}
def dump(_), do: :error
end
```
**Example — after:**
```elixir
defmodule EctoURI do
use Ecto.Type
def type, do: :map
def cast(uri) when is_binary(uri), do: {:ok, URI.parse(uri)}
def cast(%URI{} = uri), do: {:ok, uri}
def cast(_), do: :error
def load(data) when is_map(data) do
data = for {key, val} <- data, do: {String.to_existing_atom(key), val}
{:ok, struct!(URI, data)}
end
def dump(%URI{} = uri), do: {:ok, Map.from_struct(uri)}
def dump(_), do: :error
# Explicit: ensure dump/1 runs in embedded contexts to produce a clean map
def embed_as(_format), do: :dump
end
```
### When NOT to Use
**Don't use this when:**
- Your type is never used in embedded schemas (the callback has no effect)
- The in-memory and storable representations are the same (plain maps, scalars)
- You intentionally want to skip `dump/1` in embedded contexts (the default `:self` already does this)
**Over-application example:**
```elixir
# Overriding embed_as/1 on a type that stores plain integers
defmodule PriorityLevel do
use Ecto.Type
def type, do: :integer
def cast(v) when is_integer(v) and v in 1..5, do: {:ok, v}
def cast(_), do: :error
def load(v), do: {:ok, v}
def dump(v), do: {:ok, v}
def embed_as(_), do: :self # Pointless — integer is its own storable form
end
```
**Better alternative:**
```elixir
# Default embed_as/1 from `use Ecto.Type` is sufficient for scalar types
defmodule PriorityLevel do
use Ecto.Type
def type, do: :integer
def cast(v) when is_integer(v) and v in 1..5, do: {:ok, v}
def cast(_), do: :error
def load(v), do: {:ok, v}
def dump(v), do: {:ok, v}
end
```
**Why:** `embed_as/1` is only meaningful when there is a gap between what Ecto holds in memory and what must be written to the database. For scalar types where the two representations are identical, the default suffices and adding the override is noise.
---
## 3. `equal?/2` — Custom Equality for Change Detection
**Source:** [lib/ecto/type.ex](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/type.ex)
Ecto calls `equal?/2` to decide whether a field's value has changed before including it in an `UPDATE` statement. The default delegates to `==`. Override when your type's notion of equality is more nuanced than structural term equality.
Common cases: `Decimal` values where `Decimal.equal?/2` differs from `==`, sets or maps where insertion order shouldn't matter, floats with tolerance, URIs where trailing slashes are equivalent.
```elixir
defmodule EctoDecimal do
use Ecto.Type
def type, do: :string
def equal?(%Decimal{} = a, %Decimal{} = b), do: Decimal.equal?(a, b)
def equal?(a, b), do: a == b
end
```
**Why:** `Decimal.new("1.0") == Decimal.new("1.00")` is `false` in Elixir because the structs differ structurally. Without a custom `equal?/2`, Ecto would generate a spurious `UPDATE` every time a Decimal field is loaded and re-saved unchanged. The custom implementation delegates to the type's own equality semantics, preventing unnecessary database writes.
**Anti-pattern:** Allowing Ecto to generate spurious UPDATEs because `==` disagrees with logical equality:
```elixir
# BAD — default equal?/2 uses ==
# Decimal.new("1.0") != Decimal.new("1.00") structurally,
# so every load-and-save cycle marks the field dirty
defmodule EctoDecimal do
use Ecto.Type
def type, do: :string
def cast(v), do: {:ok, Decimal.new(v)}
def load(v), do: {:ok, Decimal.new(v)}
def dump(v), do: {:ok, Decimal.to_string(v)}
# Missing equal?/2 — spurious UPDATEs in production
end
```
### When to Use
**Triggers:**
- Your type wraps a value with non-structural equality (Decimal, Set, custom structs with computed fields)
- You see unexpected `UPDATE` queries in your logs when no meaningful data changed
- The type's own library provides an equality function (e.g., `Decimal.equal?/2`)
**Example — before:**
```elixir
# Tags stored as a sorted list — ["a", "b"] and ["b", "a"] treated as different
defmodule TagList do
use Ecto.Type
def type, do: {:array, :string}
def cast(tags) when is_list(tags), do: {:ok, Enum.sort(tags)}
def cast(_), do: :error
def load(v), do: {:ok, v}
def dump(v), do: {:ok, v}
# Missing equal?/2 — if DB returns unsorted list, it always appears changed
end
```
**Example — after:**
```elixir
defmodule TagList do
use Ecto.Type
def type, do: {:array, :string}
def cast(tags) when is_list(tags), do: {:ok, Enum.sort(tags)}
def cast(_), do: :error
def load(v), do: {:ok, v}
def dump(v), do: {:ok, v}
def equal?(a, b) when is_list(a) and is_list(b) do
Enum.sort(a) == Enum.sort(b)
end
def equal?(a, b), do: a == b
end
```
### When NOT to Use
**Don't use this when:**
- Structural equality (`==`) already matches your type's logical equality (most scalars, plain maps)
- You intentionally want every change in representation to trigger an update (audit fields, version counters)
- The equality logic would require database queries or I/O
**Over-application example:**
```elixir
# Overriding equal?/2 for a plain string type — == is already correct
defmodule TrimmedString do
use Ecto.Type
def type, do: :string
def cast(v) when is_binary(v), do: {:ok, String.trim(v)}
def cast(_), do: :error
def load(v), do: {:ok, v}
def dump(v), do: {:ok, v}
def equal?(a, b), do: String.trim(a) == String.trim(b) # misleading
end
```
**Better alternative:**
```elixir
# cast/1 already normalizes; == on the normalized value is correct
defmodule TrimmedString do
use Ecto.Type
def type, do: :string
def cast(v) when is_binary(v), do: {:ok, String.trim(v)}
def cast(_), do: :error
def load(v), do: {:ok, v}
def dump(v), do: {:ok, v}
# Default equal?/2 is correct — both sides are already trimmed
end
```
**Why:** Once `cast/1` normalizes values to a canonical form, `==` on that canonical form is correct. Overriding `equal?/2` to re-normalize creates two sources of truth for what "equal" means and can hide bugs where values escape normalization.
---
## 4. `Ecto.Enum` — Constrained Atom Fields
**Source:** [lib/ecto/enum.ex](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/enum.ex)
`Ecto.Enum` is Ecto's built-in parameterized type for fields that hold one of a fixed set of values. It stores atoms as strings in the database, validates membership automatically during `cast/4`, and provides a clean schema-level declaration of what values are legal.
```elixir
schema "orders" do
field :status, Ecto.Enum, values: [:pending, :processing, :shipped, :delivered, :cancelled]
end
# Changeset validation is automatic:
# cast will reject values outside the enum
# DB stores as "pending", "processing", etc.
# Can also map atoms to custom DB values:
field :status, Ecto.Enum,
values: [pending: "PENDING", shipped: "SHIPPED"]
```
**Why:** A fixed set of valid values is a domain constraint that belongs at the type level, not scattered across changeset validations. `Ecto.Enum` co-locates the constraint with the field declaration, making it impossible to add a new status without updating the schema. It also avoids the string-vs-atom impedance mismatch: your application code works with atoms, the DB stores strings.
**Anti-pattern:** Using `validate_inclusion` with a hardcoded list when the field is a fixed enum. The valid values now live in two places (schema field type and changeset validation), drift over time, and offer no guarantee that the DB stores a normalized form:
```elixir
# BAD — validation is disconnected from the field type
schema "orders" do
field :status, :string
end
def changeset(order, params) do
order
|> cast(params, [:status])
|> validate_inclusion(:status, ["pending", "processing", "shipped"])
# Atoms vs strings already a smell; easy to add a value in one place but not the other
end
```
### When to Use
**Triggers:**
- A field can only hold one of a fixed, small set of named values (status, role, priority, state machine states)
- You want `cast/4` to reject invalid values without writing a custom validator
- You want the valid values to be introspectable at runtime via `Ecto.Enum.values/2`
**Example — before:**
```elixir
schema "tickets" do
field :priority, :string
end
@valid_priorities ~w(low medium high critical)
def changeset(ticket, params) do
ticket
|> cast(params, [:priority])
|> validate_required([:priority])
|> validate_inclusion(:priority, @valid_priorities)
end
```
**Example — after:**
```elixir
schema "tickets" do
field :priority, Ecto.Enum, values: [:low, :medium, :high, :critical]
end
def changeset(ticket, params) do
ticket
|> cast(params, [:priority])
|> validate_required([:priority])
# validate_inclusion is implicit — cast rejects invalid values
end
```
### When NOT to Use
**Don't use this when:**
- The set of valid values is dynamic (loaded from the database, configurable at runtime)
- You need rich metadata per value (labels, descriptions, ordering weights) — a separate table or config map is better
- The field is an open-ended string that happens to have common values (tags, categories that grow)
**Over-application example:**
```elixir
# Enum for country codes that may expand and need localization
field :country, Ecto.Enum, values: [:us, :gb, :de, :fr, :jp]
# Adding a new country requires a schema migration AND a code change
```
**Better alternative:**
```elixir
# Reference table with a foreign key
field :country_code, :string
# Validated against a countries table at the application layer
```
**Why:** `Ecto.Enum` hard-codes valid values into the schema module. When the list is stable and small (status machines, role levels), that is exactly right. When the list is user-managed or requires non-code changes to extend, a reference table decouples the constraint from deployments.
---
## 5. `Ecto.ParameterizedType` — Types with Options
**Source:** [lib/ecto/parameterized_type.ex](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/parameterized_type.ex)
When a custom type needs configuration options set at field definition time (like `Ecto.Enum`'s `values:` option), implement `Ecto.ParameterizedType` instead of `Ecto.Type`. The `init/1` callback receives field options at compile/load time and returns a `params` term that is threaded through every other callback at runtime.
The five callbacks: `init/1`, `type/1`, `cast/2`, `load/3`, `dump/3`.
```elixir
defmodule MyApp.StatusType do
use Ecto.ParameterizedType
def init(opts) do
validate = Keyword.fetch!(opts, :validate)
%{validate: validate}
end
def type(_params), do: :string
def cast(value, %{validate: validator}) do
if validator.(value), do: {:ok, value}, else: :error
end
def load(value, _loader, _params), do: {:ok, value}
def dump(value, _dumper, _params), do: {:ok, value}
end
# Usage:
field :status, MyApp.StatusType, validate: &(&1 in ~w(a b c))
```
**Why:** `Ecto.Type` callbacks receive no configuration — the type module is a singleton with fixed behavior. `Ecto.ParameterizedType` solves the case where the same type module should behave differently per field (different valid values, different validation rules, different storage formats). `Ecto.Enum` itself is implemented as a parameterized type so the same module handles every `values:` list.
**Anti-pattern:** Defining a new type module for every variation of behavior that differs only in configuration:
```elixir
# BAD — combinatorial explosion of modules
defmodule StatusType.V1 do
use Ecto.Type
@valid ~w(draft published)
def cast(v) when v in @valid, do: {:ok, v}
def cast(_), do: :error
# ...
end
defmodule StatusType.V2 do
use Ecto.Type
@valid ~w(open in_progress closed)
# identical code, different @valid
end
```
### When to Use
**Triggers:**
- The same type logic should apply to multiple fields but with different configuration per field
- You are building a reusable library type that schema authors customize via options
- The configuration is known at schema definition time (compile-time or application startup)
**Example — before:**
```elixir
# A separate module per enum — doesn't scale
defmodule OrderStatus do
use Ecto.Type
def type, do: :string
def cast(v) when v in ~w(pending shipped), do: {:ok, v}
def cast(_), do: :error
def load(v), do: {:ok, v}
def dump(v), do: {:ok, v}
end
defmodule TicketPriority do
use Ecto.Type
def type, do: :string
def cast(v) when v in ~w(low high), do: {:ok, v}
def cast(_), do: :error
def load(v), do: {:ok, v}
def dump(v), do: {:ok, v}
end
```
**Example — after:**
```elixir
defmodule ConstrainedString do
use Ecto.ParameterizedType
def init(opts), do: %{values: Keyword.fetch!(opts, :values)}
def type(_params), do: :string
def cast(value, %{values: values}) when value in values, do: {:ok, value}
def cast(_, _), do: :error
def load(value, _, _), do: {:ok, value}
def dump(value, _, _), do: {:ok, value}
end
# In schemas:
field :status, ConstrainedString, values: ~w(pending shipped)
field :priority, ConstrainedString, values: ~w(low high)
```
### When NOT to Use
**Don't use this when:**
- The type has no configuration — use `Ecto.Type`, which has simpler callback signatures
- The configuration changes at runtime rather than at schema definition time (use `cast/1` with runtime state instead)
- You only need this for one field in one schema (inline the logic in a changeset validator)
**Over-application example:**
```elixir
# Parameterized type for a type that has no real options
defmodule MaybeString do
use Ecto.ParameterizedType
def init(_opts), do: %{} # no options used
def type(_), do: :string
def cast(v, _) when is_binary(v), do: {:ok, v}
def cast(_, _), do: :error
def load(v, _, _), do: {:ok, v}
def dump(v, _, _), do: {:ok, v}
end
```
**Better alternative:**
```elixir
# No configuration needed — plain Ecto.Type is simpler
defmodule MaybeString do
use Ecto.Type
def type, do: :string
def cast(v) when is_binary(v), do: {:ok, v}
def cast(_), do: :error
def load(v), do: {:ok, v}
def dump(v), do: {:ok, v}
end
```
**Why:** `Ecto.ParameterizedType` adds arity to every callback (`params` argument) and requires an `init/1` that must be implemented. This complexity is justified when you need per-field configuration. Without actual options, the extra arity is noise that obscures intent.
---
## 6. Schemaless Types — `{data, types}` Changesets
**Source:** [lib/ecto/changeset.ex](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/changeset.ex) (documentation for `cast/4`)
When you need to validate and cast data without defining a full schema module, pass a `{data, types}` tuple as the first argument to `Ecto.Changeset.cast/4`. The `data` map holds the current values (often `%{}`), and the `types` map specifies field names and their Ecto types. The full changeset API works normally — `validate_required`, `validate_number`, `apply_action`, etc.
```elixir
def validate_params(params) do
types = %{name: :string, age: :integer, role: :string}
{%{}, types}
|> Ecto.Changeset.cast(params, Map.keys(types))
|> Ecto.Changeset.validate_required([:name])
|> Ecto.Changeset.validate_number(:age, greater_than: 0)
|> Ecto.Changeset.apply_action(:insert)
end
```
**Why:** Defining a `use Ecto.Schema` module for a one-off params map is heavyweight: it introduces a new module, struct, and migration concern for data that never touches the database. The `{data, types}` tuple gives you Ecto's casting and validation pipeline — type coercion, error accumulation, `apply_action` — for ephemeral, transient, or API-boundary data structures.
**Anti-pattern:** Defining a full schema module solely to validate a one-off params map:
```elixir
# BAD — a schema with no table, used only for one validation function
defmodule SearchParams do
use Ecto.Schema
import Ecto.Changeset
@primary_key false
embedded_schema do
field :query, :string
field :page, :integer
field :per_page, :integer
end
def changeset(params) do
%SearchParams{}
|> cast(params, [:query, :page, :per_page])
|> validate_required([:query])
end
end
```
### When to Use
**Triggers:**
- Validating and casting controller params, webhook payloads, or CLI arguments that are never persisted
- Building a multi-step form or wizard where intermediate steps don't map to a database row
- Writing a context function that accepts a params map and needs to return normalized data or errors
**Example — before:**
```elixir
# Full schema module for transient data
defmodule ReportFilter do
use Ecto.Schema
import Ecto.Changeset
@primary_key false
embedded_schema do
field :start_date, :date
field :end_date, :date
field :user_id, :integer
end
def changeset(params) do
%ReportFilter{}
|> cast(params, [:start_date, :end_date, :user_id])
|> validate_required([:start_date, :end_date])
end
end
```
**Example — after:**
```elixir
def parse_report_filter(params) do
types = %{start_date: :date, end_date: :date, user_id: :integer}
{%{}, types}
|> Ecto.Changeset.cast(params, Map.keys(types))
|> Ecto.Changeset.validate_required([:start_date, :end_date])
|> Ecto.Changeset.apply_action(:validate)
end
```
### When NOT to Use
**Don't use this when:**
- The data structure is reused across many contexts (define a proper schema or embedded schema)
- You need associations, `autogenerate`, or `timestamps()` (requires a full schema)
- The validated data will eventually be persisted (start with a schema to avoid a rewrite)
**Over-application example:**
```elixir
# Schemaless changeset for data that has a natural schema home
def update_user_profile(user_id, params) do
types = %{name: :string, email: :string, bio: :string}
{%{}, types}
|> Ecto.Changeset.cast(params, Map.keys(types))
|> Ecto.Changeset.validate_required([:name, :email])
|> Ecto.Changeset.apply_action(:update)
# Then manually build an update query — loses Ecto.Repo integration
end
```
**Better alternative:**
```elixir
# User already has a schema — use it
def update_user_profile(user_id, params) do
Repo.get!(User, user_id)
|> User.profile_changeset(params)
|> Repo.update()
end
```
**Why:** The `{data, types}` approach trades schema infrastructure for simplicity. That trade is worth it for truly transient data (search filters, report parameters, one-time import validation). For persistent data, the schema module is not overhead — it is the source of truth for what the table contains.
---
## Decision Tree
- If storing a fixed set of values → `Ecto.Enum` with `values:`
- If wrapping an Elixir struct (URI, Decimal, etc.) in a field → `use Ecto.Type` with all four callbacks
- If your type needs configuration options at the field level → `use Ecto.ParameterizedType`
- If two values of your type may be logically equal without `==` → override `equal?/2`
- If your type is used inside `embeds_one`/`embeds_many` and has a non-trivial serialized form → verify `embed_as/1`
- If you need to validate params without a schema → schemaless `{data, types}` changeset
<!-- PATTERN_COMPLETE -->
+26
View File
@@ -2,6 +2,19 @@
Patterns extracted from the Elixir standard library source code.
## Contents
1. [Public Type with @typedoc](#1-public-type-with-typedoc)
2. [Private Types with @typep](#2-private-types-with-typep)
3. [@opaque Types (Protocol t())](#3-opaque-types-protocol-t)
4. [Union Types in @spec Return Values](#4-union-types-in-spec-return-values)
5. [`when` Constraints in Specs](#5-when-constraints-in-specs)
6. [Map Types with required/optional Keys](#6-map-types-with-requiredoptional-keys)
7. [Keyword List Types for Options](#7-keyword-list-types-for-options)
8. [Parameterized Types (t/1)](#8-parameterized-types-t1)
9. [Named Parameters in Specs (:: annotation)](#9-named-parameters-in-specs--annotation)
10. [@typedoc since: Annotation](#10-typedoc-since-annotation)
---
## 1. Public Type with @typedoc
@@ -798,4 +811,17 @@ end
**Why:** `since:` annotations are for library consumers checking compatibility across versions. Application code doesn't have "consumers" checking which version introduced a type — it's all deployed together.
## Decision Tree
- If you are defining a public `@type` that appears in any `@spec` or callback → [Public Type with @typedoc](#1-public-type-with-typedoc)
- If a type is used only internally for recursion or DRYing up repeated expressions → [Private Types with @typep](#2-private-types-with-typep)
- If you want to hide internal representation and force consumers to use accessor functions → [@opaque Types](#3-opaque-types-protocol-t)
- If a function can return multiple distinct shapes (tagged tuples, atoms) → [Union Types in @spec Return Values](#4-union-types-in-spec-return-values)
- If the return type depends on the input type (generic/polymorphic function) → [`when` Constraints in Specs](#5-when-constraints-in-specs)
- If you accept a map with a mix of mandatory and optional keys → [Map Types with required/optional Keys](#6-map-types-with-requiredoptional-keys)
- If a function accepts a keyword list of options and you want to document valid keys → [Keyword List Types for Options](#7-keyword-list-types-for-options)
- If you define a container type and want specs to express what element type is inside → [Parameterized Types (t/1)](#8-parameterized-types-t1)
- If a parameter's type alone does not convey its purpose → [Named Parameters in Specs](#9-named-parameters-in-specs--annotation)
- If you are adding a new public type to an existing library post-1.0 → [@typedoc since: Annotation](#10-typedoc-since-annotation)
<!-- PATTERN_COMPLETE -->
+4 -4
View File
@@ -84,7 +84,7 @@ end)
**What they avoid:** Tests that depend on or modify global state without cleanup.
**Source evidence:** `lib/mix/test/test_helper.exs:98-113` — MixTest.Case restores ALL global state in `on_exit`:
**Source evidence:** `lib/mix/test/test_helper.exs:99-115` — MixTest.Case restores ALL global state in `on_exit`:
- `Mix.env(:dev)`, `Mix.target(:host)`, `Mix.Task.clear()`, `Mix.Shell.Process.flush()`
- Unloads all applications that were loaded during the test
@@ -587,12 +587,12 @@ end
**What they avoid:** ETS tables, registered names, or application env used across tests without isolation.
**Source evidence:** `lib/elixir/test/elixir/registry_test.exs:28-31` — Each test gets a uniquely-named Registry:
**Source evidence:** `lib/elixir/test/elixir/registry_test.exs:29-32` — Each test gets a uniquely-named Registry:
```elixir
name = :"#{config.test}_#{partitions}_#{inspect(keys)}"
```
`lib/elixir/test/elixir/gen_server_test.exs:166` — Uses `%{test: name}` for unique process registration.
`lib/elixir/test/elixir/gen_server_test.exs:164` — Uses `%{test: name}` for unique process registration.
**Why it's bad:** Tests that share state can't run concurrently. They're order-dependent and fragile.
@@ -880,7 +880,7 @@ end
**What they avoid:** Converting untrusted strings to atoms.
**Source evidence:** `lib/elixir/lib/option_parser.ex:855` — Uses `String.to_existing_atom/1` with the `:switches` allowlist pattern. The only `String.to_atom/1` calls in library code are in compiler/macro contexts where the set is bounded.
**Source evidence:** `lib/elixir/lib/option_parser.ex:859` — Uses `String.to_existing_atom/1` with the `:switches` allowlist pattern. The only `String.to_atom/1` calls in library code are in compiler/macro contexts where the set is bounded.
**Why it's bad:** Atoms are never garbage collected. User-controlled atom creation is a denial-of-service vector (1,048,576 atom limit by default).
+28 -6
View File
@@ -91,7 +91,7 @@ setup do
end
```
**Source:** `lib/ex_unit/lib/ex_unit/callbacks.ex:277-340``start_supervised` is designed specifically for this: guaranteed shutdown in reverse order, no leaked processes, no race conditions.
**Source:** `lib/ex_unit/lib/ex_unit/callbacks.ex:520-568``start_supervised` is designed specifically for this: guaranteed shutdown in reverse order, no leaked processes, no race conditions.
### When to Apply This Rule
@@ -236,7 +236,7 @@ test "starts a server", %{test: test_name} do
end
```
**Source:** `lib/elixir/test/elixir/registry_test.exs:28``name = :"#{config.test}_#{partitions}_#{inspect(keys)}"` — always derives unique names from test context.
**Source:** `lib/elixir/test/elixir/registry_test.exs:29``name = :"#{config.test}_#{partitions}_#{inspect(keys)}"` — always derives unique names from test context.
### When to Apply This Rule
@@ -401,7 +401,7 @@ setup do
end
```
**Source:** `lib/logger/test/logger_test.exs:12-17` — Every Logger config change has a corresponding `on_exit` restoration. `lib/logger/test/test_helper.exs:57-62``capture_log` uses `after` to always restore level.
**Source:** `lib/logger/test/logger_test.exs:12-17` — Every Logger config change has a corresponding `on_exit` restoration. `lib/logger/test/test_helper.exs:57-65``capture_log` uses `after` to always restore level.
### When to Apply This Rule
@@ -474,7 +474,7 @@ describe "admin users - deletion" do
end
```
**Source:** `lib/ex_unit/lib/ex_unit/callbacks.ex:423-425``no_describe!` check prevents nesting.
**Source:** `lib/ex_unit/lib/ex_unit/callbacks.ex:433-437``no_describe!` check prevents nesting.
### When to Apply This Rule
@@ -628,7 +628,7 @@ test "handles process crash" do
end
```
**Source:** `lib/elixir/test/elixir/task_test.exs:297,305,315,330` — Every test that expects a linked process to crash sets `:trap_exit` first.
**Source:** `lib/elixir/test/elixir/task_test.exs:300,308,316,327` — Every test that expects a linked process to crash sets `:trap_exit` first.
### When to Apply This Rule
@@ -866,7 +866,7 @@ test "process stops" do
end
```
**Source:** `lib/elixir/test/elixir/supervisor_test.exs:278-285``assert_kill` helper always uses monitor + assert_receive, never `Process.alive?` polling.
**Source:** `lib/elixir/test/elixir/supervisor_test.exs:289-293``assert_kill` helper always uses monitor + assert_receive, never `Process.alive?` polling.
### When to Apply This Rule
@@ -1160,3 +1160,25 @@ end
**Why it's OK here:** There's no conditional logic — every iteration tests the exact same behavior with a trivially predictable result. If one fails, the assertion message includes the specific value. The loop is purely for conciseness.
<!-- PATTERN_COMPLETE -->
---
## Logger.warn/2 is Deprecated
**Smell:** Using `Logger.warn/2` instead of `Logger.warning/2`
**Why it's wrong:** `Logger.warn/2` was deprecated in Elixir 1.11. The standard function is `Logger.warning/2`.
**Example — broken:**
```elixir
Logger.warn("Connection lost: #{reason}")
```
**Example — fixed:**
```elixir
Logger.warning("Connection lost: #{reason}")
```
**Source:** [Elixir 1.11 Changelog](https://hexdocs.pm/elixir/1.11/changelog.html#logger-improvements)
<!-- PATTERN_COMPLETE -->
+15
View File
@@ -0,0 +1,15 @@
# Sources
Reference material extracted from specific projects. Study for ideas, don't copy blindly.
These are **descriptive** — they document what a project does and why.
The `patterns/` directory is **prescriptive** — it tells you what to do.
Patterns that prove broadly applicable get promoted from here into `patterns/`.
The rest stays as reference for understanding how mature projects solve specific problems.
## Files
- `elixir-lang.md` — conventions from the elixir-lang/elixir source
- `elixir-lang-analysis.md` — deeper analysis of elixir-lang source architecture
- `oban.md` — patterns from oban-bg/oban (job processing, plugin system)
+303
View File
@@ -0,0 +1,303 @@
# Elixir Language Source: Architectural Conventions
How does José Valim and the Elixir core team build Elixir itself?
What does the language source reveal about conventions that aren't
documented anywhere else?
**Repo:** [elixir-lang/elixir](https://github.com/elixir-lang/elixir)
---
## 1. Repo Shape
| Metric | Value |
|--------|-------|
| Size | 92M |
| Source files | 567 .ex/.exs |
| Erlang bootstrap | 33 .erl files |
| Commits | 22,032 |
| Contributors | 1,578 |
| Test files | 208 |
| Production files | 248 |
| Test ratio | 1:1.2 |
| TODOs (non-test) | 127 (all version-gated) |
### Organizational Philosophy
```
lib/
├── elixir/ # The language core (compiler + stdlib)
│ ├── src/ # 33 Erlang files (bootstrap)
│ └── lib/ # Elixir stdlib + compiler
├── eex/ # Templating (independent OTP app)
├── ex_unit/ # Testing framework (independent OTP app)
├── iex/ # Interactive shell (independent OTP app)
├── logger/ # Logging (independent OTP app)
└── mix/ # Build tool (independent OTP app)
```
Each component is a separate OTP application. They could theoretically
be released independently. This is Elixir eating its own dog food —
the umbrella project convention that Phoenix apps use comes directly
from how the language itself is organized.
---
## 2. What the Codebase Values
### By size (what gets the most lines)
| Module | Lines | Role |
|--------|-------|------|
| `Kernel` | 7,102 | The implicit language surface |
| `Module.Types.Descr` | 6,301 | Set-theoretic type descriptions |
| `Enum` | 5,242 | Collection operations |
| `String` | 3,263 | First-class string concept |
| `Macro` | 3,102 | Metaprogramming foundation |
| `Exception` | 2,720 | Error taxonomy |
| `Code.Formatter` | 2,605 | Code formatting as library |
**The surprise:** The type system (`types/descr.ex` at 6,301 lines) is
nearly as large as Kernel (7,102 lines). It's the newest and
fastest-growing module — 504 commits, 96% written by José Valim. This
is where the investment is going.
### By authorship (who shapes the language)
Type system: 396/504 commits from José, 32 from Eric Meadows-Jönsson,
31 from Guillaume Duboc. This is auteur-driven development — one person
holds the architectural vision for the most complex subsystem.
---
## 3. The Bootstrap Problem
**How does Elixir compile itself?**
The answer is 33 Erlang files in `lib/elixir/src/`:
```
elixir_bootstrap.erl — minimal Kernel for self-compilation
elixir_compiler.erl — the compiler entry point
elixir_tokenizer.erl — lexer (in Erlang for speed)
elixir_expand.erl — macro expansion
elixir_erl.erl — Elixir AST → Erlang AST
elixir_erl_pass.erl — code generation pass
elixir_env.erl — compilation environment
elixir_clauses.erl — pattern matching compilation
```
**Convention:** The tokenizer and core compiler remain in Erlang
permanently. This isn't technical debt — it's a deliberate choice.
The tokenizer benefits from Erlang's binary pattern matching
performance. The compiler needs to exist before Elixir does.
**Origin:** The bootstrap file dates to Nov 22, 2013 (commit
`260be7c8e`: "Start porting elixir_macros to pure elixir"). Before
this, MORE of the compiler was in Erlang. The trajectory is clear:
minimize Erlang over time, but keep it where it provides genuine value.
---
## 4. TODO Culture: Version-Gated Deadlines
```elixir
# TODO: Remove me on v2.0 — 16 occurrences
# TODO: Deprecate me on Elixir v1.23 — 6 occurrences
# TODO: Remove this clause on Elixir v2.0 once single-quoted charlists are removed
# TODO: Make an error on Elixir v2.0 — 3 occurrences
# TODO: Deprecate on Elixir v1.22 — 3 occurrences
```
**Convention:** Every TODO has a version target. No "someday" TODOs
exist. When a version ships, grep for that version's TODOs and resolve
them all.
**127 total TODOs** across 567 files. Contrast with Go's 3,428 TODOs
across 11K files — the Elixir team treats TODOs as time-bombs, not
documentation.
---
## 5. Unique Patterns
### 5.1 Protocol Consolidation
Protocols dispatch dynamically at runtime by default (checking each
struct's implementation). **Protocol consolidation** compiles all known
implementations into a single dispatch module at build time.
From `lib/elixir/lib/protocol.ex`:
> "Consolidation directly links the protocol to its implementations.
> Invoking a consolidated protocol is equivalent to invoking two remote
> functions."
**Convention:** Mix enables consolidation by default in production. The
`@callback __protocol__(:consolidated?)` exists so code can check at
runtime whether fast-path dispatch is active.
**When NOT to use:** Tests often disable consolidation (`consolidate_
protocols: false`) so new protocol implementations added during tests
are discoverable without recompilation.
### 5.2 Parallel Type Checker
`Module.ParallelChecker` (introduced July 2019, PR #9203 by Eric
Meadows-Jönsson as "Add ExCk chunk") enables concurrent type checking
across modules.
The type system itself (13,034 lines across 7 files in
`lib/elixir/lib/module/types/`) is set-theoretic — types are sets, and
operations are set operations (union, intersection, difference).
**Key files:**
- `descr.ex` (6,301 lines) — type descriptions and set operations
- `apply.ex` — function application typing
- `expr.ex` — expression typing
- `pattern.ex` — pattern match typing
- `of.ex` — type inference
- `helpers.ex` — shared utilities
- `traverse.ex` — AST traversal
### 5.3 Code.Formatter as Library Function
The code formatter (2,605 lines) is a library function, not a CLI tool.
You can call `Code.format_string!/2` from any Elixir code.
**Introduced:** Oct 7, 2017 (PR #6639 by José Valim). **Zero review
comments. Merged in 1 hour.** José opened and merged his own formatter
with no external review. This is the BDFL model — the language author
ships foundational infrastructure by authority.
**Convention:** The formatter uses `Inspect.Algebra` (Wadler-Lindig
pretty-printing) for layout decisions. It defines all operators and
their associativity as module attributes:
```elixir
@pipeline_operators [:|>, :~>>, :<<~, :~>, :<~, :<~>, :"<|>"]
@right_new_line_before_binary_operators [:|, :when]
@required_parens_logical_binary_operands [:|||, :||, :or, :&&&, :&&, :and]
```
### 5.4 Mix Tasks as Single-File Modules
55 Mix tasks, each in its own file. Convention:
- One task = one file
- Module name determines task name: `Mix.Tasks.Deps.Clean``deps.clean`
- `@shortdoc` for brief help, `@moduledoc` for full docs
- `@recursive true` for umbrella traversal
### 5.5 ExUnit CaseTemplate (Extension Pattern)
The `ExUnit.CaseTemplate` is how Elixir's test framework supports
extension — you define a module that `use`s `CaseTemplate`, and test
modules `use YourModule` to inherit setup callbacks and helpers.
This is the same pattern Phoenix uses for `ConnCase` and `DataCase`.
It originates from ExUnit itself — the framework demonstrates its own
extension point.
### 5.6 Logger: Erlang Integration Done Right
PR #9333 (Sep 2019, merged Nov 2019): "Use Erlang's logger as main
logging implementation." The Elixir Logger was rewritten to sit on top
of Erlang's `:logger` module rather than reimplementing log dispatch.
**Convention:** When OTP provides infrastructure, wrap it rather than
replace it. The compatibility layer translates Erlang log messages to
Elixir format, but dispatch/filtering/handlers are OTP's.
---
## 6. PR Discussion Patterns
### JSON.Encoder (PR #14021, Dec 2024)
38 review comments, 13 days to merge. Key debate:
**sabiwara** asked: "What is the reason we went with a different API
than Jason?" — questioning why the stdlib JSON module doesn't mirror
the dominant community library.
**michalmuskala** (Jason author): "Once 1.18 is released with the new
JSON module, I plan to make a new release of Jason with some small
fixes and then effectively deprecate it."
**Lesson:** When stdlib absorbs community library functionality, the
community library author participates in the review. Jason's author
blessed the replacement and planned deprecation. This is how healthy
ecosystem evolution works.
### Duration (PR #13385, Mar-Apr 2024)
75 comments + 116 review comments. The most debated PR in recent
Elixir history.
**Pattern:** Community contributor (@tfiedlerdejanze) opened a PR
adding `Date.shift/2`. José redirected to a broader `Duration` type.
The contributor iterated through multiple designs.
**José's key intervention:** "I would rather prefer to pass a Duration
to Calendar.ISO.shift_date, if we ever have such a type, rather than a
keyword list." — refusing a simpler PR because it would lock in a
suboptimal API before the full design was clear.
**Lesson:** The BDFL model means one person can say "this is the wrong
abstraction" and redirect months of work. The PR took 33 days and
several complete rewrites. The result was better because someone held
the line on "solve the whole problem, not just the immediate pain."
### Formatter (PR #6639, Oct 2017)
Zero comments. Merged in 1 hour. 2,605 lines of new code.
**Lesson:** BDFL-driven projects can ship massive foundational changes
with no review. José was both the author and the authority. This is the
opposite of CockroachDB's Handle PR (2.5 months, extensive debate).
Neither model is wrong — it depends on team structure and trust level.
---
## 7. Cross-Ecosystem Comparisons
| Aspect | Elixir | Go |
|--------|--------|-----|
| TODOs | 127, all version-gated | 3,428, all owner-attributed |
| Formatter origin | BDFL ships in 1hr, no review | `gofmt` shipped with language |
| Bootstrap | Erlang (33 files, permanent) | Assembly + Go (self-hosting since 1.5) |
| Extension | 6 protocols + CaseTemplate | `internal/` packages (61 of them) |
| Type system | Set-theoretic, 13K lines, growing | Static, mature, compile-time only |
| Test ratio | 1:1.2 (file per file) | 1:3.3 (package-level tests) |
| Governance | BDFL (José) | Committee (Russ Cox + team) |
---
## 8. What This Teaches
1. **BDFL projects can move faster on foundational infrastructure**
the formatter, type system, and JSON module all shipped because one
person had authority. But Duration took 33 days because community
contribution required iteration with the BDFL's vision.
2. **Version-gated TODOs are a superior cleanup strategy** for
projects with regular release cycles. You never have to decide "is
this worth fixing?" — the version bump forces the question.
3. **Keep the minimum viable bootstrap in the host language.** 33
Erlang files is the floor, not a ceiling. The trajectory is always
toward more Elixir, less Erlang — but the tokenizer stays in Erlang
because binary matching is genuinely faster there.
4. **The type system's growth rate predicts the language's future.**
504 commits, 96% from José, nearly as large as Kernel. Elixir's
next 5 years will be defined by gradual typing.
5. **Community library authors should bless stdlib absorption.** The
Jason → JSON.Encoder transition worked because michalmuskala
participated in the review and planned deprecation.
6. **Each OTP app is an independent unit** — this convention flows
directly into how Phoenix projects are organized. The language
teaches its own architectural pattern by example.
<!-- PATTERN_COMPLETE -->
+266
View File
@@ -0,0 +1,266 @@
# Elixir Language Source: Convention Reference
Quick-reference for conventions extracted from the elixir-lang/elixir
source code. Each entry: pattern name, location, example, when to use,
when NOT to use, origin.
---
## Version-Gated TODOs
**Location:** Throughout `lib/`
```elixir
# TODO: Remove me on v2.0
# TODO: Deprecate me on Elixir v1.23
# TODO: Make an error on Elixir v2.0
```
**When to use:** Any backward-compatible code that should be removed at
a known future version. Deprecation paths, compatibility shims, feature
flags.
**When NOT to use:** Performance improvements ("make this faster
someday"), refactoring desires, or anything without a clear version
boundary. Those belong in issues, not TODOs.
**Origin:** Consistent throughout history. The pattern predates the
repo's earliest commits — José's convention from the start.
---
## Independent OTP Applications
**Location:** `lib/` (6 top-level dirs)
```
lib/elixir/mix.exs # The language core
lib/ex_unit/mix.exs # Testing framework
lib/mix/mix.exs # Build tool
lib/logger/mix.exs # Logging
lib/iex/mix.exs # Interactive shell
lib/eex/mix.exs # Templates
```
**When to use:** When components have distinct lifecycle, deployment, or
dependency requirements. When you want components to be independently
testable.
**When NOT to use:** Small projects where the overhead of multiple
applications exceeds the organizational benefit. If components always
deploy together and never independently, a single app is simpler.
**Origin:** Elixir 0.x — the language was always structured this way.
---
## Erlang for Performance-Critical Paths
**Location:** `lib/elixir/src/` (33 .erl files)
```erlang
%% elixir_tokenizer.erl — binary pattern matching is faster in Erlang
%% elixir_erl_pass.erl — code generation benefits from proximity to BEAM
```
**When to use:** When Erlang's binary pattern matching or NIF interface
provides measurable performance advantage. When code must exist before
Elixir's compiler is available (bootstrap).
**When NOT to use:** For new features that could be written in Elixir.
The trajectory is always toward less Erlang. Don't write new Erlang
unless profiling proves it's necessary.
**Origin:** The entire language started as Erlang (2011). Bootstrap file
(`elixir_bootstrap.erl`) formalized as distinct from "Elixir written
in Erlang" in 2013 (commit `260be7c8e`).
---
## Protocol Consolidation
**Location:** `lib/elixir/lib/protocol.ex`
```elixir
# In mix.exs (default for prod):
consolidate_protocols: true
# Disable for tests:
consolidate_protocols: Mix.env() != :test
```
**When to use:** Always in production (it's the default). Consolidated
protocols dispatch in two function calls instead of dynamic lookup.
**When NOT to use:** In tests where you define new protocol
implementations at runtime. In dev if you're frequently recompiling
protocol implementations.
**Origin:** PR adding consolidation to escript.build (2014, issue
#2699). Later made default for Mix projects.
---
## Code.Formatter via Inspect.Algebra
**Location:** `lib/elixir/lib/code/formatter.ex` (2,605 lines)
```elixir
# The formatter is a library function:
Code.format_string!("def foo( x,y),do: x+y")
# => "def foo(x, y), do: x + y"
# Internally uses Wadler-Lindig algebra:
import Inspect.Algebra, except: [format: 2, surround: 3, surround: 4]
```
**When to use:** When building code generation tools, macros that
produce source, or custom formatting rules. The algebra is available
to any Elixir code.
**When NOT to use:** Don't reimplement formatting logic. Use
`Code.format_string!/2` or the `mix format` task. The algebra is for
building formatters, not for end-user formatting.
**Origin:** Oct 7, 2017 (PR #6639, José Valim). Merged in 1 hour, zero
review comments.
---
## Mix Task Convention
**Location:** `lib/mix/lib/mix/tasks/` (55 files)
```elixir
defmodule Mix.Tasks.Deps.Clean do
@moduledoc "Removes the given dependencies' build artifacts."
@shortdoc "Deletes generated files and artifacts for dependencies"
use Mix.Task
@recursive true
@impl true
def run(args) do
# ...
end
end
```
**When to use:** Any command-line operation that should be available via
`mix <task>`. One file per task, module name determines task name.
**When NOT to use:** Tasks that require interactive user input (use
escript or IEx helpers instead). Tasks that need to run without Mix
loaded.
**Origin:** Part of Mix since its creation. The one-file convention
is strict — all 55 stdlib tasks follow it.
---
## ExUnit.CaseTemplate
**Location:** `lib/ex_unit/lib/ex_unit/case_template.ex` (162 lines)
```elixir
defmodule MyApp.DataCase do
use ExUnit.CaseTemplate
setup tags do
:ok = Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo)
unless tags[:async], do: Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, {:shared, self()})
:ok
end
end
# Usage:
defmodule MyApp.SomeTest do
use MyApp.DataCase, async: true
end
```
**When to use:** When multiple test modules share setup logic,
assertions, or helper functions. The inheritance model allows
composition of test concerns.
**When NOT to use:** For test helpers that don't need lifecycle
callbacks. Simple `import` or `alias` is cleaner for utility functions
that don't need `setup`/`setup_all`.
**Origin:** Commit `1f491dfbe` — "Add support to case templates." The
pattern Phoenix adopted for `ConnCase`/`DataCase` comes directly from
ExUnit.
---
## Logger Wrapping OTP
**Location:** `lib/logger/`
```elixir
# Elixir's Logger dispatches to Erlang's :logger
# Translation layer converts Erlang messages → Elixir format
# Filters and handlers use OTP's infrastructure
```
**When to use:** When OTP provides the infrastructure you need.
Wrap it — don't replace it. Provide Elixir-idiomatic API on top.
**When NOT to use:** When OTP's solution has fundamental architectural
limitations you can't work around with a wrapper (rare).
**Origin:** PR #9333 (Sep-Nov 2019). Rewrote Logger from custom
dispatch to wrapping Erlang's `:logger`. ~2 month implementation.
---
## BDFL Merge Pattern
**Location:** Throughout git history
```
PR #6639 (Formatter): 0 comments, merged in 1 hour, 2,605 new lines
PR #14021 (JSON): 38 review comments, 13 days, community input
PR #13385 (Duration): 75+116 comments, 33 days, community redirected
```
**When to use:** Foundational infrastructure where one person holds the
architectural vision. The BDFL can ship without review when the change
is self-evidently correct (formatter) or redirect community work when
the abstraction isn't right (Duration).
**When NOT to use:** In team-maintained projects without a clear
authority. In projects where consensus is required for API decisions.
The BDFL model fails when the BDFL is wrong and no one can override.
**Origin:** José Valim has been sole authority since Elixir's creation
(2011). The core team exists but José has final say on language design.
---
## Type System Architecture (Set-Theoretic)
**Location:** `lib/elixir/lib/module/types/` (13,034 lines, 7 files)
```elixir
# Types are sets. Operations are set operations.
# descr.ex (6,301 lines) defines the algebra:
# - Union of types
# - Intersection of types
# - Difference (negation types)
# - Subtype checking via set inclusion
```
**When to use:** Understanding how Elixir's gradual type system works
internally. The set-theoretic approach means types compose naturally —
`integer() | String.t()` is literally a set union.
**When NOT to use:** This is internal compiler infrastructure. Don't
depend on `Module.Types` internals — they change frequently (504
commits and counting).
**Origin:** Aug 2019 (PR #9270 by Eric Meadows-Jönsson). Originally
just function clause exhaustiveness checking, now growing into full
gradual typing. 96% of subsequent work by José.
<!-- PATTERN_COMPLETE -->
+359
View File
@@ -0,0 +1,359 @@
# Patterns Extracted from oban-bg/oban
## Pattern: Plugin as Behaviour + GenServer
**Source:** `lib/oban/plugin.ex`
**Category:** plugin
**What:** Define a plugin interface as a behaviour with
`start_link/1` and `validate/1` callbacks. Plugins must be
OTP-compliant (GenServer/Agent). The host supervises them.
**Why:** Extensibility without coupling. Oban can start any
module that satisfies the behaviour — pruning, cron,
lifeline — without knowing implementation details. The
`validate/1` callback ensures misconfigured plugins fail at
startup, not at runtime.
**Example:**
```elixir
@callback start_link([option()]) :: GenServer.on_start()
@callback validate([option()]) :: :ok | {:error, String.t()}
@optional_callbacks [format_logger_output: 2]
```
**When to use:** When your application needs a plugin
system where third parties add behavior. The behaviour
ensures type safety; supervision ensures fault isolation.
**When NOT to use:** Internal modules that you control.
Behaviours add ceremony — if there is only one
implementation, use a module directly.
---
## Pattern: Structured Telemetry Spans
**Source:** `lib/oban/telemetry.ex`
**Category:** telemetry
**What:** Emit telemetry events as spans with
start/stop/exception structure. Every operation (job
execution, engine calls, plugin work) follows the same
three-event pattern with consistent metadata shapes.
**Why:** Uniform observability. Any monitoring tool
(AppSignal, Datadog, custom logger) can hook into the same
event structure. The span pattern (start → stop|exception)
enables latency tracking, error rates, and resource usage
measurement without custom instrumentation per feature.
**Example:**
```elixir
# Event names follow: [:oban, :component, :action, :phase]
[:oban, :job, :start]
[:oban, :job, :stop] # measurements: duration, memory
[:oban, :job, :exception] # + kind, reason, stacktrace
[:oban, :engine, :fetch_jobs, :start]
[:oban, :engine, :fetch_jobs, :stop]
[:oban, :engine, :fetch_jobs, :exception]
```
**When to use:** Any library or application that wants
observability without coupling to a specific monitoring
backend. The pattern works for database queries, HTTP
requests, background jobs, cache operations.
**When NOT to use:** Ultra-hot paths where telemetry
overhead matters (millions of events/second). Use sampling
or skip entirely.
---
## Pattern: Engine Abstraction for Backend Swap
**Source:** `lib/oban/engine.ex`
**Category:** engine
**What:** Define a behaviour (`Engine`) with callbacks for
all database operations (insert, fetch, complete, etc.).
Ship multiple implementations (Basic/Inline/Lite) that swap
at config time.
**Why:** Different environments need different backends:
Postgres for production, SQLite for development, inline
(in-memory) for testing. The engine abstraction lets you
swap without changing application code.
**Example:**
```elixir
@callback init(conf, opts) :: {:ok, meta} | {:error, term}
@callback insert_job(conf, changeset, opts) :: {:ok, Job.t()}
@callback fetch_jobs(conf, meta, opts) :: {:ok, {meta, [Job.t()]}}
@callback complete_job(conf, Job.t()) :: :ok
```
**When to use:** When your system needs to support multiple
storage backends, or when testing requires a fundamentally
different execution model (synchronous vs async).
**When NOT to use:** Single-backend applications. The
abstraction layer adds complexity that is only justified
when you actually swap implementations.
---
## Pattern: Keyword Validation with Reduce-While
**Source:** `lib/oban/validation.ex`
**Category:** config
**What:** Validate keyword options by iterating with
`Enum.reduce_while/3` and a validator function. Stop at
first error. Return `:ok` or `{:error, reason}`.
**Why:** Keyword lists are the standard Elixir config
format. Validating them procedurally (nested if/case) gets
messy. The reduce-while + validator pattern is composable:
each option validates independently, errors short-circuit,
and the validator function can be swapped or extended.
**Example:**
```elixir
def validate(opts, validator) when is_list(opts) do
Enum.reduce_while(opts, :ok, fn opt, acc ->
case validator.(opt) do
:ok -> {:cont, acc}
{:error, _} = error -> {:halt, error}
end
end)
end
```
**When to use:** Any public API that accepts keyword
options from users. Libraries, GenServer init, plugin
configs.
**When NOT to use:** Internal functions where the caller
is trusted. Also avoid for deeply nested configs — use
schema-based validation (NimbleOptions, Ecto embedded
schemas) instead.
---
## Pattern: Testing Mode Toggle
**Source:** `lib/oban/testing.ex`, `lib/oban/config.ex`
**Category:** testing
**What:** Support a `testing:` config option that switches
execution mode: `:disabled` (production), `:inline`
(execute immediately in caller process), `:manual` (enqueue
but don't execute — assert on DB state).
**Why:** Background job systems are inherently async, which
makes testing hard. The mode toggle gives you: (1) inline
for unit tests that need synchronous execution, (2) manual
for integration tests that verify enqueueing without
side effects.
**Example:**
```elixir
# In test config:
config :my_app, Oban, testing: :manual
# In tests:
use Oban.Testing, repo: MyApp.Repo
perform_job(MyWorker, %{id: 1})
assert_enqueued worker: MyWorker, args: %{id: 1}
```
**When to use:** Any async system that needs deterministic
testing — job queues, event buses, notification systems.
The testing mode replaces "sleep and hope" with explicit
control.
**When NOT to use:** Synchronous systems that are already
deterministic. Also avoid if the mode toggle leaks into
production code paths (keep it config-only, not conditional
logic scattered through business code).
---
## Pattern: Stopper for Goroutine Lifecycle (CockroachDB)
**Source:** `pkg/util/stop/stopper.go` (cockroachdb)
**Category:** concurrency
**What:** A dedicated struct that manages the lifecycle of
all goroutines in a component: tracks active tasks, refuses
new work during shutdown (quiesce), waits for completion,
then runs closers.
**Why:** In distributed systems, clean shutdown is critical.
You need to: (1) stop accepting new work, (2) finish
in-flight work, (3) release resources in order. The Stopper
centralizes this instead of scattering shutdown logic across
every goroutine.
**Example:**
```go
type Stopper struct {
quiescer chan struct{} // closed when quiescing
stopped chan struct{} // closed when fully stopped
mu struct {
syncutil.RWMutex
_numTasks int32
quiescing, stopping bool
closers []Closer
}
}
// RunAsyncTask refuses new work during quiesce
func (s *Stopper) RunAsyncTask(ctx context.Context,
taskName string, f func(context.Context)) error {
if !s.addTask() {
return ErrUnavailable
}
go func() {
defer s.decTask()
f(ctx)
}()
return nil
}
```
**When to use:** Any server or subsystem that spawns
goroutines and needs graceful shutdown. Especially in
long-running services where leaked goroutines cause
resource exhaustion.
**When NOT to use:** Simple programs with a single main
goroutine. Or when `errgroup` with context cancellation
suffices for the shutdown coordination.
---
## Pattern: Atomic File Operations with Suffix Convention
**Source:** `tsdb/db.go` (prometheus)
**Category:** storage
**What:** Use directory suffixes (`.tmp-for-creation`,
`.tmp-for-deletion`) to make multi-step file operations
crash-safe. On startup, clean up any dirs with these
suffixes (they represent incomplete operations).
**Why:** Database storage needs atomicity. If the process
crashes between creating a block and finalizing it, you
need to know the block is incomplete. The suffix convention
makes incomplete state visible at the filesystem level
without requiring a separate journal.
**Example:**
```go
const (
tmpForDeletionBlockDirSuffix = ".tmp-for-deletion"
tmpForCreationBlockDirSuffix = ".tmp-for-creation"
)
// On startup: remove any .tmp-* dirs (incomplete ops)
// On create: write to dir.tmp-for-creation, then rename
// On delete: rename to dir.tmp-for-deletion, then remove
```
**When to use:** Any system that manages files/directories
and needs crash consistency without a full WAL. Simpler
than a write-ahead log for coarse-grained operations.
**When NOT to use:** When you already have a WAL or
transaction log. Or for fine-grained operations where
rename semantics are insufficient.
---
## Pattern: Options as DefaultOptions() + Override
**Source:** `tsdb/db.go` (prometheus)
**Category:** configuration
**What:** Provide a `DefaultOptions()` function returning a
fully-populated config struct. Users copy and override only
what they need. No nil-means-default ambiguity.
**Why:** Large config structs (20+ fields) are unwieldy.
By providing sane defaults as a function (not a package-
level var), you avoid mutation bugs and make it clear what
"normal" looks like. Users only specify deviations.
**Example:**
```go
func DefaultOptions() *Options {
return &Options{
WALSegmentSize: wlog.DefaultSegmentSize,
RetentionDuration: int64(15 * 24 * time.Hour / ...),
MinBlockDuration: DefaultBlockDuration,
MaxBlockDuration: DefaultBlockDuration,
SamplesPerChunk: DefaultSamplesPerChunk,
// ... 20 more fields with sane defaults
}
}
// Usage:
opts := tsdb.DefaultOptions()
opts.RetentionDuration = 30 * 24 * time.Hour
db, err := tsdb.Open(dir, nil, nil, opts, nil)
```
**When to use:** Config structs with many fields where most
users want defaults. Especially when zero-value semantics
would be confusing (e.g., 0 retention = infinite? or off?).
**When NOT to use:** Small configs (3-4 fields) where
struct literal with zero-means-default is clear enough.
---
## Pattern: Scrape Loop with Aligned Timestamps
**Source:** `scrape/scrape.go` (prometheus)
**Category:** concurrency
**What:** Periodic scrape loops that align timestamps to
intervals with a small tolerance, enabling better storage
compression downstream.
**Why:** Time-series databases compress better when
timestamps are regular. A 2ms tolerance on alignment
means scraped data aligns to the expected grid while
accommodating real-world jitter.
**Example:**
```go
var ScrapeTimestampTolerance = 2 * time.Millisecond
var AlignScrapeTimestamps = true
// In scrape loop: if scrape finishes within tolerance
// of the expected timestamp, snap to the grid
```
**When to use:** Any periodic data collection where
downstream storage benefits from timestamp regularity.
Metrics, heartbeats, polling loops.
**When NOT to use:** Event-driven data where timestamps
must reflect actual occurrence time. Audit logs, user
actions, financial transactions.
<!-- PATTERN_COMPLETE -->