rodin/elixir-patterns
1
0
Fork 0
Code Issues 2 Pull Requests Actions Packages Projects Releases Wiki Activity

chore: merge elixir-conventions and oban-conventions into sources/

Browse Source 
Absorbed content from rodin/elixir-conventions and rodin/oban-conventions
into a sources/ directory. These are reference material — descriptive,
not prescriptive. Patterns that prove broadly applicable get promoted
into patterns/.

Part of taxonomy cleanup (issue #4):
- Pattern = prescriptive, follow these
- Convention/Source = reference, study for ideas

The original repos can now be archived.
This commit is contained in:
Aaron Weiker
2026-05-07 18:01:36 -07:00
parent f595b91030
commit c11cd3f1ce
4 changed files with 943 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
sources/README.md
+15
View File
@@ -0,0 +1,15 @@
# Sources
Reference material extracted from specific projects. Study for ideas, don't copy blindly.
These are **descriptive** — they document what a project does and why.
The `patterns/` directory is **prescriptive** — it tells you what to do.
Patterns that prove broadly applicable get promoted from here into `patterns/`.
The rest stays as reference for understanding how mature projects solve specific problems.
## Files
- `elixir-lang.md` — conventions from the elixir-lang/elixir source
- `elixir-lang-analysis.md` — deeper analysis of elixir-lang source architecture
- `oban.md` — patterns from oban-bg/oban (job processing, plugin system)
sources/elixir-lang-analysis.md
+303
View File
@@ -0,0 +1,303 @@
# Elixir Language Source: Architectural Conventions
How does José Valim and the Elixir core team build Elixir itself?
What does the language source reveal about conventions that aren't
documented anywhere else?
**Repo:** [elixir-lang/elixir](https://github.com/elixir-lang/elixir)
---
## 1. Repo Shape
| Metric | Value |
|--------|-------|
| Size | 92M |
| Source files | 567 .ex/.exs |
| Erlang bootstrap | 33 .erl files |
| Commits | 22,032 |
| Contributors | 1,578 |
| Test files | 208 |
| Production files | 248 |
| Test ratio | 1:1.2 |
| TODOs (non-test) | 127 (all version-gated) |
### Organizational Philosophy
```
lib/
├── elixir/ # The language core (compiler + stdlib)
│ ├── src/ # 33 Erlang files (bootstrap)
│ └── lib/ # Elixir stdlib + compiler
├── eex/ # Templating (independent OTP app)
├── ex_unit/ # Testing framework (independent OTP app)
├── iex/ # Interactive shell (independent OTP app)
├── logger/ # Logging (independent OTP app)
└── mix/ # Build tool (independent OTP app)
```
Each component is a separate OTP application. They could theoretically
be released independently. This is Elixir eating its own dog food —
the umbrella project convention that Phoenix apps use comes directly
from how the language itself is organized.
---
## 2. What the Codebase Values
### By size (what gets the most lines)
| Module | Lines | Role |
|--------|-------|------|
| `Kernel` | 7,102 | The implicit language surface |
| `Module.Types.Descr` | 6,301 | Set-theoretic type descriptions |
| `Enum` | 5,242 | Collection operations |
| `String` | 3,263 | First-class string concept |
| `Macro` | 3,102 | Metaprogramming foundation |
| `Exception` | 2,720 | Error taxonomy |
| `Code.Formatter` | 2,605 | Code formatting as library |
**The surprise:** The type system (`types/descr.ex` at 6,301 lines) is
nearly as large as Kernel (7,102 lines). It's the newest and
fastest-growing module — 504 commits, 96% written by José Valim. This
is where the investment is going.
### By authorship (who shapes the language)
Type system: 396/504 commits from José, 32 from Eric Meadows-Jönsson,
31 from Guillaume Duboc. This is auteur-driven development — one person
holds the architectural vision for the most complex subsystem.
---
## 3. The Bootstrap Problem
**How does Elixir compile itself?**
The answer is 33 Erlang files in `lib/elixir/src/`:
```
elixir_bootstrap.erl — minimal Kernel for self-compilation
elixir_compiler.erl — the compiler entry point
elixir_tokenizer.erl — lexer (in Erlang for speed)
elixir_expand.erl — macro expansion
elixir_erl.erl — Elixir AST → Erlang AST
elixir_erl_pass.erl — code generation pass
elixir_env.erl — compilation environment
elixir_clauses.erl — pattern matching compilation
```
**Convention:** The tokenizer and core compiler remain in Erlang
permanently. This isn't technical debt — it's a deliberate choice.
The tokenizer benefits from Erlang's binary pattern matching
performance. The compiler needs to exist before Elixir does.
**Origin:** The bootstrap file dates to Nov 22, 2013 (commit
`260be7c8e`: "Start porting elixir_macros to pure elixir"). Before
this, MORE of the compiler was in Erlang. The trajectory is clear:
minimize Erlang over time, but keep it where it provides genuine value.
---
## 4. TODO Culture: Version-Gated Deadlines
```elixir
# TODO: Remove me on v2.0 — 16 occurrences
# TODO: Deprecate me on Elixir v1.23 — 6 occurrences
# TODO: Remove this clause on Elixir v2.0 once single-quoted charlists are removed
# TODO: Make an error on Elixir v2.0 — 3 occurrences
# TODO: Deprecate on Elixir v1.22 — 3 occurrences
```
**Convention:** Every TODO has a version target. No "someday" TODOs
exist. When a version ships, grep for that version's TODOs and resolve
them all.
**127 total TODOs** across 567 files. Contrast with Go's 3,428 TODOs
across 11K files — the Elixir team treats TODOs as time-bombs, not
documentation.
---
## 5. Unique Patterns
### 5.1 Protocol Consolidation
Protocols dispatch dynamically at runtime by default (checking each
struct's implementation). **Protocol consolidation** compiles all known
implementations into a single dispatch module at build time.
From `lib/elixir/lib/protocol.ex`:
> "Consolidation directly links the protocol to its implementations.
> Invoking a consolidated protocol is equivalent to invoking two remote
> functions."
**Convention:** Mix enables consolidation by default in production. The
`@callback __protocol__(:consolidated?)` exists so code can check at
runtime whether fast-path dispatch is active.
**When NOT to use:** Tests often disable consolidation (`consolidate_
protocols: false`) so new protocol implementations added during tests
are discoverable without recompilation.
### 5.2 Parallel Type Checker
`Module.ParallelChecker` (introduced July 2019, PR #9203 by Eric
Meadows-Jönsson as "Add ExCk chunk") enables concurrent type checking
across modules.
The type system itself (13,034 lines across 7 files in
`lib/elixir/lib/module/types/`) is set-theoretic — types are sets, and
operations are set operations (union, intersection, difference).
**Key files:**
- `descr.ex` (6,301 lines) — type descriptions and set operations
- `apply.ex` — function application typing
- `expr.ex` — expression typing
- `pattern.ex` — pattern match typing
- `of.ex` — type inference
- `helpers.ex` — shared utilities
- `traverse.ex` — AST traversal
### 5.3 Code.Formatter as Library Function
The code formatter (2,605 lines) is a library function, not a CLI tool.
You can call `Code.format_string!/2` from any Elixir code.
**Introduced:** Oct 7, 2017 (PR #6639 by José Valim). **Zero review
comments. Merged in 1 hour.** José opened and merged his own formatter
with no external review. This is the BDFL model — the language author
ships foundational infrastructure by authority.
**Convention:** The formatter uses `Inspect.Algebra` (Wadler-Lindig
pretty-printing) for layout decisions. It defines all operators and
their associativity as module attributes:
```elixir
@pipeline_operators [:|>, :~>>, :<<~, :~>, :<~, :<~>, :"<|>"]
@right_new_line_before_binary_operators [:|, :when]
@required_parens_logical_binary_operands [:|||, :||, :or, :&&&, :&&, :and]
```
### 5.4 Mix Tasks as Single-File Modules
55 Mix tasks, each in its own file. Convention:
- One task = one file
- Module name determines task name: `Mix.Tasks.Deps.Clean` → `deps.clean`
- `@shortdoc` for brief help, `@moduledoc` for full docs
- `@recursive true` for umbrella traversal
### 5.5 ExUnit CaseTemplate (Extension Pattern)
The `ExUnit.CaseTemplate` is how Elixir's test framework supports
extension — you define a module that `use`s `CaseTemplate`, and test
modules `use YourModule` to inherit setup callbacks and helpers.
This is the same pattern Phoenix uses for `ConnCase` and `DataCase`.
It originates from ExUnit itself — the framework demonstrates its own
extension point.
### 5.6 Logger: Erlang Integration Done Right
PR #9333 (Sep 2019, merged Nov 2019): "Use Erlang's logger as main
logging implementation." The Elixir Logger was rewritten to sit on top
of Erlang's `:logger` module rather than reimplementing log dispatch.
**Convention:** When OTP provides infrastructure, wrap it rather than
replace it. The compatibility layer translates Erlang log messages to
Elixir format, but dispatch/filtering/handlers are OTP's.
---
## 6. PR Discussion Patterns
### JSON.Encoder (PR #14021, Dec 2024)
38 review comments, 13 days to merge. Key debate:
**sabiwara** asked: "What is the reason we went with a different API
than Jason?" — questioning why the stdlib JSON module doesn't mirror
the dominant community library.
**michalmuskala** (Jason author): "Once 1.18 is released with the new
JSON module, I plan to make a new release of Jason with some small
fixes and then effectively deprecate it."
**Lesson:** When stdlib absorbs community library functionality, the
community library author participates in the review. Jason's author
blessed the replacement and planned deprecation. This is how healthy
ecosystem evolution works.
### Duration (PR #13385, Mar-Apr 2024)
75 comments + 116 review comments. The most debated PR in recent
Elixir history.
**Pattern:** Community contributor (@tfiedlerdejanze) opened a PR
adding `Date.shift/2`. José redirected to a broader `Duration` type.
The contributor iterated through multiple designs.
**José's key intervention:** "I would rather prefer to pass a Duration
to Calendar.ISO.shift_date, if we ever have such a type, rather than a
keyword list." — refusing a simpler PR because it would lock in a
suboptimal API before the full design was clear.
**Lesson:** The BDFL model means one person can say "this is the wrong
abstraction" and redirect months of work. The PR took 33 days and
several complete rewrites. The result was better because someone held
the line on "solve the whole problem, not just the immediate pain."
### Formatter (PR #6639, Oct 2017)
Zero comments. Merged in 1 hour. 2,605 lines of new code.
**Lesson:** BDFL-driven projects can ship massive foundational changes
with no review. José was both the author and the authority. This is the
opposite of CockroachDB's Handle PR (2.5 months, extensive debate).
Neither model is wrong — it depends on team structure and trust level.
---
## 7. Cross-Ecosystem Comparisons
| Aspect | Elixir | Go |
|--------|--------|-----|
| TODOs | 127, all version-gated | 3,428, all owner-attributed |
| Formatter origin | BDFL ships in 1hr, no review | `gofmt` shipped with language |
| Bootstrap | Erlang (33 files, permanent) | Assembly + Go (self-hosting since 1.5) |
| Extension | 6 protocols + CaseTemplate | `internal/` packages (61 of them) |
| Type system | Set-theoretic, 13K lines, growing | Static, mature, compile-time only |
| Test ratio | 1:1.2 (file per file) | 1:3.3 (package-level tests) |
| Governance | BDFL (José) | Committee (Russ Cox + team) |
---
## 8. What This Teaches
1. **BDFL projects can move faster on foundational infrastructure** —
the formatter, type system, and JSON module all shipped because one
person had authority. But Duration took 33 days because community
contribution required iteration with the BDFL's vision.
2. **Version-gated TODOs are a superior cleanup strategy** for
projects with regular release cycles. You never have to decide "is
this worth fixing?" — the version bump forces the question.
3. **Keep the minimum viable bootstrap in the host language.** 33
Erlang files is the floor, not a ceiling. The trajectory is always
toward more Elixir, less Erlang — but the tokenizer stays in Erlang
because binary matching is genuinely faster there.
4. **The type system's growth rate predicts the language's future.**
504 commits, 96% from José, nearly as large as Kernel. Elixir's
next 5 years will be defined by gradual typing.
5. **Community library authors should bless stdlib absorption.** The
Jason → JSON.Encoder transition worked because michalmuskala
participated in the review and planned deprecation.
6. **Each OTP app is an independent unit** — this convention flows
directly into how Phoenix projects are organized. The language
teaches its own architectural pattern by example.
<!-- PATTERN_COMPLETE -->
sources/elixir-lang.md
+266
View File
@@ -0,0 +1,266 @@
# Elixir Language Source: Convention Reference
Quick-reference for conventions extracted from the elixir-lang/elixir
source code. Each entry: pattern name, location, example, when to use,
when NOT to use, origin.
---
## Version-Gated TODOs
**Location:** Throughout `lib/`
```elixir
# TODO: Remove me on v2.0
# TODO: Deprecate me on Elixir v1.23
# TODO: Make an error on Elixir v2.0
```
**When to use:** Any backward-compatible code that should be removed at
a known future version. Deprecation paths, compatibility shims, feature
flags.
**When NOT to use:** Performance improvements ("make this faster
someday"), refactoring desires, or anything without a clear version
boundary. Those belong in issues, not TODOs.
**Origin:** Consistent throughout history. The pattern predates the
repo's earliest commits — José's convention from the start.
---
## Independent OTP Applications
**Location:** `lib/` (6 top-level dirs)
```
lib/elixir/mix.exs # The language core
lib/ex_unit/mix.exs # Testing framework
lib/mix/mix.exs # Build tool
lib/logger/mix.exs # Logging
lib/iex/mix.exs # Interactive shell
lib/eex/mix.exs # Templates
```
**When to use:** When components have distinct lifecycle, deployment, or
dependency requirements. When you want components to be independently
testable.
**When NOT to use:** Small projects where the overhead of multiple
applications exceeds the organizational benefit. If components always
deploy together and never independently, a single app is simpler.
**Origin:** Elixir 0.x — the language was always structured this way.
---
## Erlang for Performance-Critical Paths
**Location:** `lib/elixir/src/` (33 .erl files)
```erlang
%% elixir_tokenizer.erl — binary pattern matching is faster in Erlang
%% elixir_erl_pass.erl — code generation benefits from proximity to BEAM
```
**When to use:** When Erlang's binary pattern matching or NIF interface
provides measurable performance advantage. When code must exist before
Elixir's compiler is available (bootstrap).
**When NOT to use:** For new features that could be written in Elixir.
The trajectory is always toward less Erlang. Don't write new Erlang
unless profiling proves it's necessary.
**Origin:** The entire language started as Erlang (2011). Bootstrap file
(`elixir_bootstrap.erl`) formalized as distinct from "Elixir written
in Erlang" in 2013 (commit `260be7c8e`).
---
## Protocol Consolidation
**Location:** `lib/elixir/lib/protocol.ex`
```elixir
# In mix.exs (default for prod):
consolidate_protocols: true
# Disable for tests:
consolidate_protocols: Mix.env() != :test
```
**When to use:** Always in production (it's the default). Consolidated
protocols dispatch in two function calls instead of dynamic lookup.
**When NOT to use:** In tests where you define new protocol
implementations at runtime. In dev if you're frequently recompiling
protocol implementations.
**Origin:** PR adding consolidation to escript.build (2014, issue
#2699). Later made default for Mix projects.
---
## Code.Formatter via Inspect.Algebra
**Location:** `lib/elixir/lib/code/formatter.ex` (2,605 lines)
```elixir
# The formatter is a library function:
Code.format_string!("def foo( x,y),do: x+y")
# => "def foo(x, y), do: x + y"
# Internally uses Wadler-Lindig algebra:
import Inspect.Algebra, except: [format: 2, surround: 3, surround: 4]
```
**When to use:** When building code generation tools, macros that
produce source, or custom formatting rules. The algebra is available
to any Elixir code.
**When NOT to use:** Don't reimplement formatting logic. Use
`Code.format_string!/2` or the `mix format` task. The algebra is for
building formatters, not for end-user formatting.
**Origin:** Oct 7, 2017 (PR #6639, José Valim). Merged in 1 hour, zero
review comments.
---
## Mix Task Convention
**Location:** `lib/mix/lib/mix/tasks/` (55 files)
```elixir
defmodule Mix.Tasks.Deps.Clean do
@moduledoc "Removes the given dependencies' build artifacts."
@shortdoc "Deletes generated files and artifacts for dependencies"
use Mix.Task
@recursive true
@impl true
def run(args) do
# ...
end
end
```
**When to use:** Any command-line operation that should be available via
`mix <task>`. One file per task, module name determines task name.
**When NOT to use:** Tasks that require interactive user input (use
escript or IEx helpers instead). Tasks that need to run without Mix
loaded.
**Origin:** Part of Mix since its creation. The one-file convention
is strict — all 55 stdlib tasks follow it.
---
## ExUnit.CaseTemplate
**Location:** `lib/ex_unit/lib/ex_unit/case_template.ex` (162 lines)
```elixir
defmodule MyApp.DataCase do
use ExUnit.CaseTemplate
setup tags do
:ok = Ecto.Adapters.SQL.Sandbox.checkout(MyApp.Repo)
unless tags[:async], do: Ecto.Adapters.SQL.Sandbox.mode(MyApp.Repo, {:shared, self()})
:ok
end
end
# Usage:
defmodule MyApp.SomeTest do
use MyApp.DataCase, async: true
end
```
**When to use:** When multiple test modules share setup logic,
assertions, or helper functions. The inheritance model allows
composition of test concerns.
**When NOT to use:** For test helpers that don't need lifecycle
callbacks. Simple `import` or `alias` is cleaner for utility functions
that don't need `setup`/`setup_all`.
**Origin:** Commit `1f491dfbe` — "Add support to case templates." The
pattern Phoenix adopted for `ConnCase`/`DataCase` comes directly from
ExUnit.
---
## Logger Wrapping OTP
**Location:** `lib/logger/`
```elixir
# Elixir's Logger dispatches to Erlang's :logger
# Translation layer converts Erlang messages → Elixir format
# Filters and handlers use OTP's infrastructure
```
**When to use:** When OTP provides the infrastructure you need.
Wrap it — don't replace it. Provide Elixir-idiomatic API on top.
**When NOT to use:** When OTP's solution has fundamental architectural
limitations you can't work around with a wrapper (rare).
**Origin:** PR #9333 (Sep-Nov 2019). Rewrote Logger from custom
dispatch to wrapping Erlang's `:logger`. ~2 month implementation.
---
## BDFL Merge Pattern
**Location:** Throughout git history
```
PR #6639 (Formatter): 0 comments, merged in 1 hour, 2,605 new lines
PR #14021 (JSON): 38 review comments, 13 days, community input
PR #13385 (Duration): 75+116 comments, 33 days, community redirected
```
**When to use:** Foundational infrastructure where one person holds the
architectural vision. The BDFL can ship without review when the change
is self-evidently correct (formatter) or redirect community work when
the abstraction isn't right (Duration).
**When NOT to use:** In team-maintained projects without a clear
authority. In projects where consensus is required for API decisions.
The BDFL model fails when the BDFL is wrong and no one can override.
**Origin:** José Valim has been sole authority since Elixir's creation
(2011). The core team exists but José has final say on language design.
---
## Type System Architecture (Set-Theoretic)
**Location:** `lib/elixir/lib/module/types/` (13,034 lines, 7 files)
```elixir
# Types are sets. Operations are set operations.
# descr.ex (6,301 lines) defines the algebra:
# - Union of types
# - Intersection of types
# - Difference (negation types)
# - Subtype checking via set inclusion
```
**When to use:** Understanding how Elixir's gradual type system works
internally. The set-theoretic approach means types compose naturally —
`integer() | String.t()` is literally a set union.
**When NOT to use:** This is internal compiler infrastructure. Don't
depend on `Module.Types` internals — they change frequently (504
commits and counting).
**Origin:** Aug 2019 (PR #9270 by Eric Meadows-Jönsson). Originally
just function clause exhaustiveness checking, now growing into full
gradual typing. 96% of subsequent work by José.
<!-- PATTERN_COMPLETE -->
sources/oban.md
+359
View File
@@ -0,0 +1,359 @@
# Patterns Extracted from oban-bg/oban
## Pattern: Plugin as Behaviour + GenServer
**Source:** `lib/oban/plugin.ex`
**Category:** plugin
**What:** Define a plugin interface as a behaviour with
`start_link/1` and `validate/1` callbacks. Plugins must be
OTP-compliant (GenServer/Agent). The host supervises them.
**Why:** Extensibility without coupling. Oban can start any
module that satisfies the behaviour — pruning, cron,
lifeline — without knowing implementation details. The
`validate/1` callback ensures misconfigured plugins fail at
startup, not at runtime.
**Example:**
```elixir
@callback start_link([option()]) :: GenServer.on_start()
@callback validate([option()]) :: :ok | {:error, String.t()}
@optional_callbacks [format_logger_output: 2]
```
**When to use:** When your application needs a plugin
system where third parties add behavior. The behaviour
ensures type safety; supervision ensures fault isolation.
**When NOT to use:** Internal modules that you control.
Behaviours add ceremony — if there is only one
implementation, use a module directly.
---
## Pattern: Structured Telemetry Spans
**Source:** `lib/oban/telemetry.ex`
**Category:** telemetry
**What:** Emit telemetry events as spans with
start/stop/exception structure. Every operation (job
execution, engine calls, plugin work) follows the same
three-event pattern with consistent metadata shapes.
**Why:** Uniform observability. Any monitoring tool
(AppSignal, Datadog, custom logger) can hook into the same
event structure. The span pattern (start → stop|exception)
enables latency tracking, error rates, and resource usage
measurement without custom instrumentation per feature.
**Example:**
```elixir
# Event names follow: [:oban, :component, :action, :phase]
[:oban, :job, :start]
[:oban, :job, :stop] # measurements: duration, memory
[:oban, :job, :exception] # + kind, reason, stacktrace
[:oban, :engine, :fetch_jobs, :start]
[:oban, :engine, :fetch_jobs, :stop]
[:oban, :engine, :fetch_jobs, :exception]
```
**When to use:** Any library or application that wants
observability without coupling to a specific monitoring
backend. The pattern works for database queries, HTTP
requests, background jobs, cache operations.
**When NOT to use:** Ultra-hot paths where telemetry
overhead matters (millions of events/second). Use sampling
or skip entirely.
---
## Pattern: Engine Abstraction for Backend Swap
**Source:** `lib/oban/engine.ex`
**Category:** engine
**What:** Define a behaviour (`Engine`) with callbacks for
all database operations (insert, fetch, complete, etc.).
Ship multiple implementations (Basic/Inline/Lite) that swap
at config time.
**Why:** Different environments need different backends:
Postgres for production, SQLite for development, inline
(in-memory) for testing. The engine abstraction lets you
swap without changing application code.
**Example:**
```elixir
@callback init(conf, opts) :: {:ok, meta} | {:error, term}
@callback insert_job(conf, changeset, opts) :: {:ok, Job.t()}
@callback fetch_jobs(conf, meta, opts) :: {:ok, {meta, [Job.t()]}}
@callback complete_job(conf, Job.t()) :: :ok
```
**When to use:** When your system needs to support multiple
storage backends, or when testing requires a fundamentally
different execution model (synchronous vs async).
**When NOT to use:** Single-backend applications. The
abstraction layer adds complexity that is only justified
when you actually swap implementations.
---
## Pattern: Keyword Validation with Reduce-While
**Source:** `lib/oban/validation.ex`
**Category:** config
**What:** Validate keyword options by iterating with
`Enum.reduce_while/3` and a validator function. Stop at
first error. Return `:ok` or `{:error, reason}`.
**Why:** Keyword lists are the standard Elixir config
format. Validating them procedurally (nested if/case) gets
messy. The reduce-while + validator pattern is composable:
each option validates independently, errors short-circuit,
and the validator function can be swapped or extended.
**Example:**
```elixir
def validate(opts, validator) when is_list(opts) do
Enum.reduce_while(opts, :ok, fn opt, acc ->
case validator.(opt) do
:ok -> {:cont, acc}
{:error, _} = error -> {:halt, error}
end
end)
end
```
**When to use:** Any public API that accepts keyword
options from users. Libraries, GenServer init, plugin
configs.
**When NOT to use:** Internal functions where the caller
is trusted. Also avoid for deeply nested configs — use
schema-based validation (NimbleOptions, Ecto embedded
schemas) instead.
---
## Pattern: Testing Mode Toggle
**Source:** `lib/oban/testing.ex`, `lib/oban/config.ex`
**Category:** testing
**What:** Support a `testing:` config option that switches
execution mode: `:disabled` (production), `:inline`
(execute immediately in caller process), `:manual` (enqueue
but don't execute — assert on DB state).
**Why:** Background job systems are inherently async, which
makes testing hard. The mode toggle gives you: (1) inline
for unit tests that need synchronous execution, (2) manual
for integration tests that verify enqueueing without
side effects.
**Example:**
```elixir
# In test config:
config :my_app, Oban, testing: :manual
# In tests:
use Oban.Testing, repo: MyApp.Repo
perform_job(MyWorker, %{id: 1})
assert_enqueued worker: MyWorker, args: %{id: 1}
```
**When to use:** Any async system that needs deterministic
testing — job queues, event buses, notification systems.
The testing mode replaces "sleep and hope" with explicit
control.
**When NOT to use:** Synchronous systems that are already
deterministic. Also avoid if the mode toggle leaks into
production code paths (keep it config-only, not conditional
logic scattered through business code).
---
## Pattern: Stopper for Goroutine Lifecycle (CockroachDB)
**Source:** `pkg/util/stop/stopper.go` (cockroachdb)
**Category:** concurrency
**What:** A dedicated struct that manages the lifecycle of
all goroutines in a component: tracks active tasks, refuses
new work during shutdown (quiesce), waits for completion,
then runs closers.
**Why:** In distributed systems, clean shutdown is critical.
You need to: (1) stop accepting new work, (2) finish
in-flight work, (3) release resources in order. The Stopper
centralizes this instead of scattering shutdown logic across
every goroutine.
**Example:**
```go
type Stopper struct {
quiescer chan struct{} // closed when quiescing
stopped chan struct{} // closed when fully stopped
mu struct {
syncutil.RWMutex
_numTasks int32
quiescing, stopping bool
closers []Closer
}
}
// RunAsyncTask refuses new work during quiesce
func (s *Stopper) RunAsyncTask(ctx context.Context,
taskName string, f func(context.Context)) error {
if !s.addTask() {
return ErrUnavailable
}
go func() {
defer s.decTask()
f(ctx)
}()
return nil
}
```
**When to use:** Any server or subsystem that spawns
goroutines and needs graceful shutdown. Especially in
long-running services where leaked goroutines cause
resource exhaustion.
**When NOT to use:** Simple programs with a single main
goroutine. Or when `errgroup` with context cancellation
suffices for the shutdown coordination.
---
## Pattern: Atomic File Operations with Suffix Convention
**Source:** `tsdb/db.go` (prometheus)
**Category:** storage
**What:** Use directory suffixes (`.tmp-for-creation`,
`.tmp-for-deletion`) to make multi-step file operations
crash-safe. On startup, clean up any dirs with these
suffixes (they represent incomplete operations).
**Why:** Database storage needs atomicity. If the process
crashes between creating a block and finalizing it, you
need to know the block is incomplete. The suffix convention
makes incomplete state visible at the filesystem level
without requiring a separate journal.
**Example:**
```go
const (
tmpForDeletionBlockDirSuffix = ".tmp-for-deletion"
tmpForCreationBlockDirSuffix = ".tmp-for-creation"
)
// On startup: remove any .tmp-* dirs (incomplete ops)
// On create: write to dir.tmp-for-creation, then rename
// On delete: rename to dir.tmp-for-deletion, then remove
```
**When to use:** Any system that manages files/directories
and needs crash consistency without a full WAL. Simpler
than a write-ahead log for coarse-grained operations.
**When NOT to use:** When you already have a WAL or
transaction log. Or for fine-grained operations where
rename semantics are insufficient.
---
## Pattern: Options as DefaultOptions() + Override
**Source:** `tsdb/db.go` (prometheus)
**Category:** configuration
**What:** Provide a `DefaultOptions()` function returning a
fully-populated config struct. Users copy and override only
what they need. No nil-means-default ambiguity.
**Why:** Large config structs (20+ fields) are unwieldy.
By providing sane defaults as a function (not a package-
level var), you avoid mutation bugs and make it clear what
"normal" looks like. Users only specify deviations.
**Example:**
```go
func DefaultOptions() *Options {
return &Options{
WALSegmentSize: wlog.DefaultSegmentSize,
RetentionDuration: int64(15 * 24 * time.Hour / ...),
MinBlockDuration: DefaultBlockDuration,
MaxBlockDuration: DefaultBlockDuration,
SamplesPerChunk: DefaultSamplesPerChunk,
// ... 20 more fields with sane defaults
}
}
// Usage:
opts := tsdb.DefaultOptions()
opts.RetentionDuration = 30 * 24 * time.Hour
db, err := tsdb.Open(dir, nil, nil, opts, nil)
```
**When to use:** Config structs with many fields where most
users want defaults. Especially when zero-value semantics
would be confusing (e.g., 0 retention = infinite? or off?).
**When NOT to use:** Small configs (3-4 fields) where
struct literal with zero-means-default is clear enough.
---
## Pattern: Scrape Loop with Aligned Timestamps
**Source:** `scrape/scrape.go` (prometheus)
**Category:** concurrency
**What:** Periodic scrape loops that align timestamps to
intervals with a small tolerance, enabling better storage
compression downstream.
**Why:** Time-series databases compress better when
timestamps are regular. A 2ms tolerance on alignment
means scraped data aligns to the expected grid while
accommodating real-world jitter.
**Example:**
```go
var ScrapeTimestampTolerance = 2 * time.Millisecond
var AlignScrapeTimestamps = true
// In scrape loop: if scrape finishes within tolerance
// of the expected timestamp, snap to the grid
```
**When to use:** Any periodic data collection where
downstream storage benefits from timestamp regularity.
Metrics, heartbeats, polling loops.
**When NOT to use:** Event-driven data where timestamps
must reflect actual occurrence time. Audit logs, user
actions, financial transactions.
<!-- PATTERN_COMPLETE -->