10218813d3
- Add ## Contents and ## Decision Tree to all 10 existing pattern files - Fix embed_as/1 semantics inversion in types.md (:self → :dump) - Fix fabricated __meta__.changes reference in changesets.md - Fix default primary key type (:integer → :id) in schemas.md - Combine @impl subsections into single "Minimal Callback Annotation"
894 lines
30 KiB
Markdown
894 lines
30 KiB
Markdown
# Ecto Schema Patterns
|
|
|
|
Patterns extracted from `lib/ecto/schema.ex` in the Ecto source.
|
|
|
|
## Contents
|
|
|
|
1. [Base Schema Module — App-Wide Schema Defaults](#1-base-schema-module--app-wide-schema-defaults)
|
|
2. [`@primary_key false` — Composite or No Primary Key](#2-primary_key-false--composite-or-no-primary-key)
|
|
3. [Virtual Fields — In-Memory-Only Data](#3-virtual-fields--in-memory-only-data)
|
|
4. [`embedded_schema/1` — Schemaless Validation Structs](#4-embedded_schema1--schemaless-validation-structs)
|
|
5. [`@timestamps_opts` — Consistent Timestamp Types](#5-timestamps_opts--consistent-timestamp-types)
|
|
6. [Field `:source` Option — Column Name Mapping](#6-field-source-option--column-name-mapping)
|
|
7. [`redact: true` — Protecting Sensitive Fields](#7-redact-true--protecting-sensitive-fields)
|
|
8. [`__schema__/1` Reflection — Runtime Schema Introspection](#8-__schema__1-reflection--runtime-schema-introspection)
|
|
|
|
---
|
|
|
|
## 1. Base Schema Module — App-Wide Schema Defaults
|
|
|
|
**Source:** [lib/ecto/schema.ex#L194](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L194)
|
|
|
|
```elixir
|
|
defmodule MyApp.Schema do
|
|
defmacro __using__(_) do
|
|
quote do
|
|
use Ecto.Schema
|
|
@primary_key {:id, :binary_id, autogenerate: true}
|
|
@foreign_key_type :binary_id
|
|
end
|
|
end
|
|
end
|
|
|
|
defmodule MyApp.Comment do
|
|
use MyApp.Schema
|
|
schema "comments" do
|
|
belongs_to :post, MyApp.Post
|
|
end
|
|
end
|
|
```
|
|
|
|
**Why:** `@primary_key` and `@foreign_key_type` are module attributes that must be set before each `schema/2` block. Without a shared base module, every schema in the application must repeat these two lines. One forgotten schema silently gets integer `:id` columns and integer foreign keys — a type mismatch that only surfaces at runtime when joins or associations break.
|
|
|
|
**Anti-pattern:** Setting `@primary_key` in every schema individually:
|
|
```elixir
|
|
# BAD — duplicated in every schema, easy to miss in a new module
|
|
defmodule MyApp.Comment do
|
|
use Ecto.Schema
|
|
@primary_key {:id, :binary_id, autogenerate: true}
|
|
@foreign_key_type :binary_id
|
|
schema "comments" do
|
|
belongs_to :post, MyApp.Post
|
|
end
|
|
end
|
|
|
|
defmodule MyApp.Post do
|
|
use Ecto.Schema
|
|
# Forgot to set @primary_key — now uses integer :id
|
|
schema "posts" do
|
|
has_many :comments, MyApp.Comment # Association type mismatch
|
|
end
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Application uses UUID (`:binary_id`) primary keys
|
|
- Application has more than one schema module
|
|
- You want all future schemas to inherit the same defaults without manual setup
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
defmodule MyApp.User do
|
|
use Ecto.Schema
|
|
@primary_key {:id, :binary_id, autogenerate: true}
|
|
@foreign_key_type :binary_id
|
|
schema "users" do
|
|
field :name, :string
|
|
end
|
|
end
|
|
|
|
defmodule MyApp.Post do
|
|
use Ecto.Schema
|
|
@primary_key {:id, :binary_id, autogenerate: true}
|
|
@foreign_key_type :binary_id
|
|
schema "posts" do
|
|
belongs_to :user, MyApp.User
|
|
end
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
# lib/my_app/schema.ex — define once
|
|
defmodule MyApp.Schema do
|
|
defmacro __using__(_) do
|
|
quote do
|
|
use Ecto.Schema
|
|
@primary_key {:id, :binary_id, autogenerate: true}
|
|
@foreign_key_type :binary_id
|
|
end
|
|
end
|
|
end
|
|
|
|
# All schemas use the base module
|
|
defmodule MyApp.User do
|
|
use MyApp.Schema
|
|
schema "users" do
|
|
field :name, :string
|
|
end
|
|
end
|
|
|
|
defmodule MyApp.Post do
|
|
use MyApp.Schema
|
|
schema "posts" do
|
|
belongs_to :user, MyApp.User
|
|
end
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The application uses integer primary keys (Ecto's default) — no base module needed
|
|
- Different schemas intentionally use different primary key types
|
|
- You only have one schema (the base module adds indirection for no gain)
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Unnecessary when using integer primary keys (the Ecto default)
|
|
defmodule MyApp.Schema do
|
|
defmacro __using__(_) do
|
|
quote do
|
|
use Ecto.Schema
|
|
@primary_key {:id, :id, autogenerate: true} # Same as the default (:id resolves to integer)
|
|
@foreign_key_type :id
|
|
end
|
|
end
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
# Just use Ecto.Schema directly when the defaults are fine
|
|
defmodule MyApp.Post do
|
|
use Ecto.Schema
|
|
schema "posts" do
|
|
field :title, :string
|
|
end
|
|
end
|
|
```
|
|
|
|
**Why:** The base module pattern exists to override Ecto defaults, not to mirror them. When integer primary keys are acceptable, the base module adds a layer of indirection without adding any value.
|
|
|
|
---
|
|
|
|
## 2. `@primary_key false` — Composite or No Primary Key
|
|
|
|
**Source:** [lib/ecto/schema.ex#L1525](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L1525)
|
|
|
|
```elixir
|
|
defmodule PostTag do
|
|
use Ecto.Schema
|
|
@primary_key false
|
|
schema "posts_tags" do
|
|
belongs_to :post, Post
|
|
belongs_to :tag, Tag
|
|
end
|
|
end
|
|
```
|
|
|
|
**Why:** By default, Ecto injects an `:id` field as the first field in every schema. For join tables with composite primary keys — or tables that intentionally have no primary key — this auto-injected field causes problems: migrations add a spurious column, queries include it, and Ecto's identity tracking operates on the wrong key. Setting `@primary_key false` disables the auto-injection entirely, giving the schema full control over which fields exist.
|
|
|
|
**Anti-pattern:** Defining a join table schema with the default `:id` field:
|
|
```elixir
|
|
# BAD — Ecto injects :id, which doesn't exist in posts_tags
|
|
defmodule PostTag do
|
|
use Ecto.Schema
|
|
schema "posts_tags" do
|
|
belongs_to :post, Post
|
|
belongs_to :tag, Tag
|
|
end
|
|
end
|
|
# Migration adds an :id column that the DB or application doesn't need.
|
|
# Ecto load/insert operations will reference a non-meaningful primary key.
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- The table is a join/association table with a composite primary key (e.g., `post_id + tag_id`)
|
|
- The table has no meaningful single-column primary key at all
|
|
- You're mapping a legacy table that lacks a primary key column
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
# posts_tags table has only (post_id, tag_id) — no separate :id column
|
|
defmodule PostTag do
|
|
use Ecto.Schema
|
|
schema "posts_tags" do
|
|
belongs_to :post, Post # generates post_id
|
|
belongs_to :tag, Tag # generates tag_id
|
|
timestamps()
|
|
end
|
|
end
|
|
# Ecto injects :id — but the DB column doesn't exist, causing query errors.
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
defmodule PostTag do
|
|
use Ecto.Schema
|
|
@primary_key false
|
|
schema "posts_tags" do
|
|
belongs_to :post, Post
|
|
belongs_to :tag, Tag
|
|
timestamps()
|
|
end
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The table has a legitimate single-column primary key (use the default or `@primary_key {:id, :binary_id, autogenerate: true}`)
|
|
- You want a composite primary key but still need Ecto's `Repo.get/2` and identity-based operations — those rely on a single primary key field
|
|
- The schema is used with `Repo.get/2` or `Repo.update/2`, which require a primary key
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Disabling primary key on a regular entity schema
|
|
defmodule MyApp.User do
|
|
use Ecto.Schema
|
|
@primary_key false # Users need a primary key for Repo.get/2
|
|
schema "users" do
|
|
field :email, :string
|
|
field :name, :string
|
|
end
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
defmodule MyApp.User do
|
|
use Ecto.Schema
|
|
schema "users" do # Default :id primary key is correct here
|
|
field :email, :string
|
|
field :name, :string
|
|
end
|
|
end
|
|
```
|
|
|
|
**Why:** `@primary_key false` removes Ecto's ability to uniquely identify rows. `Repo.get/2`, `Repo.update/2`, and association loading all depend on a primary key. Use it only for tables that are genuinely identified by composite keys or have no identity concept.
|
|
|
|
---
|
|
|
|
## 3. Virtual Fields — In-Memory-Only Data
|
|
|
|
**Source:** [lib/ecto/schema.ex#L332](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L332)
|
|
|
|
```elixir
|
|
schema "users" do
|
|
field :password_hash, :string
|
|
field :password, :string, virtual: true
|
|
field :delete, :boolean, virtual: true
|
|
end
|
|
```
|
|
|
|
**Why:** Virtual fields live on the struct and participate in changesets — they can be `cast`, `validate_required`, and read in changeset functions — but Ecto never attempts to load or persist them. This is exactly right for data that only exists for the duration of an operation: a plaintext password for hashing, a confirmation field, a checkbox that signals intent to delete, or a computed display value assembled in application code.
|
|
|
|
**Anti-pattern:** Storing transient data in a map or separate variable passed alongside the changeset:
|
|
```elixir
|
|
# BAD — passing raw password alongside changeset is error-prone
|
|
def register_user(params) do
|
|
changeset = User.changeset(%User{}, params)
|
|
password = Map.get(params, "password") # Not validated, not in changeset pipeline
|
|
if changeset.valid?, do: hash_and_insert(changeset, password)
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- A field is needed in changeset validation but must never be persisted (passwords, confirmation fields)
|
|
- A value is computed in application code and attached to the struct for rendering but has no DB column
|
|
- A form sends a signal (e.g., a "delete" checkbox) that controls behavior rather than being stored
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
# Confirmation field handled outside the changeset pipeline
|
|
def create_account(params) do
|
|
password = params["password"]
|
|
confirm = params["password_confirmation"]
|
|
|
|
if password != confirm do
|
|
{:error, "passwords do not match"}
|
|
else
|
|
User.changeset(%User{}, params) |> Repo.insert()
|
|
end
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
defmodule User do
|
|
use Ecto.Schema
|
|
import Ecto.Changeset
|
|
|
|
schema "users" do
|
|
field :email, :string
|
|
field :password_hash, :string
|
|
field :password, :string, virtual: true
|
|
field :password_confirmation, :string, virtual: true
|
|
end
|
|
|
|
def registration_changeset(user, params) do
|
|
user
|
|
|> cast(params, [:email, :password, :password_confirmation])
|
|
|> validate_required([:email, :password])
|
|
|> validate_confirmation(:password)
|
|
|> hash_password()
|
|
end
|
|
|
|
defp hash_password(%{valid?: true, changes: %{password: pw}} = changeset) do
|
|
put_change(changeset, :password_hash, Bcrypt.hash_pwd_salt(pw))
|
|
end
|
|
defp hash_password(changeset), do: changeset
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The value should be persisted and loaded from the DB (use a regular field)
|
|
- The value needs to participate in Ecto queries (`where`, `order_by`) — virtual fields cannot be queried
|
|
- The field has a `:default` that should be the DB column default — virtual field defaults only affect the struct, not the database
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Using virtual for a value that should live in the DB
|
|
schema "articles" do
|
|
field :title, :string
|
|
field :slug, :string, virtual: true # Wrong — slug needs to be persisted and queried
|
|
field :body, :string
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
schema "articles" do
|
|
field :title, :string
|
|
field :slug, :string # Regular field — persisted and queryable
|
|
field :body, :string
|
|
end
|
|
```
|
|
|
|
**Why:** Virtual fields are intentionally invisible to the database layer. Any field that needs to survive a page reload, be searched, sorted, or returned in a list query must be a real column.
|
|
|
|
---
|
|
|
|
## 4. `embedded_schema/1` — Schemaless Validation Structs
|
|
|
|
**Source:** [lib/ecto/schema.ex#L110](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L110)
|
|
|
|
```elixir
|
|
defmodule Search do
|
|
use Ecto.Schema
|
|
import Ecto.Changeset
|
|
|
|
embedded_schema do
|
|
field :query, :string
|
|
field :page, :integer, default: 1
|
|
field :per_page, :integer, default: 20
|
|
end
|
|
|
|
def changeset(search, params) do
|
|
search
|
|
|> cast(params, [:query, :page, :per_page])
|
|
|> validate_required([:query])
|
|
|> validate_number(:page, greater_than: 0)
|
|
end
|
|
end
|
|
```
|
|
|
|
**Why:** `embedded_schema` defines a schema module with no backing DB table. The struct, changeset pipeline, and all validations work identically to a regular schema, but there is no `Repo` involved. This makes it the right tool for validating and casting data that doesn't map to a table: search/filter forms, API request bodies, command parameters, multi-step wizard state. A `Search.changeset/2` call returns a changeset with errors just like `User.changeset/2` does — the same controller and form helper code works for both.
|
|
|
|
**Anti-pattern:** Validating formless data with raw `Map` operations and ad-hoc checks:
|
|
```elixir
|
|
# BAD — manual validation without changeset pipeline
|
|
def search(params) do
|
|
query = Map.get(params, "query")
|
|
page = Map.get(params, "page", "1") |> String.to_integer()
|
|
|
|
if is_nil(query) or query == "" do
|
|
{:error, "query is required"}
|
|
else
|
|
{:ok, %{query: query, page: page}}
|
|
end
|
|
# No type coercion, no error accumulation, no `valid?` flag
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- A form or API endpoint needs validated, typed input but no persistence
|
|
- You want `Ecto.Changeset` error formatting and `valid?` semantics without a Repo
|
|
- Multi-step wizard where intermediate state needs validation before DB write
|
|
- An external API receives a complex payload that needs casting and validation
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
# Validating filter params without Ecto — verbose and fragile
|
|
def filter_posts(params) do
|
|
status = Map.get(params, "status")
|
|
valid_statuses = ["draft", "published", "archived"]
|
|
|
|
cond do
|
|
status not in valid_statuses ->
|
|
{:error, "status must be one of: #{Enum.join(valid_statuses, ", ")}"}
|
|
true ->
|
|
Post |> where(status: ^status) |> Repo.all() |> then(&{:ok, &1})
|
|
end
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
defmodule PostFilter do
|
|
use Ecto.Schema
|
|
import Ecto.Changeset
|
|
|
|
embedded_schema do
|
|
field :status, Ecto.Enum, values: [:draft, :published, :archived]
|
|
field :page, :integer, default: 1
|
|
end
|
|
|
|
def changeset(filter \\ %PostFilter{}, params) do
|
|
filter
|
|
|> cast(params, [:status, :page])
|
|
|> validate_required([:status])
|
|
|> validate_number(:page, greater_than: 0)
|
|
end
|
|
end
|
|
|
|
def filter_posts(params) do
|
|
with %{valid?: true} = cs <- PostFilter.changeset(params),
|
|
filter <- Ecto.Changeset.apply_changes(cs) do
|
|
Post |> where(status: ^filter.status) |> Repo.all() |> then(&{:ok, &1})
|
|
else
|
|
cs -> {:error, cs}
|
|
end
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The data needs to be persisted — use a regular `schema/2` with a migration
|
|
- The validation logic is trivial (one or two fields) — a plain `cast` + `validate_required` in a context module may be sufficient
|
|
- You need DB-level constraints or uniqueness checks — `embedded_schema` has no Repo access
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Overkill for a single required string field
|
|
defmodule SearchQuery do
|
|
use Ecto.Schema
|
|
embedded_schema do
|
|
field :q, :string
|
|
end
|
|
def changeset(s, p), do: s |> cast(p, [:q]) |> validate_required([:q])
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
# Simple inline validation is clearer for trivial cases
|
|
def search(params) do
|
|
case Map.fetch(params, "q") do
|
|
{:ok, q} when q != "" -> {:ok, q}
|
|
_ -> {:error, :missing_query}
|
|
end
|
|
end
|
|
```
|
|
|
|
**Why:** `embedded_schema` earns its keep when you have multiple fields, type coercion, or multiple validators — the changeset pipeline pays for itself. For a single field with one constraint, the ceremony outweighs the benefit.
|
|
|
|
---
|
|
|
|
## 5. `@timestamps_opts` — Consistent Timestamp Types
|
|
|
|
**Source:** [lib/ecto/schema.ex#L170](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L170)
|
|
|
|
```elixir
|
|
defmodule MyApp.Schema do
|
|
defmacro __using__(_) do
|
|
quote do
|
|
use Ecto.Schema
|
|
@timestamps_opts [type: :utc_datetime_usec]
|
|
end
|
|
end
|
|
end
|
|
```
|
|
|
|
**Why:** The `timestamps()` macro inserts `inserted_at` and `updated_at` fields whose type is controlled by `@timestamps_opts`. The default type is `:naive_datetime`, which stores timestamps with no timezone information. Applications that store or compare timestamps across timezones need `:utc_datetime` or `:utc_datetime_usec`. Microsecond precision (`:utc_datetime_usec`) avoids silent truncation when high-resolution timestamps are generated in Elixir and stored in PostgreSQL (which supports microsecond precision). Setting this in the base module ensures every schema uses the same timestamp type.
|
|
|
|
**Anti-pattern:** Using the default `naive_datetime` timestamps when UTC compliance or precision is required:
|
|
```elixir
|
|
# BAD — naive_datetime drops timezone context
|
|
defmodule MyApp.Post do
|
|
use Ecto.Schema
|
|
schema "posts" do
|
|
field :title, :string
|
|
timestamps() # inserts inserted_at/updated_at as :naive_datetime
|
|
end
|
|
end
|
|
# Timestamps stored as "2024-01-15 10:30:00" — no UTC indicator.
|
|
# Comparisons and serialization can silently produce wrong results.
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Application stores or compares timestamps across timezones (almost always true for web apps)
|
|
- Timestamps need to be serialized to ISO 8601 / JSON with timezone information
|
|
- PostgreSQL or another DB with microsecond precision should not lose sub-second data
|
|
- Multiple schemas share the same timestamp behavior
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
# Each schema repeats the option
|
|
defmodule MyApp.User do
|
|
use Ecto.Schema
|
|
schema "users" do
|
|
timestamps(type: :utc_datetime_usec)
|
|
end
|
|
end
|
|
|
|
defmodule MyApp.Post do
|
|
use Ecto.Schema
|
|
schema "posts" do
|
|
timestamps(type: :utc_datetime_usec)
|
|
end
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
# Base module sets the default once
|
|
defmodule MyApp.Schema do
|
|
defmacro __using__(_) do
|
|
quote do
|
|
use Ecto.Schema
|
|
@primary_key {:id, :binary_id, autogenerate: true}
|
|
@foreign_key_type :binary_id
|
|
@timestamps_opts [type: :utc_datetime_usec]
|
|
end
|
|
end
|
|
end
|
|
|
|
defmodule MyApp.User do
|
|
use MyApp.Schema
|
|
schema "users" do
|
|
timestamps() # uses :utc_datetime_usec from base module
|
|
end
|
|
end
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The application intentionally avoids timezones (internal tooling, batch jobs)
|
|
- The DB column type is `timestamp without time zone` and you want to preserve that semantic
|
|
- Only one or two schemas use timestamps and the option is set inline
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Setting timestamps_opts for a schema that has no timestamps() call
|
|
defmodule PostTag do
|
|
use Ecto.Schema
|
|
@primary_key false
|
|
@timestamps_opts [type: :utc_datetime_usec] # Never used — no timestamps() call
|
|
schema "posts_tags" do
|
|
belongs_to :post, Post
|
|
belongs_to :tag, Tag
|
|
end
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
# The option is harmless but unnecessary — omit it when there are no timestamps
|
|
defmodule PostTag do
|
|
use MyApp.Schema # Base module sets it — no need to repeat
|
|
@primary_key false
|
|
schema "posts_tags" do
|
|
belongs_to :post, Post
|
|
belongs_to :tag, Tag
|
|
end
|
|
end
|
|
```
|
|
|
|
**Why:** `@timestamps_opts` only takes effect when `timestamps()` is called inside a `schema` block. Setting it on schemas that call `timestamps()` is correct; setting it on schemas that don't is dead code.
|
|
|
|
---
|
|
|
|
## 6. Field `:source` Option — Column Name Mapping
|
|
|
|
**Source:** [lib/ecto/schema.ex](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex)
|
|
|
|
```elixir
|
|
schema "legacy_users" do
|
|
field :email_address, :string, source: :emailaddress
|
|
field :created_at, :utc_datetime, source: :creation_timestamp
|
|
end
|
|
```
|
|
|
|
**Why:** The `:source` option on `field/3` maps an Elixir field name to a different database column name. This is the escape hatch for working with databases that don't follow Elixir naming conventions — legacy systems with camelCase or abbreviated column names, databases shared with other applications, or tables generated by external tools. The Elixir struct uses the field name (`:email_address`); SQL uses the source name (`emailaddress`). Ecto translates transparently in both directions — `select`, `where`, and `insert` all use the column name.
|
|
|
|
**Anti-pattern:** Mapping column names by wrapping all query fragments in raw SQL or using `fragment/1` everywhere:
|
|
```elixir
|
|
# BAD — forced to use raw column names in every query
|
|
from u in "legacy_users",
|
|
where: fragment("emailaddress = ?", ^email),
|
|
select: %{email: u.emailaddress, created_at: u.creation_timestamp}
|
|
# Loses type casting, association loading, and changeset integration
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- The database column name cannot be changed (shared DB, legacy system, migration risk)
|
|
- The column name violates Elixir conventions (camelCase, abbreviations, reserved words)
|
|
- You want Ecto's type casting and query DSL to work with idiomatic Elixir field names
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
# Schema mirrors the ugly DB column names directly
|
|
schema "legacy_users" do
|
|
field :emailaddress, :string
|
|
field :creation_timestamp, :utc_datetime
|
|
field :lastlogindt, :utc_datetime
|
|
end
|
|
|
|
# Every callsite uses the ugly names
|
|
from u in User, where: u.emailaddress == ^email
|
|
%{emailaddress: user.emailaddress, created: user.creation_timestamp}
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
schema "legacy_users" do
|
|
field :email, :string, source: :emailaddress
|
|
field :inserted_at, :utc_datetime, source: :creation_timestamp
|
|
field :last_login_at, :utc_datetime, source: :lastlogindt
|
|
end
|
|
|
|
# Callsites use clean Elixir names
|
|
from u in User, where: u.email == ^email
|
|
%{email: user.email, created: user.inserted_at}
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- You control the database schema — use a migration to rename the column instead
|
|
- The column name follows Elixir conventions already — `:source` adds no value
|
|
- You're creating a new table — design the column names correctly from the start
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Renaming columns that are already idiomatic
|
|
schema "users" do
|
|
field :user_name, :string, source: :username # username is fine as-is
|
|
field :created_at_time, :utc_datetime, source: :inserted_at # just use inserted_at
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
schema "users" do
|
|
field :username, :string # Keep the DB name if it's already clear
|
|
timestamps() # inserted_at / updated_at are already conventional
|
|
end
|
|
```
|
|
|
|
**Why:** `:source` is a mapping layer between two names that creates a permanent translation cost every time someone reads the schema. When you control the DB, eliminate the mismatch at the migration level rather than carrying it forever in the schema.
|
|
|
|
---
|
|
|
|
## 7. `redact: true` — Protecting Sensitive Fields
|
|
|
|
**Source:** [lib/ecto/schema.ex#L128](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L128)
|
|
|
|
```elixir
|
|
schema "users" do
|
|
field :email, :string
|
|
field :password_hash, :string, redact: true
|
|
field :api_token, :string, redact: true
|
|
end
|
|
```
|
|
|
|
**Why:** Ecto derives an `Inspect` implementation for every schema struct. Without `redact: true`, inspecting a changeset or struct in logs, IEx, or crash reports prints every field value in plain text. Marking a field with `redact: true` replaces its value with `**redacted**` in `inspect/2` output and in `changeset.changes` inspection. This is a passive, always-on protection that requires no additional code at each log site — once set on the field, the field is protected everywhere the struct is printed.
|
|
|
|
**Anti-pattern:** Relying on callers to manually omit sensitive fields before logging:
|
|
```elixir
|
|
# BAD — easy to forget, must be repeated at every log site
|
|
def create_user(params) do
|
|
changeset = User.changeset(%User{}, params)
|
|
# Must remember to drop :password_hash before logging
|
|
safe = Map.drop(changeset.changes, [:password_hash, :api_token])
|
|
Logger.info("Creating user: #{inspect(safe)}")
|
|
Repo.insert(changeset)
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- A field stores a secret, credential, or token (passwords, API keys, session tokens)
|
|
- A field stores PII that should not appear in logs or error reports (SSN, credit card data)
|
|
- A changeset for the schema might be logged, inspected in IEx, or appear in crash reports
|
|
- You use `Logger.info/debug` calls that could inadvertently print struct values
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
# Sensitive values appear in plain text in logs and crash dumps
|
|
defmodule User do
|
|
use Ecto.Schema
|
|
schema "users" do
|
|
field :email, :string
|
|
field :password_hash, :string # Logged as "password_hash: \"$2b$12$...\""
|
|
field :reset_token, :string # Logged as "reset_token: \"abc123secret\""
|
|
end
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
defmodule User do
|
|
use Ecto.Schema
|
|
schema "users" do
|
|
field :email, :string
|
|
field :password_hash, :string, redact: true
|
|
field :reset_token, :string, redact: true
|
|
end
|
|
end
|
|
# inspect(user) => #User<email: "alice@example.com", password_hash: **redacted**, ...>
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- The field contains non-sensitive data — redaction adds visual noise to inspect output
|
|
- You need to see the field value during development debugging (temporarily remove `redact: true` locally, or use pattern matching to extract the value directly)
|
|
- The struct is used exclusively in contexts where it is never printed (pure computation)
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Redacting non-sensitive fields makes debugging needlessly difficult
|
|
schema "products" do
|
|
field :name, :string, redact: true # Why is a product name sensitive?
|
|
field :price, :decimal, redact: true # Price data isn't a secret
|
|
field :sku, :string, redact: true
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
schema "products" do
|
|
field :name, :string
|
|
field :price, :decimal
|
|
field :sku, :string
|
|
end
|
|
```
|
|
|
|
**Why:** `redact: true` makes fields invisible to standard Elixir tooling. Overusing it on non-sensitive fields makes crash reports and debug output harder to read without adding any security benefit. Apply it precisely to fields whose values would constitute a security or privacy exposure if logged.
|
|
|
|
---
|
|
|
|
## 8. `__schema__/1` Reflection — Runtime Schema Introspection
|
|
|
|
**Source:** [lib/ecto/schema.ex#L450](https://github.com/elixir-ecto/ecto/blob/fd2ec52b5ae1f775747308f0fd9ffc160515514b/lib/ecto/schema.ex#L450)
|
|
|
|
```elixir
|
|
# Get all field names
|
|
User.__schema__(:fields) #=> [:id, :name, :email, :age]
|
|
User.__schema__(:associations) #=> [:posts, :comments]
|
|
User.__schema__(:type, :email) #=> :string
|
|
User.__schema__(:virtual_fields) #=> [:password]
|
|
```
|
|
|
|
**Why:** Every module that calls `use Ecto.Schema` gets `__schema__/1` generated at compile time. These reflection callbacks expose the schema's structure at runtime without requiring the caller to know the schema's field list at compile time. Generic code — CSV exporters, audit loggers, API serializers, test factories — can introspect any schema and work with all its fields automatically. When a new field is added to the schema, the generic code picks it up without modification.
|
|
|
|
**Anti-pattern:** Hardcoding field names in generic helpers:
|
|
```elixir
|
|
# BAD — must be updated every time any schema gains or loses a field
|
|
def audit_changes(changeset) do
|
|
fields = [:name, :email, :role] # Hardcoded — User-specific, not generic
|
|
Enum.each(fields, fn field ->
|
|
if Map.has_key?(changeset.changes, field) do
|
|
AuditLog.record(field, changeset.changes[field])
|
|
end
|
|
end)
|
|
end
|
|
```
|
|
|
|
### When to Use
|
|
|
|
**Triggers:**
|
|
- Writing a helper that must work across multiple schema modules without knowing their fields
|
|
- Building a generic CSV exporter, JSON serializer, or audit logger
|
|
- Generating test factories or seed data for any schema
|
|
- Writing a linter or validation tool that checks schema conventions
|
|
|
|
**Example — before:**
|
|
```elixir
|
|
# Must manually maintain field lists for each schema
|
|
defmodule CsvExporter do
|
|
def export_users(records) do
|
|
headers = ["id", "name", "email", "age"]
|
|
rows = Enum.map(records, fn u -> [u.id, u.name, u.email, u.age] end)
|
|
[headers | rows]
|
|
end
|
|
|
|
def export_posts(records) do
|
|
headers = ["id", "title", "body", "user_id"]
|
|
rows = Enum.map(records, fn p -> [p.id, p.title, p.body, p.user_id] end)
|
|
[headers | rows]
|
|
end
|
|
end
|
|
```
|
|
|
|
**Example — after:**
|
|
```elixir
|
|
defmodule CsvExporter do
|
|
def export(schema, records) do
|
|
fields = schema.__schema__(:fields)
|
|
headers = Enum.map(fields, &to_string/1)
|
|
rows = Enum.map(records, fn record ->
|
|
Enum.map(fields, &Map.get(record, &1))
|
|
end)
|
|
[headers | rows]
|
|
end
|
|
end
|
|
|
|
# Works for any schema without modification
|
|
CsvExporter.export(User, users)
|
|
CsvExporter.export(Post, posts)
|
|
```
|
|
|
|
### When NOT to Use
|
|
|
|
**Don't use this when:**
|
|
- You're writing code for one specific schema and the fields are known and stable — just use them directly
|
|
- You need to control exactly which fields are exposed (e.g., public API — use an explicit allowlist)
|
|
- The schema has virtual fields or sensitive fields that should not be included — reflection returns all fields regardless of `virtual: true` or `redact: true`
|
|
|
|
**Over-application example:**
|
|
```elixir
|
|
# Using reflection for a single known schema adds indirection with no benefit
|
|
def display_user(user) do
|
|
User.__schema__(:fields)
|
|
|> Enum.map(fn field -> "#{field}: #{Map.get(user, field)}" end)
|
|
|> Enum.join(", ")
|
|
# Dumps :password_hash, :reset_token, and every internal field
|
|
end
|
|
```
|
|
|
|
**Better alternative:**
|
|
```elixir
|
|
# For a known schema, use explicit fields — safer and clearer intent
|
|
def display_user(user) do
|
|
"#{user.name} <#{user.email}>"
|
|
end
|
|
```
|
|
|
|
**Why:** `__schema__(:fields)` returns all persisted fields including internal ones (`inserted_at`, `updated_at`, foreign keys). For display or serialization of a known schema, an explicit field list documents intent and prevents sensitive data from leaking. Use reflection for genuinely generic code where the alternative is a hardcoded list that diverges from the schema over time.
|
|
|
|
---
|
|
|
|
## Decision Tree
|
|
|
|
- If the app uses UUID primary keys across all schemas → create a base schema module with `@primary_key {:id, :binary_id, autogenerate: true}` and `@foreign_key_type :binary_id`
|
|
- If a table has no single primary key (join table, composite key) → `@primary_key false`
|
|
- If a field holds runtime-only data (password input, confirmation field, computed display value) → `virtual: true`
|
|
- If you need changeset validation without a DB table (form objects, API request bodies) → `embedded_schema`
|
|
- If timestamps need UTC compliance or microsecond precision → set `@timestamps_opts [type: :utc_datetime_usec]` in the base schema module
|
|
- If a DB column name doesn't match Elixir conventions and can't be migrated → `:source` option on `field/3`
|
|
- If a field holds sensitive data (passwords, tokens, PII) that must not appear in logs → `redact: true`
|
|
- If writing generic code that works across schema modules without knowing their fields → `__schema__/1` reflection
|
|
|
|
<!-- PATTERN_COMPLETE -->
|