patterns(testing): add async test filtering pattern (#21)
When testing telemetry, pub/sub, or any broadcast mechanism with async: true, events from concurrent tests can leak into mailbox. Fix: Pin-match on unique identifier to filter. Two-part pattern: 1. Filter at source — only forward events matching test's identifier 2. Pin in assertion — ^variable rejects mismatches that slip through Applies to telemetry handlers, PubSub, GenStage/Broadway consumers, any shared message bus. Triggered by PR #710 flaky test fix in gargoyle.
This commit is contained in:
@@ -1772,3 +1772,76 @@ end
|
||||
- If you want to make test dependencies on setup context explicit → [Context Pattern Matching](#20-context-pattern-matching-in-test-signatures)
|
||||
|
||||
<!-- PATTERN_COMPLETE -->
|
||||
|
||||
## 21. Filtering Events in Async Tests
|
||||
|
||||
**Source:** Gargoyle PR #710 (flaky telemetry test fix)
|
||||
|
||||
**What it does:** Pin-matches on a unique identifier to filter events from concurrent tests.
|
||||
|
||||
**Why:** When testing telemetry, pub/sub, or any broadcast mechanism with `async: true`, events from other tests can leak into your mailbox. Without filtering, tests pass in isolation but fail randomly when run in parallel.
|
||||
|
||||
**Pattern:**
|
||||
|
||||
Wrong — receives events from other tests:
|
||||
```elixir
|
||||
test "emits telemetry on create", %{user: user} do
|
||||
:telemetry.attach("test", [:user, :created], &send_to_test/4, self())
|
||||
create_post(user)
|
||||
assert_receive {:telemetry, %{user_id: uid}}
|
||||
assert uid == user.id # Might match event from another test!
|
||||
end
|
||||
```
|
||||
|
||||
Right — pin filters to only your test's events:
|
||||
```elixir
|
||||
test "emits telemetry on create", %{user: user} do
|
||||
test_pid = self()
|
||||
expected_uid = user.id
|
||||
|
||||
:telemetry.attach("test-#{inspect(test_pid)}", [:user, :created], fn _, _, meta, pid ->
|
||||
if meta.user_id == expected_uid, do: send(pid, {:telemetry, meta})
|
||||
end, test_pid)
|
||||
|
||||
create_post(user)
|
||||
assert_receive {:telemetry, %{user_id: ^expected_uid}}
|
||||
end
|
||||
```
|
||||
|
||||
### Key Insight
|
||||
|
||||
The fix happens in two places:
|
||||
|
||||
1. **Filter at the source** — only send messages that match your test's unique identifier
|
||||
2. **Pin in the assertion** — use `^variable` to reject mismatches that slip through
|
||||
|
||||
### When to Use
|
||||
|
||||
**Triggers:**
|
||||
- Telemetry handlers (filter by user_id, request_id, or test pid)
|
||||
- Phoenix.PubSub subscriptions (use unique topic per test)
|
||||
- GenStage/Broadway consumers (tag events with test pid)
|
||||
- Any shared message bus in async tests
|
||||
- Tests pass alone but fail with `--seed` or in CI
|
||||
|
||||
### When NOT to Use
|
||||
|
||||
**Don't use this when:**
|
||||
- Tests run with `async: false` (no concurrent tests to leak events)
|
||||
- You control the event source and can make it test-aware by design
|
||||
- The event contains a natural unique key you already have (just pin on it)
|
||||
|
||||
**Over-application example:**
|
||||
```elixir
|
||||
# Overkill when you already have a unique key
|
||||
test "user update", %{user: user} do
|
||||
# user.id is already unique — just pin it directly
|
||||
assert_receive {:updated, %{id: ^user.id}}
|
||||
end
|
||||
```
|
||||
|
||||
### Decision Tree Addition
|
||||
|
||||
- If async tests randomly fail due to events from other tests → [Filtering Events in Async Tests](#21-filtering-events-in-async-tests)
|
||||
|
||||
<!-- PATTERN_COMPLETE -->
|
||||
|
||||
Reference in New Issue
Block a user