patterns(testing): add async test filtering pattern (#21)
When testing telemetry, pub/sub, or any broadcast mechanism with async: true, events from concurrent tests can leak into mailbox. Fix: Pin-match on unique identifier to filter. Two-part pattern: 1. Filter at source — only forward events matching test's identifier 2. Pin in assertion — ^variable rejects mismatches that slip through Applies to telemetry handlers, PubSub, GenStage/Broadway consumers, any shared message bus. Triggered by PR #710 flaky test fix in gargoyle.
This commit is contained in:
@@ -1772,3 +1772,76 @@ end
|
|||||||
- If you want to make test dependencies on setup context explicit → [Context Pattern Matching](#20-context-pattern-matching-in-test-signatures)
|
- If you want to make test dependencies on setup context explicit → [Context Pattern Matching](#20-context-pattern-matching-in-test-signatures)
|
||||||
|
|
||||||
<!-- PATTERN_COMPLETE -->
|
<!-- PATTERN_COMPLETE -->
|
||||||
|
|
||||||
|
## 21. Filtering Events in Async Tests
|
||||||
|
|
||||||
|
**Source:** Gargoyle PR #710 (flaky telemetry test fix)
|
||||||
|
|
||||||
|
**What it does:** Pin-matches on a unique identifier to filter events from concurrent tests.
|
||||||
|
|
||||||
|
**Why:** When testing telemetry, pub/sub, or any broadcast mechanism with `async: true`, events from other tests can leak into your mailbox. Without filtering, tests pass in isolation but fail randomly when run in parallel.
|
||||||
|
|
||||||
|
**Pattern:**
|
||||||
|
|
||||||
|
Wrong — receives events from other tests:
|
||||||
|
```elixir
|
||||||
|
test "emits telemetry on create", %{user: user} do
|
||||||
|
:telemetry.attach("test", [:user, :created], &send_to_test/4, self())
|
||||||
|
create_post(user)
|
||||||
|
assert_receive {:telemetry, %{user_id: uid}}
|
||||||
|
assert uid == user.id # Might match event from another test!
|
||||||
|
end
|
||||||
|
```
|
||||||
|
|
||||||
|
Right — pin filters to only your test's events:
|
||||||
|
```elixir
|
||||||
|
test "emits telemetry on create", %{user: user} do
|
||||||
|
test_pid = self()
|
||||||
|
expected_uid = user.id
|
||||||
|
|
||||||
|
:telemetry.attach("test-#{inspect(test_pid)}", [:user, :created], fn _, _, meta, pid ->
|
||||||
|
if meta.user_id == expected_uid, do: send(pid, {:telemetry, meta})
|
||||||
|
end, test_pid)
|
||||||
|
|
||||||
|
create_post(user)
|
||||||
|
assert_receive {:telemetry, %{user_id: ^expected_uid}}
|
||||||
|
end
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key Insight
|
||||||
|
|
||||||
|
The fix happens in two places:
|
||||||
|
|
||||||
|
1. **Filter at the source** — only send messages that match your test's unique identifier
|
||||||
|
2. **Pin in the assertion** — use `^variable` to reject mismatches that slip through
|
||||||
|
|
||||||
|
### When to Use
|
||||||
|
|
||||||
|
**Triggers:**
|
||||||
|
- Telemetry handlers (filter by user_id, request_id, or test pid)
|
||||||
|
- Phoenix.PubSub subscriptions (use unique topic per test)
|
||||||
|
- GenStage/Broadway consumers (tag events with test pid)
|
||||||
|
- Any shared message bus in async tests
|
||||||
|
- Tests pass alone but fail with `--seed` or in CI
|
||||||
|
|
||||||
|
### When NOT to Use
|
||||||
|
|
||||||
|
**Don't use this when:**
|
||||||
|
- Tests run with `async: false` (no concurrent tests to leak events)
|
||||||
|
- You control the event source and can make it test-aware by design
|
||||||
|
- The event contains a natural unique key you already have (just pin on it)
|
||||||
|
|
||||||
|
**Over-application example:**
|
||||||
|
```elixir
|
||||||
|
# Overkill when you already have a unique key
|
||||||
|
test "user update", %{user: user} do
|
||||||
|
# user.id is already unique — just pin it directly
|
||||||
|
assert_receive {:updated, %{id: ^user.id}}
|
||||||
|
end
|
||||||
|
```
|
||||||
|
|
||||||
|
### Decision Tree Addition
|
||||||
|
|
||||||
|
- If async tests randomly fail due to events from other tests → [Filtering Events in Async Tests](#21-filtering-events-in-async-tests)
|
||||||
|
|
||||||
|
<!-- PATTERN_COMPLETE -->
|
||||||
|
|||||||
Reference in New Issue
Block a user