From df9c856d96b56f01822db0484312d7530c0d2848 Mon Sep 17 00:00:00 2001 From: Rodin Date: Sat, 9 May 2026 18:27:05 -0700 Subject: [PATCH] patterns(testing): add async test filtering pattern (#21) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When testing telemetry, pub/sub, or any broadcast mechanism with async: true, events from concurrent tests can leak into mailbox. Fix: Pin-match on unique identifier to filter. Two-part pattern: 1. Filter at source — only forward events matching test's identifier 2. Pin in assertion — ^variable rejects mismatches that slip through Applies to telemetry handlers, PubSub, GenStage/Broadway consumers, any shared message bus. Triggered by PR #710 flaky test fix in gargoyle. --- patterns/testing.md | 73 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/patterns/testing.md b/patterns/testing.md index 5ef0486..21c0826 100644 --- a/patterns/testing.md +++ b/patterns/testing.md @@ -1772,3 +1772,76 @@ end - If you want to make test dependencies on setup context explicit → [Context Pattern Matching](#20-context-pattern-matching-in-test-signatures) + +## 21. Filtering Events in Async Tests + +**Source:** Gargoyle PR #710 (flaky telemetry test fix) + +**What it does:** Pin-matches on a unique identifier to filter events from concurrent tests. + +**Why:** When testing telemetry, pub/sub, or any broadcast mechanism with `async: true`, events from other tests can leak into your mailbox. Without filtering, tests pass in isolation but fail randomly when run in parallel. + +**Pattern:** + +Wrong — receives events from other tests: +```elixir +test "emits telemetry on create", %{user: user} do + :telemetry.attach("test", [:user, :created], &send_to_test/4, self()) + create_post(user) + assert_receive {:telemetry, %{user_id: uid}} + assert uid == user.id # Might match event from another test! +end +``` + +Right — pin filters to only your test's events: +```elixir +test "emits telemetry on create", %{user: user} do + test_pid = self() + expected_uid = user.id + + :telemetry.attach("test-#{inspect(test_pid)}", [:user, :created], fn _, _, meta, pid -> + if meta.user_id == expected_uid, do: send(pid, {:telemetry, meta}) + end, test_pid) + + create_post(user) + assert_receive {:telemetry, %{user_id: ^expected_uid}} +end +``` + +### Key Insight + +The fix happens in two places: + +1. **Filter at the source** — only send messages that match your test's unique identifier +2. **Pin in the assertion** — use `^variable` to reject mismatches that slip through + +### When to Use + +**Triggers:** +- Telemetry handlers (filter by user_id, request_id, or test pid) +- Phoenix.PubSub subscriptions (use unique topic per test) +- GenStage/Broadway consumers (tag events with test pid) +- Any shared message bus in async tests +- Tests pass alone but fail with `--seed` or in CI + +### When NOT to Use + +**Don't use this when:** +- Tests run with `async: false` (no concurrent tests to leak events) +- You control the event source and can make it test-aware by design +- The event contains a natural unique key you already have (just pin on it) + +**Over-application example:** +```elixir +# Overkill when you already have a unique key +test "user update", %{user: user} do + # user.id is already unique — just pin it directly + assert_receive {:updated, %{id: ^user.id}} +end +``` + +### Decision Tree Addition + +- If async tests randomly fail due to events from other tests → [Filtering Events in Async Tests](#21-filtering-events-in-async-tests) + +