New analytical lens examining implicit assumptions about broker APIs,
market behavior, network conditions, and timing.
Document: gargoyle's feeds-and-instruments.md (115 lines)
Models: GPT-5 (24 findings), Opus (15), Sonnet (15)
Key insight: External system assumptions benefit more from reasoning
depth than internal architecture analysis. GPT-5's exhaustive coverage
of broker implementation details and network failure modes justifies
the token cost for critical external interfaces.
Union of all models finds ~30 distinct assumptions vs ~24 max single model.
Tested on signal-lifecycle.md (111 lines). Results:
- GPT-5: 17 gaps (7,744 reasoning tokens)
- Opus: 11 gaps (design-level focus)
- Sonnet: 8 gaps (fastest, protocol-level)
Key insight: Union of all models (~26 gaps) far exceeds any single
model (max 17). Only 5 gaps found by all three — highly differentiated
outputs make multi-model runs valuable for interface documents.
New task type: analyzing prescriptive/specification documents for completeness.
- GPT-5 dominates with exhaustive enumeration (34 findings)
- Opus traces gaps to consequences (routing failures, compiler issues)
- Sonnet surface-level (not recommended for thorough analysis)
Key insight: GPT-5 found internal contradiction (telemetry verb rule vs example)
that neither Claude model caught. Opus unique in tracing PubSub collision
to actual routing failure scenario.
Task taxonomy: convention gap analysis follows same pattern as architecture
docs - GPT-5 for coverage, Opus for consequences.
GPT-5 finds 16 gaps, Opus 11, Sonnet 9. GPT-5 excels at exhaustive
state space enumeration; Opus finds convention-vs-enforcement gaps;
Sonnet adequate but less thorough.
Key insight: state machine completeness is a GPT-5 sweet spot due to
reasoning tokens enabling systematic combinatorial coverage.
Tests a novel lens for event-sourced architectures: can all state be
reconstructed from documented events alone?
Key findings:
- GPT-5 brings external domain knowledge (broker APIs, compliance)
- Opus reasons through failure modes systematically (crash boundaries)
- Sonnet does rapid structural analysis (missing pieces)
21 unique findings across three models with only 5 in common.
Each model's reasoning style reveals different issue categories.
New pattern: event flow analysis exposes model reasoning styles
that gap-finding and contradiction detection don't surface.
Rodin