New task type: analyzing prescriptive/specification documents for completeness.
- GPT-5 dominates with exhaustive enumeration (34 findings)
- Opus traces gaps to consequences (routing failures, compiler issues)
- Sonnet surface-level (not recommended for thorough analysis)
Key insight: GPT-5 found internal contradiction (telemetry verb rule vs example)
that neither Claude model caught. Opus unique in tracing PubSub collision
to actual routing failure scenario.
Task taxonomy: convention gap analysis follows same pattern as architecture
docs - GPT-5 for coverage, Opus for consequences.
GPT-5 finds 16 gaps, Opus 11, Sonnet 9. GPT-5 excels at exhaustive
state space enumeration; Opus finds convention-vs-enforcement gaps;
Sonnet adequate but less thorough.
Key insight: state machine completeness is a GPT-5 sweet spot due to
reasoning tokens enabling systematic combinatorial coverage.
Tests a novel lens for event-sourced architectures: can all state be
reconstructed from documented events alone?
Key findings:
- GPT-5 brings external domain knowledge (broker APIs, compliance)
- Opus reasons through failure modes systematically (crash boundaries)
- Sonnet does rapid structural analysis (missing pieces)
21 unique findings across three models with only 5 in common.
Each model's reasoning style reveals different issue categories.
New pattern: event flow analysis exposes model reasoning styles
that gap-finding and contradiction detection don't surface.
Rodin