python-patterns/PROCESS.md

Python Patterns Process

This file documents the workflow used to build and refine this repo so the extraction can be repeated without guesswork.

Goal

Turn repeated patterns from mature Python codebases into concise, prescriptive docs with verifiable citations.

Scope split

Keep this repo at the language/library design level.

Good fits:

• module/public-surface design
• exception design
• sync vs async API boundaries
• typing and protocol design
• data-model design
• Python testing patterns

Push framework/service-boundary concerns into a separate repo instead of muddying this one.

Upstream selection rule

Choose mature repos that expose different parts of the problem space.

Current first-wave set:

• python/cpython
• encode/httpx
• pytest-dev/pytest
• pydantic/pydantic

Current refinement set:

• python-attrs/attrs
• pallets/click
• sqlalchemy/sqlalchemy

Selection criteria:

• respected and maintained
• stable public APIs
• enough repetition to support non-hand-wavy conclusions
• examples/tests/internal structure that reveal tradeoffs, not just happy paths

Directory contract

• sources/ = raw evidence notes, one file per upstream repo
• patterns/ = synthesized guidance from repeated evidence
• comparison/ = short notes for places where a library or framework bends the generic Python rule
• smells/ = anti-patterns derived from the same evidence base

Step-by-step workflow

1) Split the problem cleanly

Do not mix Python-wide guidance with FastAPI/service conventions in one repo.

2) Clone or otherwise make upstream sources available locally

Work from local checkouts so citations can be verified quickly.

3) Write source notes first

For each upstream repo, create sources/<repo>.md with:

• why this repo is useful
• repeated patterns
• caveats/counterexamples
• exact file:line citations
• pattern candidates supported by the evidence

Important: source notes are not polished guidance. They are reusable evidence.

4) Synthesize only after evidence exists

Turn strong repeated signals into patterns/*.md docs.

Use comparison/*.md only when the generic Python rule still matters, but a specific library or framework changes how it should be applied.

Each pattern doc should usually include:

• what the pattern is
• why it exists
• when to use it
• when not to use it
• preferred shapes
• counterexamples
• source signals/citations

5) Mine anti-patterns explicitly

After the positive patterns are stable enough, write smell docs by inverting the evidence:

• broad catch-all exceptions with no recovery meaning
• hidden resource lifetime
• accidental public APIs
• fake sync wrappers over async code
• overuse of Any
• mixed transport/business/persistence responsibilities in one model

6) Run a refinement wave before broadening source coverage

Once the main docs exist, do not immediately add more repos.

Instead:

• improve synthesized docs in fresh contexts
• tighten weak citations
• rewrite source-note files to be denser and more reusable
• reduce duplicated guidance across docs

Handling mixed-concern source code

Upstream code will often mix generic Python design with framework-specific behavior in the same function, block, or line.

When that happens:

• do not classify the whole snippet by repo name or file path alone
• split the evidence into Python-level signal and framework-owned signal
• keep only the reusable language/library-design lesson in python-patterns
• move the framework-owned lesson into the framework repo or comparison note

If the claim depends on request parsing, dependency injection, response modeling, or HTTP error semantics, it no longer belongs here as a base Python rule.

Review checklist

For source notes

• Does the file separate repeated patterns from one-off examples?
• Are caveats preserved?
• Are citations exact and easy to re-check?
• Does it avoid vague claims like “mature code usually...” unless evidence is shown?

For pattern docs

• Does each rule follow from repeated evidence?
• Is the guidance prescriptive without pretending there were no tradeoffs?
• Are counterexamples concrete?
• Is the doc still Python-level rather than framework-specific?

Local git workflow used here

When the repo is ready for human review:

1. initialize a local git repo
2. stage the current documentation set
3. create a single initial commit so review has a stable baseline

This repo intentionally avoids pushing or creating remotes unless explicitly requested.

What to avoid

• writing docs from memory
• mixing Python and framework guidance together
• broadening source coverage before tightening weak docs
• flattening caveats away during synthesis
• leaving citations too vague to verify quickly
• treating source notes as prose polish instead of evidence storage