chore: merge golang-conventions and prometheus-conventions into sources/
Absorbed content from rodin/golang-conventions and rodin/prometheus-conventions into a sources/ directory. Reference material — descriptive, not prescriptive. Part of taxonomy cleanup (elixir-patterns issue #4).
This commit is contained in:
Rodin
@@ -0,0 +1,15 @@
|
||||
# Sources
|
||||
|
||||
Reference material extracted from specific projects. Study for ideas, don't copy blindly.
|
||||
|
||||
These are **descriptive** — they document what a project does and why.
|
||||
The `patterns/` directory is **prescriptive** — it tells you what to do.
|
||||
|
||||
Patterns that prove broadly applicable get promoted from here into `patterns/`.
|
||||
The rest stays as reference for understanding how mature projects solve specific problems.
|
||||
|
||||
## Files
|
||||
|
||||
- `golang.md` — conventions from the golang/go source
|
||||
- `golang-analysis.md` — deeper analysis of golang/go source architecture
|
||||
- `prometheus.md` — patterns from prometheus/prometheus (TSDB, storage, metrics)
|
||||
@@ -0,0 +1,364 @@
|
||||
# Go Language Source: Architectural Conventions
|
||||
|
||||
How does the Go team build Go itself? What does the language source
|
||||
reveal about conventions, governance, and infrastructure decisions?
|
||||
|
||||
**Repo:** [golang/go](https://github.com/golang/go)
|
||||
|
||||
---
|
||||
|
||||
## 1. Repo Shape
|
||||
|
||||
| Metric | Value |
|
||||
|--------|-------|
|
||||
| Size | 632M |
|
||||
| Go files | 11,245 |
|
||||
| Assembly files (runtime) | 200 |
|
||||
| Commits | 66,142 |
|
||||
| Contributors | 2,842 |
|
||||
| Test files | 1,811 |
|
||||
| Non-test files | 6,065 |
|
||||
| Test ratio | 1:3.3 |
|
||||
| TODOs (non-test) | 3,428 (all owner-attributed) |
|
||||
|
||||
### Organizational Philosophy
|
||||
|
||||
```
|
||||
src/
|
||||
├── cmd/ # The toolchain (compile, link, go, gofmt, vet)
|
||||
│ └── compile/
|
||||
│ └── internal/ssa/ # 417,686 lines of SSA compiler
|
||||
├── internal/ # 61 hidden packages (API firewall)
|
||||
├── runtime/ # 129,141 lines (scheduler, GC, memory)
|
||||
├── encoding/ # Serialization (json, xml, gob)
|
||||
├── net/ # Networking
|
||||
├── io/ # Stream interfaces
|
||||
└── testing/ # Test framework
|
||||
```
|
||||
|
||||
The Go source has three layers:
|
||||
1. **Flat stdlib** — user-visible packages (`fmt`, `io`, `net`)
|
||||
2. **`internal/`** — shared infrastructure hidden from users (61 packages)
|
||||
3. **`cmd/`** — the toolchain itself (compiler, linker, build tool)
|
||||
|
||||
---
|
||||
|
||||
## 2. What the Codebase Values
|
||||
|
||||
### By import frequency
|
||||
|
||||
| Package | Imports | Role |
|
||||
|---------|---------|------|
|
||||
| `fmt` | 2,031 | Formatting (the universal tool) |
|
||||
| `testing` | 1,658 | Tests are first-class citizens |
|
||||
| `strings` | 1,454 | String manipulation |
|
||||
| `os` | 1,306 | System interaction |
|
||||
| `unsafe` | 1,304 | Low-level memory access |
|
||||
| `runtime` | 970 | Runtime introspection |
|
||||
| `io` | 924 | Stream abstraction |
|
||||
|
||||
**The surprise:** `unsafe` (1,304 imports) ranks 5th — nearly tied with
|
||||
`os`. The language that preaches memory safety uses `unsafe` extensively
|
||||
in its own implementation. This is the "know the rules so you know
|
||||
where they don't apply" principle.
|
||||
|
||||
### By size (what consumes the most lines)
|
||||
|
||||
| Component | Lines | Nature |
|
||||
|-----------|-------|--------|
|
||||
| SSA compiler | 417,686 | Mostly generated rewrite rules |
|
||||
| Runtime | 129,141 | Scheduler + GC + memory |
|
||||
| `testing/` | 4,770 | Testing framework |
|
||||
| `internal/poll` | 5,087 | OS I/O multiplexing |
|
||||
| `internal/fuzz` | 4,220 | Fuzzing infrastructure |
|
||||
| `encoding/json/v2` | 6,387 | The JSON rewrite (experiment) |
|
||||
|
||||
---
|
||||
|
||||
## 3. The Bootstrap Problem
|
||||
|
||||
**How does Go compile itself?**
|
||||
|
||||
Go has been self-hosting since May 2015 (Russ Cox, commit `0f4132c907`:
|
||||
"all: build and use go tool compile, go tool link"). The bootstrap
|
||||
chain:
|
||||
|
||||
1. A pre-built Go 1.4 binary compiles the bootstrap tools
|
||||
2. Those tools compile the current Go toolchain
|
||||
3. The new toolchain recompiles itself for correctness
|
||||
|
||||
**The `cmd/dist` tool** orchestrates this multi-stage build. Unlike
|
||||
Elixir (which keeps 33 Erlang files permanently), Go eliminated its
|
||||
non-Go dependencies entirely. The only external requirement is a
|
||||
previous Go binary.
|
||||
|
||||
**Convention:** Self-hosting means the compiler, linker, and runtime
|
||||
are all written in Go. The runtime includes 200 assembly files for
|
||||
architecture-specific operations (scheduler context switches, atomic
|
||||
operations, system calls).
|
||||
|
||||
---
|
||||
|
||||
## 4. TODO Culture: Owned Accountability
|
||||
|
||||
```go
|
||||
// TODO(gri) — 320 occurrences (Robert Griesemer)
|
||||
// TODO(mdempsky) — 198 occurrences (Matthew Dempsky)
|
||||
// TODO(adonovan) — 170 occurrences (Alan Donovan)
|
||||
// TODO(mknyszek) — 98 occurrences (Michael Knyszek)
|
||||
// TODO(rsc) — 96 occurrences (Russ Cox)
|
||||
```
|
||||
|
||||
**3,428 TODOs** in non-test code. Every TODO has an owner. The top 5
|
||||
TODO authors are all core team members (compiler, runtime, and tools
|
||||
leads).
|
||||
|
||||
**Convention:** `// TODO(username): description`
|
||||
|
||||
**Growth pattern:** linkname directives — 43 touches in 2019, 48 in
|
||||
2022, 92 in 2024, 72 in 2025. The codebase is actively evolving its
|
||||
relationship with internal/external boundaries.
|
||||
|
||||
**Contrast with Elixir:** Go TODOs are permanent documentation of known
|
||||
limitations. Elixir TODOs are time-bombs with version deadlines. Go
|
||||
accepts technical debt as a layer; Elixir refuses to let it accumulate.
|
||||
|
||||
---
|
||||
|
||||
## 5. Unique Patterns
|
||||
|
||||
### 5.1 `internal/` as API Firewall (61 packages)
|
||||
|
||||
Go's most distinctive structural pattern. The `internal/` directory is
|
||||
enforced by the compiler — code outside the tree cannot import these
|
||||
packages.
|
||||
|
||||
**Key internal packages:**
|
||||
- `internal/godebug` — runtime feature flags (79 settings)
|
||||
- `internal/goexperiment` — compile-time experiment guards
|
||||
- `internal/singleflight` — dedup concurrent function calls
|
||||
- `internal/bisect` — binary search for debugging (Russ Cox, 2023)
|
||||
- `internal/poll` — OS I/O polling (5,087 lines)
|
||||
- `internal/fuzz` — fuzzing coordinator (4,220 lines)
|
||||
- `internal/coverage` — code coverage (3,821 lines)
|
||||
|
||||
**Convention:** Code shared between stdlib packages but not suitable for
|
||||
public API goes in `internal/`. This is Go's answer to "how do you
|
||||
share utilities without committing to backward compatibility."
|
||||
|
||||
### 5.2 GODEBUG: Runtime Feature Flags
|
||||
|
||||
Introduced 2021 (Brad Fitzpatrick), formalized as a compatibility
|
||||
mechanism by Russ Cox in 2022 (proposal #56986: "extended backwards
|
||||
compatibility for Go", 70 comments).
|
||||
|
||||
```go
|
||||
var http2server = godebug.New("http2server")
|
||||
|
||||
func ServeConn(c net.Conn) {
|
||||
if http2server.Value() == "0" {
|
||||
// disable HTTP/2
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**79 godebug settings** across the codebase. Each time a non-default
|
||||
setting causes a behavior change, code calls `IncNonDefault()` to
|
||||
increment a counter readable via `runtime/metrics`.
|
||||
|
||||
**Convention:** When a behavior change might break existing programs,
|
||||
add a GODEBUG setting. The old behavior remains accessible via
|
||||
`GODEBUG=setting=old_value`. New Go versions automatically use the
|
||||
old behavior when building code that declared an older `go` directive
|
||||
in `go.mod`.
|
||||
|
||||
**This is the Go compatibility promise made machine-enforceable.**
|
||||
|
||||
### 5.3 GOEXPERIMENT: Compile-Time Feature Gates
|
||||
|
||||
```go
|
||||
// In internal/goexperiment/flags.go:
|
||||
// GOEXPERIMENT=jsonv2 enables the new JSON API
|
||||
```
|
||||
|
||||
**Convention:** Major API additions ship behind experiment flags.
|
||||
`encoding/json/v2` (6,387 lines, Apr 2025) exists in the tree but is
|
||||
only visible when `GOEXPERIMENT=jsonv2` is set. This allows the code
|
||||
to be developed in-tree, tested by adventurous users, and refined
|
||||
before the compatibility promise applies.
|
||||
|
||||
### 5.4 Compiler Directives as Hidden Language
|
||||
|
||||
```go
|
||||
//go:linkname localFunction remote/package.Function
|
||||
//go:nosplit
|
||||
//go:nowritebarrier
|
||||
//go:noescape
|
||||
//go:systemstack
|
||||
```
|
||||
|
||||
**1,711 `go:linkname` directives** and **2,428 runtime compiler
|
||||
directives** in non-test code. These are effectively a hidden language
|
||||
within Go — they bypass the type system, calling conventions, and
|
||||
garbage collector safety for performance-critical paths.
|
||||
|
||||
**Convention:** Directives are ONLY acceptable in the runtime and
|
||||
compiler. The Go team is actively trying to reduce `go:linkname` usage
|
||||
(92 touches in 2024 — many are removals). Third-party packages that use
|
||||
`go:linkname` to access internals are explicitly unsupported.
|
||||
|
||||
### 5.5 Generated Code: The SSA Compiler
|
||||
|
||||
The SSA (Static Single Assignment) compiler backend is 417,686 lines,
|
||||
but the largest files are generated:
|
||||
|
||||
```
|
||||
opGen.go — 97,135 lines (generated)
|
||||
rewriteAMD64.go — 79,703 lines (generated)
|
||||
rewritegeneric.go — 38,337 lines (generated)
|
||||
rewriteARM64.go — 26,203 lines (generated)
|
||||
```
|
||||
|
||||
**Convention:** Generated files are checked into the repo (not
|
||||
generated at build time). They contain a `// Code generated` header.
|
||||
The generators live alongside their output. This means `git blame`
|
||||
works on generated code — you can trace when a rewrite rule was added.
|
||||
|
||||
### 5.6 The Runtime: 129K Lines of Go + Assembly
|
||||
|
||||
```
|
||||
proc.go — 8,156 lines (THE goroutine scheduler)
|
||||
malloc.go — 2,501 lines (memory allocator)
|
||||
mgc.go — 2,315 lines (garbage collector)
|
||||
mheap.go — 3,030 lines (heap management)
|
||||
panic.go — 1,788 lines (panic/recover machinery)
|
||||
```
|
||||
|
||||
The scheduler (`proc.go`) documents its own design at the top:
|
||||
> "The main concepts are: G - goroutine. M - worker thread, or machine.
|
||||
> P - processor, a resource that is required to execute Go code."
|
||||
|
||||
**Convention:** The runtime combines Go and assembly (200 `.s` files).
|
||||
Architecture-specific operations (context switches, atomic ops, system
|
||||
calls) are in assembly. Everything else is in Go, using compiler
|
||||
directives to bypass safety checks where needed.
|
||||
|
||||
---
|
||||
|
||||
## 6. PR Discussion Patterns
|
||||
|
||||
Go uses GitHub issues for proposals, not PRs for discussion. The key
|
||||
governance mechanism is the **proposal process** with designated
|
||||
reviewers.
|
||||
|
||||
### json/v2 (Issue #71497, 201 comments, Jan 2025)
|
||||
|
||||
"The largest major revision of a standard Go package to date."
|
||||
|
||||
**Key design decision:** Split into `encoding/json/v2` (semantic) and
|
||||
`encoding/jsontext` (syntactic). The syntactic layer has no reflection
|
||||
dependency — it's a pure JSON tokenizer.
|
||||
|
||||
**Ship strategy:** Land behind `GOEXPERIMENT=jsonv2`, iterate with
|
||||
community feedback, then graduate. The code lives in-tree but doesn't
|
||||
count as a compatibility commitment until the experiment flag is
|
||||
removed.
|
||||
|
||||
**External validation:** Built as `github.com/go-json-experiment/json`
|
||||
first, iterated for years, then proposed for stdlib. The implementation
|
||||
preceded the proposal.
|
||||
|
||||
### GODEBUG (Issue #56986, 70 comments, Nov 2022)
|
||||
|
||||
Russ Cox's proposal for machine-enforced backward compatibility.
|
||||
|
||||
**The problem:** Sort algorithm changes, bug fixes, and behavior
|
||||
improvements can break programs that depend on old behavior.
|
||||
|
||||
**The solution:** `go.mod`'s `go` directive becomes a compatibility
|
||||
declaration. Programs built with Go 1.22 but declaring `go 1.20`
|
||||
automatically get Go 1.20 behavior for any setting that changed between
|
||||
1.20 and 1.22.
|
||||
|
||||
**Lesson:** The Go team solved "how do you improve a language without
|
||||
breaking users" with a general mechanism rather than case-by-case
|
||||
migration. Each new GODEBUG setting is a structured backward
|
||||
compatibility opt-out.
|
||||
|
||||
### Generics (Issue #15292, 874 comments, 2016-2021)
|
||||
|
||||
5 years of discussion. The most debated language change in Go's history.
|
||||
|
||||
**Lesson:** Committee-driven projects can take years on foundational
|
||||
decisions because consensus requires addressing every edge case. Compare
|
||||
to Elixir's formatter (1 hour, zero comments) — the BDFL model moves
|
||||
faster but accepts more single-point-of-failure risk.
|
||||
|
||||
### slog (Issue #56345, 841 comments, 2022-2023)
|
||||
|
||||
Structured logging took 10 months and 841 comments to land.
|
||||
|
||||
**Lesson:** Even when the need is clear and the solution is well-known,
|
||||
Go's process requires exhaustive discussion. The result is usually
|
||||
better (slog is well-designed), but the cost is measured in months.
|
||||
|
||||
---
|
||||
|
||||
## 7. Cross-Ecosystem Comparisons
|
||||
|
||||
| Aspect | Go | Elixir |
|
||||
|--------|-----|--------|
|
||||
| TODOs | 3,428, owner-attributed, permanent | 127, version-gated, deadlines |
|
||||
| Self-hosting | Complete since 2015 | 33 Erlang files permanently |
|
||||
| Feature gates | GOEXPERIMENT (compile-time) | None (ship or don't) |
|
||||
| Compat mechanism | GODEBUG (79 settings) | Deprecation → removal on version |
|
||||
| Governance | Committee (proposals, 874-comment threads) | BDFL (José, 1-hour merges) |
|
||||
| Internal boundary | `internal/` (compiler-enforced, 61 packages) | OTP applications (convention-enforced) |
|
||||
| Generated code | Checked in (97K-line files) | Compile-time (no artifacts) |
|
||||
| Assembly | 200 .s files in runtime | None (delegates to Erlang/BEAM) |
|
||||
| Biggest file | 97,135 lines (generated) | 7,102 lines (Kernel) |
|
||||
|
||||
---
|
||||
|
||||
## 8. What This Teaches
|
||||
|
||||
1. **`internal/` solves "shared but not public" at the language level.**
|
||||
61 packages that other ecosystems have to solve with conventions,
|
||||
Go solves with compiler enforcement. This is why Go projects
|
||||
rarely have "utils" packages that leak abstraction — the pattern
|
||||
exists in the language itself.
|
||||
|
||||
2. **GODEBUG is the most sophisticated backward compatibility mechanism
|
||||
in any language runtime.** It makes the compatibility promise
|
||||
*machine-verifiable* rather than *socially-enforced*. Programs don't
|
||||
just get old behavior by default — they get it because their `go.mod`
|
||||
declares what era they belong to.
|
||||
|
||||
3. **GOEXPERIMENT enables fearless iteration in the stdlib.** json/v2
|
||||
can exist in-tree, be tested, be refined — all without triggering
|
||||
the compatibility promise. This is "feature flags for a language."
|
||||
|
||||
4. **3,428 TODOs is honest, not sloppy.** Each one has an owner. They
|
||||
document known limitations rather than hiding them. Go prefers
|
||||
"explicitly imperfect" over "implicitly broken."
|
||||
|
||||
5. **Compiler directives are a hidden language.** 4,139 directives
|
||||
(linkname + runtime pragmas) bypass Go's safety model. The Go team
|
||||
accepts that the runtime needs a different language than users —
|
||||
but actively restricts this power from escaping to third-party code.
|
||||
|
||||
6. **Generated code checked in > generated at build time** for
|
||||
archaeology. `git blame` on `rewriteAMD64.go` tells you when a
|
||||
codegen rule was added and why. Build-time generation loses this
|
||||
history.
|
||||
|
||||
7. **Committee governance produces better designs but 10x slower.** Go's
|
||||
slog (841 comments, 10 months) vs Elixir's formatter (0 comments,
|
||||
1 hour). The designs are comparable in quality — the process cost
|
||||
is where they differ.
|
||||
|
||||
8. **`unsafe` in Go's own source (1,304 imports) proves that safety
|
||||
rules are for users, not for runtime implementors.** The people who
|
||||
wrote the safety rules know exactly where they don't apply.
|
||||
|
||||
<!-- PATTERN_COMPLETE -->
|
||||
@@ -0,0 +1,270 @@
|
||||
# Go Language Source: Convention Reference
|
||||
|
||||
Quick-reference for conventions extracted from the golang/go source
|
||||
code. Each entry: pattern name, location, example, when to use,
|
||||
when NOT to use, origin.
|
||||
|
||||
---
|
||||
|
||||
## Owner-Attributed TODOs
|
||||
|
||||
**Location:** Throughout `src/`
|
||||
|
||||
```go
|
||||
// TODO(gri): consider using a different approach here
|
||||
// TODO(rsc): this should be cleaned up in the next release
|
||||
```
|
||||
|
||||
**When to use:** Known limitations that a specific person should
|
||||
address. The owner tag creates accountability without creating an
|
||||
issue (issues are for user-visible problems; TODOs are for internal
|
||||
engineering debt).
|
||||
|
||||
**When NOT to use:** User-visible bugs (file an issue instead).
|
||||
Aspirational improvements with no clear owner. Anything that should
|
||||
block a release.
|
||||
|
||||
**Origin:** Convention since Go's earliest commits. The Go team
|
||||
averages 3,428 TODOs across the codebase — this is a conscious
|
||||
engineering culture, not neglect.
|
||||
|
||||
---
|
||||
|
||||
## `internal/` Packages
|
||||
|
||||
**Location:** `src/internal/` (61 packages)
|
||||
|
||||
```go
|
||||
// internal/singleflight — dedup concurrent calls
|
||||
// internal/godebug — runtime feature flags
|
||||
// internal/poll — OS I/O polling
|
||||
// internal/bisect — binary search debugging
|
||||
```
|
||||
|
||||
**When to use:** Code shared between stdlib packages that should NOT
|
||||
become public API. Utility code that isn't stable enough for the
|
||||
compatibility promise. Implementation details that users shouldn't
|
||||
depend on.
|
||||
|
||||
**When NOT to use:** Code that external packages need. Code that's
|
||||
stable enough for public API (promote it). One-off helpers that only
|
||||
one package uses (keep them package-private).
|
||||
|
||||
**Origin:** `src/internal/` existed since Go moved sources from
|
||||
`src/pkg` to `src` (2014). The compiler enforces the import restriction
|
||||
— no code outside the tree can import internal packages.
|
||||
|
||||
---
|
||||
|
||||
## GODEBUG: Runtime Feature Flags
|
||||
|
||||
**Location:** `internal/godebug/godebug.go` (316 lines)
|
||||
|
||||
```go
|
||||
var http2server = godebug.New("http2server")
|
||||
|
||||
func ServeConn(c net.Conn) {
|
||||
if http2server.Value() == "0" {
|
||||
// user opted out of HTTP/2
|
||||
}
|
||||
// IncNonDefault must be called each time non-default behavior fires
|
||||
http2server.IncNonDefault()
|
||||
}
|
||||
```
|
||||
|
||||
**When to use:** Behavior changes that might break existing programs.
|
||||
The old behavior becomes accessible via `GODEBUG=setting=value`. Tied
|
||||
to `go.mod`'s `go` directive for automatic version-based defaults.
|
||||
|
||||
**When NOT to use:** Bug fixes that no reasonable program depends on.
|
||||
New features (use GOEXPERIMENT instead). Performance optimizations that
|
||||
don't change observable behavior.
|
||||
|
||||
**Origin:** Brad Fitzpatrick added the package in Aug 2021. Russ Cox
|
||||
formalized it as the backward compatibility mechanism in proposal
|
||||
#56986 (Nov 2022, 70 comments). 79 settings now exist.
|
||||
|
||||
---
|
||||
|
||||
## GOEXPERIMENT: Compile-Time Feature Gates
|
||||
|
||||
**Location:** `internal/goexperiment/flags.go` (136 lines)
|
||||
|
||||
```go
|
||||
// Set via: GOEXPERIMENT=jsonv2 go build
|
||||
// Check via build tag: //go:build goexperiment.jsonv2
|
||||
```
|
||||
|
||||
**When to use:** Major new APIs or behavior changes that need real-world
|
||||
testing before committing to the compatibility promise. The code lives
|
||||
in-tree but isn't visible without the flag.
|
||||
|
||||
**When NOT to use:** Small features that can ship directly. Bug fixes.
|
||||
Internal refactoring. Anything that doesn't need user feedback before
|
||||
committing.
|
||||
|
||||
**Origin:** The GOEXPERIMENT mechanism predates its formalization — used
|
||||
for fieldtrack, regabi, unified. json/v2 (Apr 2025) is the highest-
|
||||
profile use: 6,387 lines shipped behind a flag.
|
||||
|
||||
---
|
||||
|
||||
## Compiler Directives
|
||||
|
||||
**Location:** Throughout `runtime/` and `cmd/`
|
||||
|
||||
```go
|
||||
//go:linkname localName remote/pkg.ExportedName
|
||||
//go:nosplit
|
||||
//go:nowritebarrier
|
||||
//go:noescape
|
||||
//go:systemstack
|
||||
```
|
||||
|
||||
**When to use:** ONLY in the runtime and compiler. `linkname` accesses
|
||||
unexported symbols across packages. `nosplit` prevents stack growth
|
||||
checks. `nowritebarrier` asserts no GC barriers. `systemstack` forces
|
||||
execution on the system stack.
|
||||
|
||||
**When NOT to use:** In application code. In stdlib packages outside
|
||||
runtime. In third-party packages (explicitly unsupported — the Go team
|
||||
actively removes `go:linkname` targets that external packages depend
|
||||
on).
|
||||
|
||||
**Origin:** 1,711 `go:linkname` and 2,428 other runtime directives.
|
||||
The Go team is actively trying to reduce linkname usage (92 touches
|
||||
in 2024, many removals).
|
||||
|
||||
---
|
||||
|
||||
## Generated Code (Checked In)
|
||||
|
||||
**Location:** `cmd/compile/internal/ssa/`
|
||||
|
||||
```go
|
||||
// Code generated by ssa/gen/*.go; DO NOT EDIT.
|
||||
|
||||
// opGen.go — 97,135 lines
|
||||
// rewriteAMD64.go — 79,703 lines
|
||||
// rewritegeneric.go — 38,337 lines
|
||||
```
|
||||
|
||||
**When to use:** When the generated output needs `git blame` history.
|
||||
When generators should be auditable alongside their output. When build
|
||||
reproducibility matters (no external tool dependency).
|
||||
|
||||
**When NOT to use:** When the generated output is large AND changes
|
||||
frequently (diff noise). When the generator is trivial (just use
|
||||
`go generate` at build time).
|
||||
|
||||
**Origin:** SSA compiler introduced on dev.ssa branch (2015). The
|
||||
rewrite rules are generated from a DSL but checked in so reviewers
|
||||
can see exactly what changed.
|
||||
|
||||
---
|
||||
|
||||
## Assembly in Runtime
|
||||
|
||||
**Location:** `runtime/*.s` (200 files)
|
||||
|
||||
```asm
|
||||
// runtime/asm_amd64.s
|
||||
TEXT runtime·gogo(SB), NOSPLIT, $0-8
|
||||
MOVQ buf+0(FP), BX // gobuf
|
||||
MOVQ gobuf_g(BX), DX
|
||||
...
|
||||
```
|
||||
|
||||
**When to use:** Context switches, atomic operations, system calls,
|
||||
and anything that needs direct control over registers/stack. Each
|
||||
architecture has its own set of assembly files.
|
||||
|
||||
**When NOT to use:** Anything that can be written in Go with acceptable
|
||||
performance. The Go team actively converts assembly to Go where
|
||||
possible (e.g., crypto packages moving from asm to Go+intrinsics).
|
||||
|
||||
**Origin:** Runtime has always included assembly — Go's scheduler and
|
||||
GC require direct hardware control that no high-level language can
|
||||
provide.
|
||||
|
||||
---
|
||||
|
||||
## Proposal Process (Committee Governance)
|
||||
|
||||
**Location:** GitHub issues, not PRs
|
||||
|
||||
```
|
||||
Issue #71497 (json/v2): 201 comments, Jan 2025
|
||||
Issue #56986 (GODEBUG): 70 comments, Nov 2022
|
||||
Issue #56345 (slog): 841 comments, 10 months
|
||||
Issue #15292 (generics): 874 comments, 5 years
|
||||
```
|
||||
|
||||
**When to use:** Any user-visible API addition or behavior change.
|
||||
The proposal process ensures all edge cases are considered before
|
||||
committing to the compatibility promise.
|
||||
|
||||
**When NOT to use:** Internal refactoring, performance improvements
|
||||
that don't change API, bug fixes. These go through normal code review
|
||||
(Gerrit/GitHub).
|
||||
|
||||
**Origin:** The Go proposal process evolved from informal (early Go)
|
||||
to formalized (2015+). Russ Cox and the Go team manage a proposal
|
||||
review meeting that triages and decides.
|
||||
|
||||
---
|
||||
|
||||
## The Scheduler as Documentation
|
||||
|
||||
**Location:** `runtime/proc.go` (8,156 lines)
|
||||
|
||||
```go
|
||||
// Goroutine scheduler
|
||||
// The scheduler's job is to distribute ready-to-run goroutines over
|
||||
// worker threads.
|
||||
//
|
||||
// The main concepts are:
|
||||
// G - goroutine.
|
||||
// M - worker thread, or machine.
|
||||
// P - processor, a resource that is required to execute Go code.
|
||||
```
|
||||
|
||||
**When to use:** Complex algorithms should document their model at the
|
||||
top of the file. The Go runtime uses extensive comments to explain
|
||||
the scheduler's invariants, the GC's phases, and the memory
|
||||
allocator's structure.
|
||||
|
||||
**When NOT to use:** Simple code that's self-documenting. Comments
|
||||
that restate what the code does rather than WHY it does it.
|
||||
|
||||
**Origin:** The G-M-P model comment dates to the scheduler rewrite
|
||||
(2014). The convention of architectural documentation at file-top
|
||||
extends throughout the runtime.
|
||||
|
||||
---
|
||||
|
||||
## json/v2: Experiment-to-Stdlib Pipeline
|
||||
|
||||
**Location:** `encoding/json/v2/` (6,387 lines, 13 files)
|
||||
|
||||
```go
|
||||
// encoding/json/v2 — semantic layer (uses reflection)
|
||||
// encoding/jsontext — syntactic layer (no reflection dependency)
|
||||
```
|
||||
|
||||
**When to use:** When building a major stdlib revision. The pattern:
|
||||
1. Build as external module (`go-json-experiment/json`)
|
||||
2. Iterate with real users for years
|
||||
3. Propose for stdlib with working implementation
|
||||
4. Ship behind GOEXPERIMENT flag
|
||||
5. Graduate when stable
|
||||
|
||||
**When NOT to use:** Small additions that don't need years of
|
||||
iteration. Features that can be backward-compatible additions to
|
||||
existing packages.
|
||||
|
||||
**Origin:** `go-json-experiment/json` existed since 2022. Proposed as
|
||||
#71497 (Jan 2025, 201 comments). Landed behind flag Apr 2025 (commit
|
||||
`0e17905793` by Damien Neil).
|
||||
|
||||
<!-- PATTERN_COMPLETE -->
|
||||
@@ -0,0 +1,182 @@
|
||||
# Patterns Extracted from prometheus/prometheus
|
||||
|
||||
## Pattern: Atomic File Operations with Suffix Convention
|
||||
|
||||
**Source:** `tsdb/db.go`
|
||||
**Category:** storage
|
||||
|
||||
**What:** Use directory suffixes (`.tmp-for-creation`,
|
||||
`.tmp-for-deletion`) to make multi-step file operations
|
||||
crash-safe. On startup, clean up any dirs with these
|
||||
suffixes (they represent incomplete operations).
|
||||
|
||||
**Why:** Database storage needs atomicity. If the process
|
||||
crashes between creating a block and finalizing it, you
|
||||
need to know the block is incomplete. The suffix convention
|
||||
makes incomplete state visible at the filesystem level
|
||||
without requiring a separate journal.
|
||||
|
||||
**Example:**
|
||||
|
||||
```go
|
||||
const (
|
||||
tmpForDeletionBlockDirSuffix = ".tmp-for-deletion"
|
||||
tmpForCreationBlockDirSuffix = ".tmp-for-creation"
|
||||
)
|
||||
|
||||
// On startup: remove any .tmp-* dirs (incomplete ops)
|
||||
// On create: write to dir.tmp-for-creation, then rename
|
||||
// On delete: rename to dir.tmp-for-deletion, then remove
|
||||
```
|
||||
|
||||
**When to use:** Any system that manages files/directories
|
||||
and needs crash consistency without a full WAL. Simpler
|
||||
than a write-ahead log for coarse-grained operations.
|
||||
|
||||
**When NOT to use:** When you already have a WAL or
|
||||
transaction log. Or for fine-grained operations where
|
||||
rename semantics are insufficient.
|
||||
|
||||
---
|
||||
|
||||
## Pattern: DefaultOptions() Function
|
||||
|
||||
**Source:** `tsdb/db.go`
|
||||
**Category:** configuration
|
||||
|
||||
**What:** Provide a `DefaultOptions()` function returning a
|
||||
fully-populated config struct. Users copy and override only
|
||||
what they need. No nil-means-default ambiguity.
|
||||
|
||||
**Why:** Large config structs (20+ fields) are unwieldy.
|
||||
By providing sane defaults as a function (not a
|
||||
package-level var), you avoid mutation bugs and make it
|
||||
clear what "normal" looks like. Users only specify
|
||||
deviations.
|
||||
|
||||
**Example:**
|
||||
|
||||
```go
|
||||
func DefaultOptions() *Options {
|
||||
return &Options{
|
||||
WALSegmentSize: wlog.DefaultSegmentSize,
|
||||
RetentionDuration: int64(15*24*time.Hour / ...),
|
||||
MinBlockDuration: DefaultBlockDuration,
|
||||
MaxBlockDuration: DefaultBlockDuration,
|
||||
SamplesPerChunk: DefaultSamplesPerChunk,
|
||||
// ... 20 more fields with sane defaults
|
||||
}
|
||||
}
|
||||
|
||||
// Usage:
|
||||
opts := tsdb.DefaultOptions()
|
||||
opts.RetentionDuration = 30 * 24 * time.Hour
|
||||
db, err := tsdb.Open(dir, nil, nil, opts, nil)
|
||||
```
|
||||
|
||||
**When to use:** Config structs with many fields where most
|
||||
users want defaults. Especially when zero-value semantics
|
||||
would be confusing (e.g., 0 retention = infinite? or off?).
|
||||
|
||||
**When NOT to use:** Small configs (3-4 fields) where
|
||||
struct literal with zero-means-default is clear enough.
|
||||
|
||||
---
|
||||
|
||||
## Pattern: Scrape Loop with Aligned Timestamps
|
||||
|
||||
**Source:** `scrape/scrape.go`
|
||||
**Category:** concurrency
|
||||
|
||||
**What:** Periodic scrape loops that align timestamps to
|
||||
intervals with a small tolerance, enabling better storage
|
||||
compression downstream.
|
||||
|
||||
**Why:** Time-series databases compress better when
|
||||
timestamps are regular. A 2ms tolerance on alignment
|
||||
means scraped data aligns to the expected grid while
|
||||
accommodating real-world jitter.
|
||||
|
||||
**Example:**
|
||||
|
||||
```go
|
||||
var ScrapeTimestampTolerance = 2 * time.Millisecond
|
||||
var AlignScrapeTimestamps = true
|
||||
|
||||
// In scrape loop: if scrape finishes within tolerance
|
||||
// of expected timestamp, snap to the grid
|
||||
```
|
||||
|
||||
**When to use:** Any periodic data collection where
|
||||
downstream storage benefits from timestamp regularity.
|
||||
Metrics, heartbeats, polling loops.
|
||||
|
||||
**When NOT to use:** Event-driven data where timestamps
|
||||
must reflect actual occurrence time. Audit logs, user
|
||||
actions, financial transactions.
|
||||
|
||||
---
|
||||
|
||||
## Pattern: Sentinel Errors with Interface Check
|
||||
|
||||
**Source:** `tsdb/db.go`
|
||||
**Category:** error-handling
|
||||
|
||||
**What:** Define package-level sentinel errors with
|
||||
`errors.New()` and use compile-time interface assertions
|
||||
to verify implementations satisfy storage interfaces.
|
||||
|
||||
**Why:** `ErrNotReady` as a sentinel lets callers use
|
||||
`errors.Is` for retry logic. The pattern ensures error
|
||||
identity is stable across versions (not string-matched).
|
||||
|
||||
**Example:**
|
||||
|
||||
```go
|
||||
var ErrNotReady = errors.New("TSDB not ready")
|
||||
|
||||
// Callers can reliably detect this:
|
||||
if errors.Is(err, tsdb.ErrNotReady) {
|
||||
// Retry later — DB is still initializing
|
||||
}
|
||||
```
|
||||
|
||||
**When to use:** Any error that callers need to handle
|
||||
programmatically (retry, fallback, special UI). Make it a
|
||||
named sentinel, not a string comparison.
|
||||
|
||||
**When NOT to use:** Errors that are always terminal or
|
||||
always logged-and-discarded. Not every error needs a name.
|
||||
|
||||
---
|
||||
|
||||
## Pattern: Compile-Time Interface Satisfaction
|
||||
|
||||
**Source:** `scrape/scrape.go`
|
||||
**Category:** organization
|
||||
|
||||
**What:** Use `var _ Interface = (*Type)(nil)` to verify at
|
||||
compile time that a type satisfies an interface, even if
|
||||
the type is only used dynamically.
|
||||
|
||||
**Why:** Without this, you discover missing methods only
|
||||
when the type is actually used — which might be in a
|
||||
rarely-exercised code path or only in production. The
|
||||
compile-time check catches it immediately.
|
||||
|
||||
**Example:**
|
||||
|
||||
```go
|
||||
var _ FailureLogger = (*logging.JSONFileLogger)(nil)
|
||||
// Fails at compile time if JSONFileLogger doesn't
|
||||
// implement FailureLogger
|
||||
```
|
||||
|
||||
**When to use:** Any type that implements an interface
|
||||
consumed dynamically (registered in a map, stored as
|
||||
interface value, passed to framework code).
|
||||
|
||||
**When NOT to use:** Types whose interface satisfaction is
|
||||
already enforced by direct usage in the same package.
|
||||
|
||||
<!-- PATTERN_COMPLETE -->
|
||||
Reference in New Issue
Block a user