docs: idiomatic Go patterns from stdlib + Kubernetes with source citations
This commit is contained in:
Rodin
@@ -0,0 +1,707 @@
|
||||
# Go Concurrency Patterns
|
||||
|
||||
Patterns extracted from the Go standard library source code.
|
||||
|
||||
---
|
||||
|
||||
## 1. sync.Mutex — The Basic Lock
|
||||
|
||||
### Source: `src/sync/mutex.go:18-34`, `src/sync/mutex.go:42-67`
|
||||
|
||||
```go
|
||||
// src/sync/mutex.go:18-34
|
||||
// A Mutex is a mutual exclusion lock.
|
||||
// The zero value for a Mutex is an unlocked mutex.
|
||||
//
|
||||
// A Mutex must not be copied after first use.
|
||||
type Mutex struct {
|
||||
_ noCopy
|
||||
mu isync.Mutex
|
||||
}
|
||||
|
||||
// src/sync/mutex.go:36-39
|
||||
type Locker interface {
|
||||
Lock()
|
||||
Unlock()
|
||||
}
|
||||
|
||||
// src/sync/mutex.go:43-46
|
||||
func (m *Mutex) Lock() {
|
||||
m.mu.Lock()
|
||||
}
|
||||
|
||||
// src/sync/mutex.go:64-67
|
||||
func (m *Mutex) Unlock() {
|
||||
m.mu.Unlock()
|
||||
}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
- **Zero value is ready to use** — no constructor needed
|
||||
- **Must not be copied** — enforced by `noCopy` field (go vet detects copies)
|
||||
- **Not associated with a goroutine** — one goroutine can Lock, another can Unlock
|
||||
- **Locker interface** — abstracts over Mutex and RWMutex
|
||||
|
||||
### Idiomatic Usage
|
||||
|
||||
```go
|
||||
var mu sync.Mutex
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
// ... critical section ...
|
||||
```
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Copy a mutex
|
||||
type Config struct {
|
||||
mu sync.Mutex
|
||||
data map[string]string
|
||||
}
|
||||
c2 := *c1 // COPIES the mutex — data race waiting to happen
|
||||
|
||||
// DON'T: Forget defer
|
||||
mu.Lock()
|
||||
// if this panics, the mutex stays locked forever
|
||||
doSomething()
|
||||
mu.Unlock()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. sync.Once — Exactly-Once Initialization
|
||||
|
||||
### Source: `src/sync/once.go:12-36`, `src/sync/once.go:56-79`
|
||||
|
||||
```go
|
||||
// src/sync/once.go:12-23
|
||||
type Once struct {
|
||||
_ noCopy
|
||||
done atomic.Bool
|
||||
m Mutex
|
||||
}
|
||||
|
||||
// src/sync/once.go:56-63
|
||||
func (o *Once) Do(f func()) {
|
||||
if !o.done.Load() {
|
||||
o.doSlow(f)
|
||||
}
|
||||
}
|
||||
|
||||
// src/sync/once.go:65-72
|
||||
func (o *Once) doSlow(f func()) {
|
||||
o.m.Lock()
|
||||
defer o.m.Unlock()
|
||||
if !o.done.Load() {
|
||||
defer o.done.Store(true)
|
||||
f()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
The implementation reveals a subtle guarantee: **when Do returns, f has finished**. The naive CAS-only approach (documented in the comment at line 56-63) would let the second caller return before f completes. The mutex ensures all callers wait.
|
||||
|
||||
The `done` field is first in the struct for hot-path performance on amd64/386 (noted in comment at line 24-27).
|
||||
|
||||
### Idiomatic Usage
|
||||
|
||||
```go
|
||||
var (
|
||||
instance *DB
|
||||
once sync.Once
|
||||
)
|
||||
|
||||
func GetDB() *DB {
|
||||
once.Do(func() {
|
||||
instance = connectToDB()
|
||||
})
|
||||
return instance
|
||||
}
|
||||
```
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Implement once yourself with a bool
|
||||
var initialized bool
|
||||
var mu sync.Mutex
|
||||
func init() {
|
||||
mu.Lock()
|
||||
if !initialized {
|
||||
// ... setup ...
|
||||
initialized = true
|
||||
}
|
||||
mu.Unlock()
|
||||
}
|
||||
|
||||
// DON'T: Call Do recursively (deadlocks)
|
||||
var once sync.Once
|
||||
once.Do(func() {
|
||||
once.Do(func() { /* deadlock */ })
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. sync.WaitGroup — Waiting for Goroutine Completion
|
||||
|
||||
### Source: `src/sync/waitgroup.go:14-43`, `src/sync/waitgroup.go:236-260`
|
||||
|
||||
```go
|
||||
// src/sync/waitgroup.go:14-43
|
||||
// A WaitGroup is a counting semaphore typically used to wait
|
||||
// for a group of goroutines or tasks to finish.
|
||||
//
|
||||
// Typically, a main goroutine will start tasks, each in a new
|
||||
// goroutine, by calling WaitGroup.Go and then wait for all tasks to
|
||||
// complete by calling WaitGroup.Wait. For example:
|
||||
//
|
||||
// var wg sync.WaitGroup
|
||||
// wg.Go(task1)
|
||||
// wg.Go(task2)
|
||||
// wg.Wait()
|
||||
type WaitGroup struct {
|
||||
noCopy noCopy
|
||||
state atomic.Uint64
|
||||
sema uint32
|
||||
}
|
||||
```
|
||||
|
||||
### Go 1.25+: WaitGroup.Go
|
||||
|
||||
```go
|
||||
// src/sync/waitgroup.go:236-260
|
||||
func (wg *WaitGroup) Go(f func()) {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer func() {
|
||||
if x := recover(); x != nil {
|
||||
panic(x) // don't call Done — let panic propagate
|
||||
}
|
||||
wg.Done()
|
||||
}()
|
||||
f()
|
||||
}()
|
||||
}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
`WaitGroup.Go` (new in Go 1.25) encapsulates the Add/go/Done pattern. Key design: if `f` panics, it re-panics **without** calling Done, preventing the main goroutine from racing to exit.
|
||||
|
||||
### Classic Pattern (pre-Go 1.25)
|
||||
|
||||
```go
|
||||
var wg sync.WaitGroup
|
||||
for _, item := range items {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
process(item)
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
```
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Add inside the goroutine (race with Wait)
|
||||
for _, item := range items {
|
||||
go func() {
|
||||
wg.Add(1) // might run after Wait is called!
|
||||
defer wg.Done()
|
||||
process(item)
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
|
||||
// DON'T: Forget Done (Wait blocks forever)
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
process()
|
||||
// forgot wg.Done()
|
||||
}()
|
||||
wg.Wait() // hangs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. sync.Pool — Object Reuse for GC Pressure
|
||||
|
||||
### Source: `src/sync/pool.go:44-63`
|
||||
|
||||
```go
|
||||
// src/sync/pool.go:44-63
|
||||
// A Pool is a set of temporary objects that may be individually saved and
|
||||
// retrieved.
|
||||
//
|
||||
// Any item stored in the Pool may be removed automatically at any time without
|
||||
// notification. If the Pool holds the only reference when this happens, the
|
||||
// item might be deallocated.
|
||||
//
|
||||
// Pool's purpose is to cache allocated but unused items for later reuse,
|
||||
// relieving pressure on the garbage collector.
|
||||
type Pool struct {
|
||||
noCopy noCopy
|
||||
local unsafe.Pointer
|
||||
localSize uintptr
|
||||
victim unsafe.Pointer
|
||||
victimSize uintptr
|
||||
New func() any
|
||||
}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
Pool is **not** a general cache. Items can vanish between GC cycles. It's for reducing allocation pressure on hot paths — `fmt` uses it for print buffers, `encoding/json` for encoder state.
|
||||
|
||||
### Idiomatic Usage (from fmt package)
|
||||
|
||||
```go
|
||||
var ppFree = sync.Pool{
|
||||
New: func() any { return new(pp) },
|
||||
}
|
||||
|
||||
func newPrinter() *pp {
|
||||
p := ppFree.Get().(*pp)
|
||||
p.reset()
|
||||
return p
|
||||
}
|
||||
|
||||
func (p *pp) free() {
|
||||
p.buf = p.buf[:0] // reset before returning
|
||||
ppFree.Put(p)
|
||||
}
|
||||
```
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Use Pool for connection pooling (items disappear!)
|
||||
var connPool = sync.Pool{
|
||||
New: func() any { return connectToDB() },
|
||||
}
|
||||
// Connections may be GC'd at any time — use database/sql's pool instead
|
||||
|
||||
// DON'T: Put dirty objects back without resetting
|
||||
pool.Put(buf) // still has data from last use — memory leak or data leak
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Channel as Done Signal (Context Pattern)
|
||||
|
||||
### Source: `src/context/context.go:83-100` (Done channel), `src/io/pipe.go:42-45`
|
||||
|
||||
```go
|
||||
// src/context/context.go:83-100
|
||||
// Done returns a channel that's closed when work done on behalf of this
|
||||
// context should be canceled.
|
||||
Done() <-chan struct{}
|
||||
|
||||
// src/io/pipe.go:42-45
|
||||
type pipe struct {
|
||||
wrMu sync.Mutex
|
||||
wrCh chan []byte
|
||||
rdCh chan int
|
||||
once sync.Once
|
||||
done chan struct{} // closed on pipe close
|
||||
rerr onceError
|
||||
werr onceError
|
||||
}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
A `chan struct{}` costs zero bytes per send and closing it broadcasts to all receivers simultaneously. This is the canonical "done" signal in Go.
|
||||
|
||||
```go
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case result := <-work:
|
||||
return result, nil
|
||||
}
|
||||
```
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Use chan bool for done signals
|
||||
done := make(chan bool) // wastes 1 byte per signal, true/false meaningless
|
||||
|
||||
// DON'T: Send to done (only works once, only one receiver)
|
||||
done <- struct{}{} // only unblocks one goroutine
|
||||
|
||||
// DO: Close the channel (broadcasts to all)
|
||||
close(done)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 6. Context Propagation
|
||||
|
||||
### Source: `src/context/context.go:37-48` (rules), `src/net/http/request.go:368-380`
|
||||
|
||||
From the package doc:
|
||||
```go
|
||||
// src/context/context.go:37-48
|
||||
// Programs that use Contexts should follow these rules:
|
||||
//
|
||||
// Do not store Contexts inside a struct type; instead, pass a Context
|
||||
// explicitly to each function that needs it. The Context should be the first
|
||||
// parameter, typically named ctx:
|
||||
//
|
||||
// func DoSomething(ctx context.Context, arg Arg) error {
|
||||
// // ... use ctx ...
|
||||
// }
|
||||
//
|
||||
// Do not pass a nil Context, even if a function permits it. Pass context.TODO
|
||||
// if you are unsure about which Context to use.
|
||||
```
|
||||
|
||||
### Request context in net/http:
|
||||
|
||||
```go
|
||||
// src/net/http/request.go:368-380
|
||||
func (r *Request) WithContext(ctx context.Context) *Request {
|
||||
if ctx == nil {
|
||||
panic("nil context")
|
||||
}
|
||||
r2 := new(Request)
|
||||
*r2 = *r
|
||||
r2.ctx = ctx
|
||||
return r2
|
||||
}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
Context flows **down** the call chain, never stored in structs. `WithContext` returns a shallow copy — the original request is not mutated. This is the immutable-context pattern.
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Store context in a struct
|
||||
type Server struct {
|
||||
ctx context.Context // stale context persists beyond request lifecycle
|
||||
}
|
||||
|
||||
// DON'T: Pass nil context
|
||||
doWork(nil, data) // use context.TODO() if unsure
|
||||
|
||||
// DON'T: Put context anywhere other than first parameter
|
||||
func doWork(data Data, ctx context.Context) // wrong position
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. Context Cancellation (WithCancel/WithTimeout)
|
||||
|
||||
### Source: `src/context/context.go:242-249` (WithCancel), `src/net/http/server.go:4007-4050` (TimeoutHandler)
|
||||
|
||||
```go
|
||||
// src/context/context.go:242-249
|
||||
func WithCancel(parent Context) (ctx Context, cancel CancelFunc) {
|
||||
c := withCancel(parent)
|
||||
return c, func() { c.cancel(true, Canceled, nil) }
|
||||
}
|
||||
```
|
||||
|
||||
Real-world use — net/http TimeoutHandler:
|
||||
```go
|
||||
// src/net/http/server.go:4011-4014
|
||||
func (h *timeoutHandler) ServeHTTP(w ResponseWriter, r *Request) {
|
||||
ctx, cancelCtx := context.WithTimeout(r.Context(), h.dt)
|
||||
defer cancelCtx()
|
||||
r = r.WithContext(ctx)
|
||||
done := make(chan struct{})
|
||||
// ...
|
||||
go func() {
|
||||
h.handler.ServeHTTP(tw, r)
|
||||
close(done)
|
||||
}()
|
||||
select {
|
||||
case <-done:
|
||||
// handler completed
|
||||
case <-ctx.Done():
|
||||
// timeout
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
`defer cancelCtx()` is critical — it releases resources (timers, goroutines) when the parent returns, even if the child hasn't timed out yet. The go vet tool checks for this.
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Forget to call cancel (leaks goroutines)
|
||||
ctx, _ := context.WithCancel(parent) // cancel function discarded!
|
||||
|
||||
// DON'T: Cancel before work starts
|
||||
ctx, cancel := context.WithTimeout(parent, 5*time.Second)
|
||||
cancel() // immediately cancels — no work can happen
|
||||
doWork(ctx)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Select with Done Channel
|
||||
|
||||
### Source: `src/context/context.go:83-100` (Done in select), `src/io/pipe.go:51-60`
|
||||
|
||||
```go
|
||||
// src/io/pipe.go:51-60
|
||||
func (p *pipe) read(b []byte) (n int, err error) {
|
||||
select {
|
||||
case <-p.done:
|
||||
return 0, p.readCloseError()
|
||||
default:
|
||||
}
|
||||
|
||||
select {
|
||||
case bw := <-p.wrCh:
|
||||
nr := copy(b, bw)
|
||||
p.rdCh <- nr
|
||||
return nr, nil
|
||||
case <-p.done:
|
||||
return 0, p.readCloseError()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
The double-select pattern: first a non-blocking check (with `default`), then a blocking wait. The non-blocking check prevents a race where `done` was closed between the last operation and the current one.
|
||||
|
||||
### Standard Context Select Pattern
|
||||
|
||||
```go
|
||||
// From context package doc (line 83-100)
|
||||
func Stream(ctx context.Context, out chan<- Value) error {
|
||||
for {
|
||||
v, err := DoSomething(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case out <- v:
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Check ctx.Done() in a busy loop
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
default:
|
||||
}
|
||||
// busy-spins CPU at 100%
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 9. Goroutine-per-Connection (net/http Server)
|
||||
|
||||
### Source: `src/net/http/server.go` (conceptual — the serve loop spawns goroutines per connection)
|
||||
|
||||
```go
|
||||
// The pattern (simplified from server.go serve loop):
|
||||
for {
|
||||
conn, err := listener.Accept()
|
||||
if err != nil {
|
||||
// handle
|
||||
continue
|
||||
}
|
||||
go srv.handleConn(conn) // one goroutine per connection
|
||||
}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
Go's goroutines are cheap (~2KB initial stack). The server doesn't need a thread pool or async/await — it spawns a goroutine per connection and lets the runtime scheduler handle multiplexing.
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Limit yourself to a fixed thread pool for I/O-bound work
|
||||
pool := make(chan struct{}, 10) // artificial limit on connections
|
||||
for {
|
||||
pool <- struct{}{} // blocks at 10
|
||||
conn := accept()
|
||||
go func() {
|
||||
defer func() { <-pool }()
|
||||
handle(conn)
|
||||
}()
|
||||
}
|
||||
// Only appropriate for CPU-bound work or resource-constrained scenarios
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 10. Channel as Synchronous Pipe (io.Pipe)
|
||||
|
||||
### Source: `src/io/pipe.go:38-45`, `src/io/pipe.go:195-205`
|
||||
|
||||
```go
|
||||
// src/io/pipe.go:38-45
|
||||
type pipe struct {
|
||||
wrMu sync.Mutex
|
||||
wrCh chan []byte // writer sends data slices
|
||||
rdCh chan int // reader returns bytes consumed
|
||||
once sync.Once
|
||||
done chan struct{}
|
||||
rerr onceError
|
||||
werr onceError
|
||||
}
|
||||
|
||||
// src/io/pipe.go:195-205
|
||||
func Pipe() (*PipeReader, *PipeWriter) {
|
||||
pw := &PipeWriter{r: PipeReader{pipe: pipe{
|
||||
wrCh: make(chan []byte),
|
||||
rdCh: make(chan int),
|
||||
done: make(chan struct{}),
|
||||
}}}
|
||||
return &pw.r, pw
|
||||
}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
`io.Pipe` connects a Writer to a Reader using **unbuffered channels** — each Write blocks until the corresponding Read consumes the data. No internal buffering means backpressure is automatic. The `done` channel signals when either end closes.
|
||||
|
||||
### Pattern: Channel Pipeline
|
||||
|
||||
```go
|
||||
func generate(ctx context.Context) <-chan int {
|
||||
out := make(chan int)
|
||||
go func() {
|
||||
defer close(out)
|
||||
for i := 0; ; i++ {
|
||||
select {
|
||||
case out <- i:
|
||||
case <-ctx.Done():
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
return out
|
||||
}
|
||||
```
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Forget to close channels (receivers block forever)
|
||||
func produce() <-chan int {
|
||||
ch := make(chan int)
|
||||
go func() {
|
||||
for i := 0; i < 10; i++ {
|
||||
ch <- i
|
||||
}
|
||||
// forgot close(ch) — receivers hang on range
|
||||
}()
|
||||
return ch
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 11. database/sql Connection Opener Goroutine
|
||||
|
||||
### Source: `src/database/sql/sql.go:836-843`
|
||||
|
||||
```go
|
||||
// src/database/sql/sql.go:836-843
|
||||
func OpenDB(c driver.Connector) *DB {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
db := &DB{
|
||||
connector: c,
|
||||
openerCh: make(chan struct{}, connectionRequestQueueSize),
|
||||
lastPut: make(map[*driverConn]string),
|
||||
stop: cancel,
|
||||
}
|
||||
go db.connectionOpener(ctx)
|
||||
return db
|
||||
}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
A dedicated background goroutine (`connectionOpener`) processes connection requests from a buffered channel. The goroutine is controlled by a context — calling `cancel()` (stored as `db.stop`) shuts it down cleanly. This is the "long-lived worker goroutine with context shutdown" pattern.
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Start background goroutines without shutdown mechanism
|
||||
go func() {
|
||||
for {
|
||||
processWork() // runs forever, no way to stop it
|
||||
}
|
||||
}()
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 12. noCopy — Preventing Value Copies at Vet Time
|
||||
|
||||
### Source: `src/sync/cond.go:120-126`
|
||||
|
||||
```go
|
||||
// src/sync/cond.go:120-126
|
||||
type noCopy struct{}
|
||||
|
||||
// Lock is a no-op used by -copylocks checker from `go vet`.
|
||||
func (*noCopy) Lock() {}
|
||||
func (*noCopy) Unlock() {}
|
||||
```
|
||||
|
||||
### Why
|
||||
|
||||
Embedding `noCopy` in a struct makes `go vet` report an error when the struct is copied. All sync primitives use this because copying a locked mutex or in-use WaitGroup is always a bug.
|
||||
|
||||
### Anti-pattern
|
||||
|
||||
```go
|
||||
// DON'T: Pass sync types by value
|
||||
func doWork(wg sync.WaitGroup) { // copies the WaitGroup!
|
||||
defer wg.Done()
|
||||
}
|
||||
|
||||
// DO: Pass by pointer
|
||||
func doWork(wg *sync.WaitGroup) {
|
||||
defer wg.Done()
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary: Concurrency Decision Guide
|
||||
|
||||
| Need | Use |
|
||||
|------|-----|
|
||||
| Protect shared state | `sync.Mutex` + `defer Unlock()` |
|
||||
| One-time initialization | `sync.Once` |
|
||||
| Wait for N goroutines | `sync.WaitGroup` (prefer `.Go()` in 1.25+) |
|
||||
| Reduce allocation pressure | `sync.Pool` (not for connections!) |
|
||||
| Signal completion/cancellation | `chan struct{}` + `close()` |
|
||||
| Deadline/timeout propagation | `context.WithTimeout` / `context.WithCancel` |
|
||||
| Backpressure between producer/consumer | Unbuffered channels |
|
||||
| Fan-out with results | Buffered channel + WaitGroup |
|
||||
| Long-lived background worker | Goroutine + context cancellation |
|
||||
| Prevent struct copying | Embed `noCopy` field |
|
||||
Reference in New Issue
Block a user