Go was designed at Google by Rob Pike, Ken Thompson, and Robert Griesemer to address frustrations with C++ and Java at scale. Its core philosophy: simplicity, readability, and explicit design over cleverness.

Key differentiators:

• Fast compilation: Compiles entire large codebases in seconds. No header files, circular import prevention, fast linker.
• Built-in concurrency: Goroutines and channels are first-class language features, not library additions.
• Garbage collected with low latency: Concurrent mark-and-sweep GC. Sub-millisecond pauses since Go 1.14+.
• Static typing with type inference: := infers types at compile time — no runtime overhead.
• No inheritance: Composition via embedding; polymorphism via interfaces (structural typing, not nominal).
• Opinionated toolchain: gofmt, go test, go vet, go mod are all built in — no ecosystem fragmentation.
• Single static binary: Compiles to a self-contained executable — no runtime, no JVM, no interpreter needed.

What Go deliberately omits: generics (added in 1.18 — minimally), exceptions (uses explicit error values), inheritance, operator overloading, macros, function/method overloading, implicit type conversions. These omissions are intentional — they reduce cognitive overhead and make code more readable at scale.

Go's target is programming in the large — large teams, large codebases, long maintenance horizons. Simplicity compounds over time.

Go uses structural typing — types are defined by their structure, not their name. Methods can be defined on any named type (not just structs).

type User struct {
    ID    int
    Name  string
    Email string
}

// Value receiver — receives a copy; does not modify the original
func (u User) String() string {
    return fmt.Sprintf("%s <%s>", u.Name, u.Email)
}

// Pointer receiver — receives a pointer; can modify the original
func (u *User) SetEmail(email string) {
    u.Email = email  // modifies the original
}

// Methods on non-struct types
type Celsius float64
func (c Celsius) ToFahrenheit() float64 { return float64(c)*9/5 + 32 }

Value vs pointer receivers — when to use each:

• Use pointer receiver when: the method needs to modify the receiver; the receiver is a large struct (avoid copying); you need consistent interface satisfaction
• Use value receiver when: the type is small and cheap to copy; the method doesn't modify state; the type is a basic type or small struct
• Be consistent: if any method on a type uses a pointer receiver, prefer pointer receivers for all methods on that type

Embedding — composition over inheritance:

type Animal struct { Name string }
func (a Animal) Speak() string { return a.Name + " speaks" }

type Dog struct {
    Animal          // embedded — promotes Animal's fields and methods
    Breed string
}

d := Dog{Animal: Animal{Name: "Rex"}, Breed: "Lab"}
d.Speak()     // promoted — calls Animal.Speak()
d.Animal.Speak() // explicit — same thing
d.Name        // promoted field access

Embedding is not inheritance — there is no subtype relationship. A Dog is not an Animal. But Dog has all of Animal's methods promoted, so it can satisfy the same interfaces.

A slice is a three-word descriptor: a pointer to an underlying array, a length, and a capacity. It is a view into an array — not a copy of the data.

// Slice header: {ptr *T, len int, cap int}
s := make([]int, 3, 5)  // len=3, cap=5
s[0], s[1], s[2] = 10, 20, 30

// Appending within capacity — no allocation
s = append(s, 40)  // len=4, cap=5, same underlying array

// Appending beyond capacity — new array allocated, data copied
s = append(s, 50, 60)  // len=6, cap grows (typically doubles)

Pitfall 1 — shared underlying array:

a := []int{1, 2, 3, 4, 5}
b := a[1:3]   // b shares a's array: b = [2, 3], cap=4
b[0] = 99    // MODIFIES a too! a = [1, 99, 3, 4, 5]

// Fix: use full slice expression to limit capacity, or copy
b := a[1:3:3]        // cap=2, appends won't affect a
b := append([]int{}, a[1:3]...)  // independent copy

Pitfall 2 — nil vs empty slice:

var s []int      // nil slice: s == nil, len=0, cap=0
s = []int{}     // empty slice: s != nil, len=0, cap=0
s = make([]int, 0)  // empty slice

// Both are safe to append to and range over
// JSON: nil → null, empty → []
// Prefer nil for "no data", empty for "zero items"

Pitfall 3 — memory leak via large backing array:

func findFirst(data []byte) []byte {
    return data[:10]  // holds reference to entire large slice!
}
// Fix:
result := make([]byte, 10)
copy(result, data[:10])
return result

Go maps are hash tables implemented as an array of buckets, each holding up to 8 key-value pairs. The runtime uses a hash of the key to find the bucket.

// Always initialize before use
m := make(map[string]int)
m := map[string]int{"a": 1, "b": 2}  // literal

// Two-value assignment to check existence
val, ok := m["key"]
if !ok { /* key not present */ }

// Delete
delete(m, "key")

// Iteration — order is intentionally randomized each run
for k, v := range m {
    fmt.Println(k, v)
}

Pitfall 1 — nil map panics on write:

var m map[string]int  // nil map
m["key"] = 1          // PANIC: assignment to entry in nil map
val := m["key"]       // OK — reads return zero value, no panic

Pitfall 2 — concurrent access causes panic:

// Maps are NOT safe for concurrent read+write
// Runtime detects this and panics with "concurrent map read and map write"

// Fix 1: sync.RWMutex
var mu sync.RWMutex
mu.Lock(); m[k] = v; mu.Unlock()
mu.RLock(); val = m[k]; mu.RUnlock()

// Fix 2: sync.Map (optimized for high-read, low-write concurrent use)
var sm sync.Map
sm.Store("key", 42)
val, ok := sm.Load("key")
sm.LoadOrStore("key", 42)  // atomic check-and-set

Iteration order: Intentionally randomized since Go 1.0 to prevent programs from depending on map order. If you need sorted keys, collect them into a slice and sort explicitly.

make(T, args) — creates and initializes a slice, map, or channel. Returns the value itself (not a pointer). The type must be slice, map, or chan.

s := make([]int, 5, 10)     // slice: len=5, cap=10
m := make(map[string]int)   // map: initialized, ready to use
ch := make(chan int, 5)      // buffered channel with capacity 5

new(T) — allocates zeroed memory for type T and returns a pointer *T. Rarely needed — composite literals with & are more idiomatic.

p := new(int)    // *int pointing to zeroed int
// Equivalent to:
var x int; p := &x

// Idiomatic alternative for structs:
u := &User{Name: "Alice"}  // preferred over new(User)

copy(dst, src []T) — copies min(len(dst), len(src)) elements. Returns the number copied. Works for slices of the same type, and []byte from string.

n := copy(dst, src)       // n = min(len(dst), len(src))
copy(b, "hello")          // copy string into []byte

append(s []T, elems ...T) — appends to a slice, allocating a new backing array if capacity is exceeded. Always reassign the result: s = append(s, x).

close(ch) — closes a channel. Signals receivers that no more values will be sent. Sending on a closed channel panics. Receiving from a closed channel returns the zero value and false.

delete(m, key) — removes a key from a map. No-op if key doesn't exist (no panic).

len(v) and cap(v) — length and capacity of slices, arrays, maps, channels, strings.

panic(v) and recover() — trigger and recover from runtime panics. recover() only works inside a deferred function.

defer schedules a function call to execute when the surrounding function returns, regardless of whether it returns normally or via panic. Deferred calls are pushed onto a per-goroutine stack and execute LIFO (last in, first out).

func processFile(path string) error {
    f, err := os.Open(path)
    if err != nil { return err }
    defer f.Close()  // guaranteed to run when processFile returns

    // multiple defers — LIFO order
    defer fmt.Println("third")   // runs first (last pushed)
    defer fmt.Println("second")  // runs second
    defer fmt.Println("first")   // runs last (first pushed)
    // Output: third, second, first
    return nil
}

Arguments are evaluated immediately at defer statement, not at execution:

x := 10
defer fmt.Println(x)  // captures x=10 NOW
x = 20
// Prints: 10 (not 20)

Named return values + defer for cleanup:

func divide(a, b float64) (result float64, err error) {
    defer func() {
        if r := recover(); r != nil {
            err = fmt.Errorf("recovered: %v", r)
        }
    }()
    return a / b, nil
}

Common use cases: resource cleanup (files, DB connections, mutexes), logging entry/exit, transaction commit/rollback, recovering from panics, restoring state. Prefer defer for cleanup even if it adds slight overhead — correctness beats micro-optimization.

Performance note: Defer has overhead (heap allocation in older Go). Go 1.14+ uses "open-coded" defer for most cases, reducing this to near zero. Avoid defer in the innermost loop of hot paths if profiling shows it matters.

Generics (type parameters) allow writing functions and types that work with multiple types while maintaining type safety. Introduced in Go 1.18 after years of deliberation — the Go team kept the design minimal.

// Generic function — T is a type parameter
func Map[T, U any](s []T, f func(T) U) []U {
    result := make([]U, len(s))
    for i, v := range s { result[i] = f(v) }
    return result
}

names := Map([]int{1,2,3}, strconv.Itoa)  // []string{"1","2","3"}

// Constraints — restrict what types T can be
type Number interface {
    ~int | ~int32 | ~int64 | ~float32 | ~float64
}
// ~ means "any type whose underlying type is..."

func Sum[T Number](s []T) T {
    var total T
    for _, v := range s { total += v }
    return total
}

Sum([]int{1,2,3})      // 6
Sum([]float64{1.1,2.2}) // 3.3

// Generic data structure
type Stack[T any] struct { items []T }
func (s *Stack[T]) Push(item T) { s.items = append(s.items, item) }
func (s *Stack[T]) Pop() (T, bool) {
    var zero T
    if len(s.items) == 0 { return zero, false }
    item := s.items[len(s.items)-1]
    s.items = s.items[:len(s.items)-1]
    return item, true
}

// Built-in constraints (golang.org/x/exp/constraints or slices/maps packages)
// constraints.Ordered — all types that support < > <= >=
// comparable — types that support == !=

When to use generics: Data structures (stacks, queues, trees, sets); utility functions that operate uniformly on collections (Map, Filter, Reduce); when you're currently writing the same function for int and float64 and string. When NOT to: When interfaces suffice; when the function does different things for different types (use interfaces); premature generalization — start concrete, generalize when needed.

A package is a directory of .go files sharing the same package declaration. A module is a collection of packages with a go.mod file defining the module path and dependencies.

// go.mod — defines your module
module github.com/myorg/myapp

go 1.22

require (
    github.com/gin-gonic/gin v1.9.1
    google.golang.org/grpc v1.62.0
)

// Common commands
go mod init github.com/myorg/myapp  // create go.mod
go get github.com/pkg/errors@v0.9.1  // add dependency
go mod tidy                          // remove unused, add missing
go mod vendor                        // copy deps to vendor/
go mod download                      // download to module cache

// go.sum — checksums for all dependencies (commit this!)
// Lock file equivalent — ensures reproducible builds

Semantic versioning and major versions: Go modules follow semver. For major versions v2+, the import path changes: github.com/foo/bar/v2. This allows multiple major versions to coexist in the same build.

// v1: import "github.com/foo/bar"
// v2: import "github.com/foo/bar/v2"  — different package!

Replace directive: Override a dependency with a local path or different version — useful for development and forking:

replace github.com/some/dep => ../local-dep
replace github.com/some/dep v1.2.3 => github.com/my-fork v1.2.3-patched

Visibility: Identifiers starting with an uppercase letter are exported (public). Lowercase identifiers are unexported (package-private). There are no other access modifiers.

init() is a special function that runs automatically during package initialization, after all package-level variable declarations are evaluated. Key guarantees:

• No parameters, no return values
• Multiple init() functions allowed per file and per package — all run
• Within a package: variable declarations → init() functions, in source order
• Across packages: all of a dependency's init() functions run before the importing package's
• Cannot be called directly from code

var db *sql.DB  // declared at package level

func init() {
    var err error
    db, err = sql.Open("postgres", os.Getenv("DATABASE_URL"))
    if err != nil { log.Fatal(err) }
}

// Multiple init() in same file — both run
func init() { registerMetrics() }
func init() { setupLogging() }

// Blank import — import for side effects (init() only)
import _ "github.com/lib/pq"  // registers postgres driver

Pitfalls:

• init() runs even if the package is imported for side effects only — keep it lightweight
• Hard to test — logic in init() runs before any test setup
• Execution order across packages is determined by import graph — complex dependency chains are hard to reason about
• Prefer explicit initialization functions called from main() for better testability and clarity

Go closures capture variables by reference — the closure holds a reference to the variable, not a copy of its value at capture time.

// Classic loop variable capture bug
funcs := make([]func(), 5)
for i := 0; i < 5; i++ {
    funcs[i] = func() { fmt.Println(i) }  // captures &i, not i's value
}
for _, f := range funcs { f() }
// Prints: 5 5 5 5 5 — all see i=5 after the loop

// Fix 1 — create a new variable each iteration
for i := 0; i < 5; i++ {
    i := i  // shadow i with a new variable scoped to this iteration
    funcs[i] = func() { fmt.Println(i) }
}
// Prints: 0 1 2 3 4

// Fix 2 — pass as argument
for i := 0; i < 5; i++ {
    funcs[i] = func(n int) func() {
        return func() { fmt.Println(n) }
    }(i)  // pass i by value
}

// NOTE: Go 1.22+ fixes this — loop variables are per-iteration by default
// The classic bug no longer exists in Go 1.22+

Closures for stateful functions:

func makeCounter() func() int {
    count := 0            // captured by reference
    return func() int {
        count++
        return count      // count persists between calls
    }
}
counter := makeCounter()
counter()  // 1
counter()  // 2
counter()  // 3

Variable escape to heap: When a closure captures a local variable, that variable escapes to the heap (it must outlive the function's stack frame). The compiler's escape analysis determines this automatically.

Arrays in Go are value types with a fixed size that is part of the type. [3]int and [4]int are different types. Copying an array copies all elements. Comparing arrays with == compares element by element.

// Array — size is part of the type
var a [3]int               // [0 0 0]
b := [3]int{1, 2, 3}
c := [...]int{1, 2, 3}    // compiler counts elements — still [3]int
b == c                     // true — value comparison!

// Array is passed by value — function gets a copy
func sum(arr [5]int) int { ... }  // copies 5 ints

// Slice — backed by an array, dynamically sized
s := []int{1, 2, 3}
s = append(s, 4)           // can grow

When to use arrays:

• Fixed-size data where the length is semantically meaningful and must not change — cryptographic keys ([32]byte), RGB values ([3]uint8), coordinates
• Stack allocation: small arrays are stack-allocated (no GC pressure), slices of arrays may heap-allocate the backing array
• Comparable as map keys: arrays are comparable (==), slices are not — so [2]int{1,2} can be a map key but []int{1,2} cannot
• When you want value semantics — copying an array creates an independent copy

In practice: Slices are used 95% of the time. Arrays appear in cryptography ([32]byte for SHA-256), network protocols, and performance-critical code where stack allocation matters.

A Go string is an immutable sequence of bytes (a read-only slice header: pointer + length). It is NOT a sequence of characters — it's raw UTF-8 encoded bytes.

s := "Hello, 世界"   // UTF-8 encoded

len(s)            // 13 bytes (not 9 characters!)
s[0]              // byte value 72 ('H')

// Iterating by bytes — wrong for multibyte characters
for i := 0; i < len(s); i++ {
    fmt.Printf("%x ", s[i])  // raw bytes
}

// Iterating by rune (Unicode code point) — correct
for i, r := range s {
    fmt.Printf("%d: %c\n", i, r)
}

// Conversions
b := []byte(s)    // copy to mutable byte slice
r := []rune(s)    // copy to Unicode code points
len(r)            // 9 — actual character count

s2 := string(b)   // convert back to string (copies)
s3 := string(r)   // convert back to string

rune is an alias for int32 — represents a Unicode code point. Use []rune when you need to work with characters (indexing, reversing strings with non-ASCII content).

strings package for common operations:

strings.Contains(s, "世界")
strings.Split(s, ",")
strings.TrimSpace(s)
strings.Builder{}  // efficient incremental string building

// Efficient concatenation
var b strings.Builder
for _, word := range words {
    b.WriteString(word)
    b.WriteByte(' ')
}
result := b.String()

Performance: String concatenation with + in a loop is O(n²) — each concatenation creates a new string. Use strings.Builder or fmt.Sprintf for building strings incrementally.

Go uses a concurrent, tri-color mark-and-sweep garbage collector. Its primary goal is low latency (sub-millisecond stop-the-world pauses) over maximum throughput — the right trade-off for server applications.

How it works:

1. Mark setup (STW): Very brief stop-the-world to initialize marking state
2. Concurrent mark: GC goroutines scan the heap concurrently with the program, marking reachable objects. Uses write barriers to handle mutations during marking.
3. Mark termination (STW): Brief stop-the-world to finalize marking
4. Concurrent sweep: Reclaim unmarked memory concurrently — program continues running

GOGC tuning:

// GOGC controls when the next GC triggers
// Default: 100 — GC when live heap doubles from previous collection
// Lower = more frequent GC (less memory, more CPU)
// Higher = less frequent GC (more memory, less CPU)
GOGC=200  // GC when heap triples — good for batch jobs

// Go 1.19+: GOMEMLIMIT — hard cap on memory usage
GOMEMLIMIT=1GiB  // GC more aggressively if heap approaches 1GB
// Best practice: set GOMEMLIMIT to ~90% of container memory limit

// In code (not recommended for prod — use env vars):
import "runtime/debug"
debug.SetGCPercent(200)
debug.SetMemoryLimit(1 << 30)  // 1 GiB

// Force GC (testing/benchmarks only)
runtime.GC()

GC-friendly patterns: Reduce allocations in hot paths (reuse objects with sync.Pool); prefer stack allocation (small, non-escaping objects); use value types instead of pointers for small structs; avoid pointer-heavy data structures in hot paths (GC must scan all pointers).

The Go compiler performs escape analysis at compile time to determine whether a variable can live on the goroutine's stack or must be allocated on the heap.

Variables escape to the heap when:

• A pointer to a local variable is returned or stored in a data structure that outlives the function
• The variable is captured by a closure that outlives the function
• The variable is too large for the stack
• The variable is sent over a channel or stored in an interface

// Stays on stack — no pointer escapes
func stackAlloc() int {
    x := 42      // stays on stack
    return x     // value returned, not pointer
}

// Escapes to heap — pointer returned
func heapAlloc() *int {
    x := 42      // escapes to heap
    return &x    // pointer to x outlives the function
}

// Inspect escape analysis:
go build -gcflags="-m" ./...
// Output: "./main.go:7:2: moved to heap: x"

// More verbose:
go build -gcflags="-m -m" ./...

Interface conversion often causes escape:

func printVal(v interface{}) { fmt.Println(v) }

x := 42
printVal(x)     // x may escape — storing into interface{}

sync.Pool — reuse heap-allocated objects:

var bufPool = sync.Pool{
    New: func() interface{} { return new(bytes.Buffer) },
}

buf := bufPool.Get().(*bytes.Buffer)
buf.Reset()
defer bufPool.Put(buf)
// Process using buf

Go uses a work-stealing M:N scheduler that multiplexes goroutines (G) onto OS threads (M) via processors (P). This enables millions of goroutines with minimal OS overhead.

G-M-P model:

• G (Goroutine): Lightweight coroutine. Starts with a ~2KB stack that grows/shrinks dynamically (up to 1GB by default). Cheap to create (~2–8µs, a few hundred bytes).
• M (Machine/OS Thread): Actual OS thread. Number determined by GOMAXPROCS and blocking I/O. Each M must hold a P to execute Go code.
• P (Processor): Logical processor. Holds a local run queue of goroutines. Number = GOMAXPROCS (default = number of CPU cores). The P is the scheduling context — it carries memory caches and the goroutine queue.

Scheduling lifecycle:

1. When a goroutine is ready to run, it's placed on a P's local run queue
2. Each M picks goroutines from its P's local queue
3. If a P's local queue is empty, it steals from another P's queue (work stealing)
4. When a goroutine blocks on a syscall, the M is detached from P; another M picks up the P and continues scheduling
5. On goroutine unblock (I/O ready), it's placed back on a run queue

// Control parallelism
runtime.GOMAXPROCS(4)   // 4 OS threads for Go code
runtime.GOMAXPROCS(0)   // query current value

// Default: GOMAXPROCS = number of CPUs
// In containers: may need explicit setting based on container CPU limit

Goroutine preemption: Since Go 1.14, goroutines are asynchronously preemptible — the scheduler can interrupt a goroutine at any safe point, not just at function calls. This prevents long-running goroutines from starving others.

Go's memory model specifies the conditions under which a goroutine's write to a variable is guaranteed to be visible to a read by another goroutine. Without explicit synchronization, there are no such guarantees — the compiler and CPU can reorder operations.

Happens-before guarantees:

• Channel sends happen-before the corresponding receive completes. A send on an unbuffered channel happens-before the receive returns.
• Channel close happens-before a receive that returns the zero value because the channel is closed.
• sync.Mutex Lock/Unlock: An Unlock call happens-before any Lock that it enables.
• sync.Once.Do: The completion of the first Do call happens-before any other Do returns.
• Goroutine creation: The go statement happens-before the goroutine's execution begins.

// UNSAFE — no synchronization
var x int
go func() { x = 1 }()
fmt.Println(x)   // may print 0 — no happens-before guarantee

// SAFE — channel provides happens-before
done := make(chan struct{})
go func() {
    x = 1
    close(done)  // happens-before receive below
}()
<-done
fmt.Println(x)   // guaranteed to print 1

// SAFE — mutex
var mu sync.Mutex
go func() { mu.Lock(); x = 1; mu.Unlock() }()
mu.Lock(); fmt.Println(x); mu.Unlock()

Data races: Concurrent unsynchronized access to shared memory (at least one write) is a data race — undefined behavior in Go. Use the race detector: go test -race ./... or go run -race main.go. Always run in CI.

Each goroutine starts with a small stack (~2–4KB). Unlike OS threads which have a fixed-size stack (typically 1–8MB), goroutine stacks grow and shrink dynamically. This is what makes it practical to have millions of goroutines.

Stack growth mechanism (segmented → contiguous):

• Go 1.3 and earlier: Segmented stacks — when a goroutine needed more stack space, a new segment was linked to the current one. The "hot split" problem: a function call at the stack boundary repeatedly triggered expensive stack growth/shrink.
• Go 1.4+: Contiguous stacks — when the stack overflows, a new stack 2× the size is allocated, all stack frames are copied to the new stack, and pointers are updated. Much simpler, no hot split issue.

Stack limits:

// Default max stack size per goroutine: 1GB (64-bit systems)
// Change with:
debug.SetMaxStack(512 * 1024 * 1024)  // 512 MB limit

// Stack overflow (infinite recursion) panics:
// "runtime: goroutine stack exceeds 1000000000-byte limit"
// "goroutine stack: [signal SIGSEGV: segmentation violation]"

Performance implications: Stack copying on growth has a cost. If you know a goroutine needs a large stack (deep recursion, large local variables), there's no way to pre-allocate (unlike some other languages). Keep function call stacks shallow in hot paths. Avoid large value types as local variables in frequently-called functions — they stay on the stack and cause expensive copies on growth.

Go has excellent built-in profiling via net/http/pprof and the pprof tool.

// Enable HTTP profiling endpoint
import _ "net/http/pprof"
go http.ListenAndServe("localhost:6060", nil)

// Collect profiles:
go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30  // CPU
go tool pprof http://localhost:6060/debug/pprof/heap                // Memory
go tool pprof http://localhost:6060/debug/pprof/goroutine           // Goroutines
go tool pprof http://localhost:6060/debug/pprof/block               // Blocking
go tool pprof http://localhost:6060/debug/pprof/mutex               // Mutex contention

// In pprof interactive mode:
top10        // top 10 by self time
top10 -cum   // top 10 by cumulative time
web          // open flame graph in browser (requires graphviz)
list funcName // source-annotated view of a function

Profile types:

• CPU profile: Samples goroutine stacks every 10ms. Shows where CPU time is spent. Best for finding hot paths.
• Heap profile: Shows current live heap allocations by allocation site. Identifies memory hogs and leaks. Use -alloc_objects for allocation rate.
• Goroutine profile: Stack traces of all goroutines. Identifies goroutine leaks (goroutines accumulating over time).
• Block profile: Where goroutines block on synchronization (channels, mutexes). Needs runtime.SetBlockProfileRate(1).
• Mutex profile: Contended mutex locks. Needs runtime.SetMutexProfileFraction(1).

// Benchmark-integrated profiling
go test -bench=. -cpuprofile=cpu.out -memprofile=mem.out
go tool pprof cpu.out

Unlike languages without GC, Go memory leaks happen when reachable references prevent GC from collecting objects — not from forgotten frees.

Common causes:

• Goroutine leaks: The most common. A goroutine blocked on a channel or I/O that never unblocks — it holds its stack and all referenced data forever.

// Goroutine leak — nobody reads from ch
func leaky() {
    ch := make(chan int)
    go func() { ch <- work() }()  // goroutine blocked forever if nobody receives
    // ch goes out of scope — goroutine stuck
}

// Fix: use context for cancellation
func fixed(ctx context.Context) (int, error) {
    ch := make(chan int, 1)
    go func() {
        select {
        case ch <- work():
        case <-ctx.Done():
        }
    }()
    select {
    case result := <-ch: return result, nil
    case <-ctx.Done(): return 0, ctx.Err()
    }
}

• Slice/map retaining large backing arrays (covered in slices section)
• Caches without eviction: Unbounded maps or slices used as caches grow forever. Use golang.org/x/sync/singleflight + TTL or an LRU cache.
• Lingering references in global variables: Adding to a global slice/map, never removing.
• Finalizers not being called: Objects with runtime.SetFinalizer in cycles prevent collection.

Detection:

// Monitor goroutine count
runtime.NumGoroutine()  // in metrics, alert if growing unboundedly

// Heap profile over time — compare two snapshots
// Use Prometheus + Grafana: go_goroutines, go_memstats_heap_inuse_bytes

// goleak — test that no goroutines leak after tests
import "go.uber.org/goleak"
func TestMain(m *testing.M) { goleak.VerifyTestMain(m) }

sync.Pool is a goroutine-safe pool of temporary objects that can be reused to reduce GC pressure. Objects in the pool may be garbage-collected at any time — it is NOT a resource pool for things like DB connections.

var bufPool = sync.Pool{
    New: func() any {
        return &bytes.Buffer{}  // called when pool is empty
    },
}

func processRequest(data []byte) []byte {
    buf := bufPool.Get().(*bytes.Buffer)  // get from pool or allocate
    defer func() {
        buf.Reset()           // clean up before returning
        bufPool.Put(buf)      // return to pool
    }()

    buf.Write(data)
    buf.WriteString(" processed")
    result := make([]byte, buf.Len())
    copy(result, buf.Bytes())
    return result
}

// Real-world: encoding/json uses sync.Pool for encoder buffers
// fmt package uses sync.Pool for pp (print) objects

Guarantees and properties:

• Objects in the pool may be GC'd at any GC cycle — the pool is cleared on GC
• Safe for concurrent use — no external locking needed
• Per-P local pools reduce contention — objects are associated with the current P
• No guarantee an object put back will be the same one retrieved next

When to use: Short-lived, expensive-to-allocate objects used in hot paths — byte buffers, encoder/decoder objects, parser state. The pooled object must be safely resettable to a clean state before reuse.

When NOT to use: Objects with exclusive ownership semantics (DB connections, files); objects that must outlive a single operation; when correctness depends on the object being a specific instance.

Go's concurrency model is based on CSP (Communicating Sequential Processes) — goroutines communicate by passing data through channels rather than sharing mutable state protected by locks.

Traditional approach (share memory): Multiple threads share a common data structure, protect it with a mutex. All coordination is implicit — any thread can access the shared state at any time. Subtle bugs arise from forgotten locks, wrong lock granularity, deadlocks.

Go's preferred approach (communicate ownership): Data is owned by exactly one goroutine at a time. Ownership is transferred via channels. No shared mutable state → no data races → no locks needed.

// Traditional: shared state with mutex
var counter int
var mu sync.Mutex
// Multiple goroutines call:
mu.Lock(); counter++; mu.Unlock()

// Go idiomatic: communicate ownership
func counter(increment <-chan struct{}, query <-chan chan int) {
    count := 0  // owned exclusively by this goroutine
    for {
        select {
        case <-increment: count++
        case reply := <-query: reply <- count  // send ownership of the value
        }
    }
}

// Caller
inc := make(chan struct{})
qry := make(chan chan int)
go counter(inc, qry)

inc <- struct{}{}  // increment
reply := make(chan int)
qry <- reply       // query
fmt.Println(<-reply)

In practice: Both approaches are valid and common in Go. Use channels for orchestration (pipelines, fan-out, fan-in) and mutexes for protecting simple shared state (counters, caches). The key insight: if data flows through your program (requests, tasks, results), channels model that naturally. If you're protecting a cache or counter, a mutex is simpler.

Unbuffered channel (make(chan T)): Sender blocks until a receiver is ready; receiver blocks until a sender is ready. A send and receive happen simultaneously — a synchronization point. Also called a "rendezvous" channel.

Buffered channel (make(chan T, n)): Sender only blocks when the buffer is full; receiver blocks when the buffer is empty. Decouples sender and receiver — they don't need to synchronize.

// Unbuffered — synchronous handoff
ch := make(chan int)
go func() { ch <- 42 }()  // blocks until receiver ready
val := <-ch                // both synchronize here

// Buffered — async within capacity
ch := make(chan int, 3)
ch <- 1; ch <- 2; ch <- 3  // non-blocking (buffer not full)
ch <- 4                     // blocks — buffer full
val := <-ch                // 1 (FIFO)

When to use unbuffered:

• When you need guaranteed synchronization between goroutines
• Done signals: done := make(chan struct{})
• When you want the sender to wait until the receiver has processed the value

When to use buffered:

• Worker pools — decouple task submission from processing
• Rate limiting — capacity limits outstanding work
• When a goroutine should not block when the consumer is temporarily slow
• Semaphore pattern: sem := make(chan struct{}, maxConcurrent)

// Semaphore — limit concurrent operations
sem := make(chan struct{}, 5)  // max 5 concurrent
for _, url := range urls {
    sem <- struct{}{}   // acquire
    go func(u string) {
        defer func() { <-sem }()  // release
        fetch(u)
    }(url)
}

select waits on multiple channel operations simultaneously, proceeding when one is ready. If multiple cases are ready, one is chosen at random.

// Basic select
select {
case msg := <-ch1: fmt.Println("ch1:", msg)
case msg := <-ch2: fmt.Println("ch2:", msg)
case ch3 <- "hello": fmt.Println("sent to ch3")
}

// Timeout pattern
select {
case result := <-resultCh:
    return result, nil
case <-time.After(5 * time.Second):
    return nil, errors.New("timeout")
}
// Note: time.After leaks a timer until it fires.
// For repeated use, prefer:
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()

// Non-blocking channel operation — default case
select {
case msg := <-ch:
    process(msg)
default:
    // ch is empty, do something else
}

// Cancellation via context
func doWork(ctx context.Context, ch <-chan int) error {
    for {
        select {
        case val, ok := <-ch:
            if !ok { return nil }  // channel closed
            process(val)
        case <-ctx.Done():
            return ctx.Err()  // cancelled or timed out
        }
    }
}

// nil channel — permanently blocks, useful for disabling a case
var slowCh chan int  // nil
select {
case <-fastCh: doFast()
case <-slowCh:  // never selected — nil channel blocks forever
}

context.Context carries deadlines, cancellation signals, and request-scoped values across API boundaries and goroutines. It is Go's standard mechanism for cancellation propagation.

// Creating contexts
ctx := context.Background()                         // root — never cancelled
ctx := context.TODO()                               // placeholder — replace later

ctx, cancel := context.WithCancel(parent)           // manual cancel
defer cancel()                                      // ALWAYS defer cancel

ctx, cancel := context.WithTimeout(parent, 5*time.Second)  // deadline in duration
defer cancel()

ctx, cancel := context.WithDeadline(parent, time.Now().Add(5*time.Second))
defer cancel()

ctx = context.WithValue(ctx, requestIDKey{}, "req-123")  // attach value
rid := ctx.Value(requestIDKey{}).(string)                 // retrieve value

// Check if cancelled
select {
case <-ctx.Done():
    return ctx.Err()  // context.Canceled or context.DeadlineExceeded
default:
}

Rules for using context correctly:

• First parameter, named ctx: Pass ctx as the first argument to every function that needs it — func DoSomething(ctx context.Context, ...) error
• Never store in a struct: Store in a local variable or pass explicitly. The exception is when implementing an interface you don't control.
• Always call the cancel function: Not calling cancel leaks resources. Use defer cancel() immediately after creation.
• Context values for request-scoped data only: Don't pass function parameters via context. Use only for cross-cutting data: request IDs, trace spans, auth tokens. Use typed unexported keys to avoid collisions.
• Pass the context down, not sideways: Every goroutine you launch should receive and respect the parent context.

// Typed key to avoid collisions
type contextKey struct{ name string }
var requestIDKey = contextKey{"requestID"}

ctx = context.WithValue(ctx, requestIDKey, rid)

// PIPELINE — chain of goroutines connected by channels
func generate(nums ...int) <-chan int {
    out := make(chan int)
    go func() { defer close(out); for _, n := range nums { out <- n } }()
    return out
}
func square(in <-chan int) <-chan int {
    out := make(chan int)
    go func() { defer close(out); for n := range in { out <- n * n } }()
    return out
}
// Chain: generate → square → print
for n := range square(generate(2, 3, 4)) { fmt.Println(n) }

// FAN-OUT — distribute work across multiple goroutines
func fanOut(in <-chan Work, workers int) []<-chan Result {
    channels := make([]<-chan Result, workers)
    for i := range channels {
        channels[i] = process(in)  // each reads from the same input
    }
    return channels
}

// FAN-IN (merge) — combine multiple channels into one
func merge(channels ...<-chan int) <-chan int {
    var wg sync.WaitGroup
    merged := make(chan int, 100)
    output := func(ch <-chan int) {
        defer wg.Done()
        for v := range ch { merged <- v }
    }
    wg.Add(len(channels))
    for _, ch := range channels { go output(ch) }
    go func() { wg.Wait(); close(merged) }()
    return merged
}

// WORKER POOL
func workerPool(ctx context.Context, jobs <-chan Job, numWorkers int) <-chan Result {
    results := make(chan Result, numWorkers)
    var wg sync.WaitGroup
    for i := 0; i < numWorkers; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for job := range jobs {
                select {
                case results <- process(job):
                case <-ctx.Done(): return
                }
            }
        }()
    }
    go func() { wg.Wait(); close(results) }()
    return results
}

// WaitGroup — wait for a collection of goroutines to finish
var wg sync.WaitGroup
for _, task := range tasks {
    wg.Add(1)
    go func(t Task) {
        defer wg.Done()  // ALWAYS in defer — safe even on panic
        process(t)
    }(task)
}
wg.Wait()  // blocks until all Done() calls

// Mutex — mutual exclusion for shared state
type SafeCounter struct {
    mu    sync.Mutex
    count int
}
func (c *SafeCounter) Inc() {
    c.mu.Lock()
    defer c.mu.Unlock()
    c.count++
}

// RWMutex — multiple readers OR one writer
type Cache struct {
    mu    sync.RWMutex
    items map[string]string
}
func (c *Cache) Get(key string) (string, bool) {
    c.mu.RLock()          // concurrent reads allowed
    defer c.mu.RUnlock()
    v, ok := c.items[key]
    return v, ok
}
func (c *Cache) Set(key, val string) {
    c.mu.Lock()           // exclusive write
    defer c.mu.Unlock()
    c.items[key] = val
}

// sync.Once — exactly-once initialization
var (
    instance *DB
    once     sync.Once
)
func GetDB() *DB {
    once.Do(func() { instance = connectDB() })  // runs exactly once
    return instance
}

// sync.Cond — wait for condition (advanced, rarely needed)
// Prefer channels or sync.WaitGroup for most cases

Mutex vs channel: Use a mutex when protecting a data structure (cache, counter, map). Use a channel when communicating between goroutines or expressing a workflow. Mutexes are simpler and more efficient for pure state protection; channels are better for coordination and signaling.

golang.org/x/sync/errgroup provides a WaitGroup-like API that also collects errors and (optionally) cancels a shared context when the first error occurs.

import "golang.org/x/sync/errgroup"

// Basic errgroup — wait for all goroutines, collect first error
func fetchAll(urls []string) ([][]byte, error) {
    g := new(errgroup.Group)
    results := make([][]byte, len(urls))

    for i, url := range urls {
        i, url := i, url  // capture loop variables (pre-1.22)
        g.Go(func() error {
            data, err := fetch(url)
            if err != nil { return err }
            results[i] = data
            return nil
        })
    }

    if err := g.Wait(); err != nil { return nil, err }
    return results, nil
}

// With context — cancel all goroutines on first error
func fetchAllWithCancel(ctx context.Context, urls []string) error {
    g, ctx := errgroup.WithContext(ctx)  // derives cancellable context

    for _, url := range urls {
        url := url
        g.Go(func() error {
            req, _ := http.NewRequestWithContext(ctx, "GET", url, nil)
            resp, err := http.DefaultClient.Do(req)
            if err != nil { return err }  // other goroutines will be cancelled
            defer resp.Body.Close()
            return process(resp.Body)
        })
    }

    return g.Wait()  // all errors → returns first non-nil
}

// Limit concurrency
g.SetLimit(10)  // at most 10 goroutines at a time (errgroup v0.4+)

errgroup is the idiomatic Go solution for "launch N goroutines, wait for all, return first error" — a pattern that would otherwise require careful management of WaitGroup + error channel + cancellation.

A data race occurs when two goroutines access the same memory location concurrently and at least one access is a write, without synchronization. Data races cause undefined behavior — silent data corruption, crashes, or apparently correct but wrong results.

Race detector — always use in testing:

go test -race ./...           // run all tests with race detection
go run -race main.go          // run with race detection
go build -race -o app_race    // build with race detection

// Output on detection:
// WARNING: DATA RACE
// Write at 0x00c0000b4010 by goroutine 7:
//   main.increment(...)
// Previous read at 0x00c0000b4010 by goroutine 6:
//   main.read(...)

Prevention patterns:

// 1. Use atomic operations for simple counters/flags
var count int64
atomic.AddInt64(&count, 1)          // atomic increment
val := atomic.LoadInt64(&count)     // atomic read

// 2. Use channels — only one goroutine owns data at a time
// 3. Use sync.Mutex/RWMutex for shared state
// 4. Use sync.Map for concurrent map access
// 5. Confine data to a single goroutine — pass copies, not references

// Common race: modifying a slice across goroutines
// RACE:
results := []int{}
var wg sync.WaitGroup
for i := 0; i < 10; i++ {
    wg.Add(1)
    go func(n int) { defer wg.Done(); results = append(results, n) }(i)
}
// FIX: pre-allocate and write to separate indices
results := make([]int, 10)
for i := 0; i < 10; i++ {
    wg.Add(1)
    go func(n int) { defer wg.Done(); results[n] = compute(n) }(i)
}
wg.Wait()  // each goroutine writes to its own index — no race

A goroutine leak occurs when a goroutine is created but never terminates — it runs forever (or until the program exits), consuming memory and CPU. In long-running services, leaks accumulate and eventually exhaust resources.

Common causes and fixes:

// LEAK 1: Goroutine blocked on unbuffered channel with no receiver
func leaky() {
    ch := make(chan result)
    go func() { ch <- doWork() }()  // blocks forever if caller returns early
    // Fix: buffer the channel or ensure receive always happens
    ch := make(chan result, 1)  // buffered: goroutine can always send
}

// LEAK 2: Goroutine waiting on channel in infinite loop with no exit
func worker(jobs <-chan Job) {
    for job := range jobs { process(job) }  // exits when jobs is closed
    // Must close(jobs) from sender to allow this to return
}

// LEAK 3: Context not propagated — goroutine ignores cancellation
go func() {
    for {
        data := fetchFromDB()    // blocks — ignores ctx
        process(data)
    }
}()
// Fix: pass context, select on ctx.Done()
go func(ctx context.Context) {
    for {
        select {
        case <-ctx.Done(): return
        default:
            data := fetchWithCtx(ctx)
            process(data)
        }
    }
}(ctx)

// Detection
import "go.uber.org/goleak"
func TestNoLeaks(t *testing.T) {
    defer goleak.VerifyNone(t)
    // ... test code ...
}

// Monitor in production
go func() {
    for {
        metrics.Gauge("goroutines", float64(runtime.NumGoroutine()))
        time.Sleep(30 * time.Second)
    }
}()

The sync/atomic package provides low-level atomic operations that are guaranteed to be executed as a single, indivisible hardware operation — no goroutine can observe a partially-completed atomic operation.

import "sync/atomic"

// Atomic counter — no mutex needed
var requestCount int64
atomic.AddInt64(&requestCount, 1)          // increment
n := atomic.LoadInt64(&requestCount)       // read
atomic.StoreInt64(&requestCount, 0)        // reset

// CAS — Compare And Swap (optimistic locking)
var state int32
for {
    old := atomic.LoadInt32(&state)
    new := old | flagBit
    if atomic.CompareAndSwapInt32(&state, old, new) {
        break  // success — state updated atomically
    }
    // retry — another goroutine changed state
}

// Go 1.19+: typed atomic values
var flag atomic.Bool
flag.Store(true)
if flag.Load() { ... }
flag.Swap(false)
flag.CompareAndSwap(true, false)

var ptr atomic.Pointer[Config]
ptr.Store(newConfig)
cfg := ptr.Load()  // always gets a consistent pointer

atomic vs mutex:

• Use atomic when: operating on a single primitive value (int, bool, pointer); the operation is a single load, store, add, or CAS; you need maximum performance on a hot path. Atomics are lock-free — no context switches, no scheduler involvement.
• Use mutex when: protecting a compound data structure (map, struct with multiple fields); when invariants span multiple variables; when you need to read-then-write atomically (check-then-act). Atomics only protect individual values — you can't atomically update two separate variables with atomic operations.

Premature optimization with atomics makes code harder to reason about. Default to mutexes, switch to atomics when profiling shows lock contention is a bottleneck.

The thundering herd problem: when a cache expires or a service restarts, many concurrent requests for the same key simultaneously miss the cache and all make the same expensive downstream call, overwhelming the backend.

golang.org/x/sync/singleflight deduplicates concurrent identical calls — only one call executes for a given key; all other callers wait and receive the same result.

import "golang.org/x/sync/singleflight"

type UserService struct {
    db    *DB
    cache *Cache
    sfg   singleflight.Group
}

func (s *UserService) GetUser(ctx context.Context, id int64) (*User, error) {
    // Check cache first
    if user, ok := s.cache.Get(id); ok { return user, nil }

    // Use singleflight to deduplicate concurrent requests for same ID
    key := fmt.Sprintf("user:%d", id)
    v, err, _ := s.sfg.Do(key, func() (any, error) {
        // Only ONE goroutine executes this block per key
        // All other concurrent callers with same key wait here
        user, err := s.db.GetUser(ctx, id)
        if err != nil { return nil, err }
        s.cache.Set(id, user, 5*time.Minute)
        return user, nil
    })
    if err != nil { return nil, err }
    return v.(*User), nil
}

// The third return value is "shared" — true if result was shared
// with other callers (not the one who actually did the work)

// DoChan — async variant, returns a channel
ch := sfg.DoChan("key", func() (any, error) { return doWork() })
select {
case res := <-ch.Chan: ...
case <-ctx.Done(): ...
}

import "golang.org/x/time/rate"

// Token bucket rate limiter — most common
limiter := rate.NewLimiter(rate.Limit(100), 10)
// 100 requests/second, burst of 10

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    if err := limiter.Wait(r.Context()); err != nil {
        http.Error(w, "rate limited", http.StatusTooManyRequests)
        return
    }
    // handle request
}

// Per-client limiting with a map of limiters
type IPRateLimiter struct {
    mu       sync.RWMutex
    limiters map[string]*rate.Limiter
    rate     rate.Limit
    burst    int
}

func (l *IPRateLimiter) getLimiter(ip string) *rate.Limiter {
    l.mu.RLock()
    limiter, ok := l.limiters[ip]
    l.mu.RUnlock()
    if ok { return limiter }

    limiter = rate.NewLimiter(l.rate, l.burst)
    l.mu.Lock()
    l.limiters[ip] = limiter
    l.mu.Unlock()
    return limiter
}

// Simple channel-based rate limiter
ticker := time.NewTicker(time.Second / 10)  // 10 req/s
defer ticker.Stop()
for req := range requests {
    <-ticker.C  // block until next tick
    go handle(req)
}

// Allow/Reserve for non-blocking check
if !limiter.Allow() {
    return errors.New("rate limit exceeded")
}

For distributed rate limiting (across multiple instances), use Redis with sliding window or token bucket algorithms — libraries like go-redis/redis_rate implement this with Lua scripts for atomicity.

Go interfaces are implicitly satisfied — a type implements an interface by having all the required methods. No implements keyword, no declaration of intent. This is structural typing.

type Stringer interface { String() string }

type User struct{ Name string }
func (u User) String() string { return u.Name }  // implicitly satisfies Stringer

func printAll(items []Stringer) {
    for _, item := range items { fmt.Println(item.String()) }
}

// Verify at compile time (useful for library authors)
var _ Stringer = User{}  // compile error if User doesn't implement Stringer

Interface internals — two-word structure: An interface value holds two pointers: a pointer to the concrete type's method table (itab), and a pointer to the concrete value. A nil interface has both pointers nil.

// The nil interface trap
var err error        // nil interface — both pointers are nil
err == nil           // true

var e *MyError = nil // concrete nil pointer
err = e              // err is interface with non-nil type pointer!
err == nil           // FALSE — the type pointer is set

// Fix: return nil directly, not a typed nil
func mayFail() error {
    var err *MyError
    if problem { err = &MyError{} }
    if err != nil { return err }
    return nil  // return untyped nil, not err
}

Empty interface any (alias for interface{}): Accepts any value. Use sparingly — it loses type safety. Prefer generics or concrete types. Common legitimate uses: fmt.Println, JSON marshaling, middleware that handles arbitrary request bodies.

Type assertion and type switch:

var i any = "hello"
s := i.(string)         // panics if not string
s, ok := i.(string)     // safe: ok=false instead of panic

switch v := i.(type) {
case string:  fmt.Println("string:", v)
case int:     fmt.Println("int:", v)
default:      fmt.Printf("unknown: %T\n", v)
}

Interface Segregation Principle is especially important in Go: Keep interfaces small. The ideal Go interface has one or two methods. This is idiomatic — the standard library is full of tiny interfaces.

// Standard library examples — small, focused interfaces
type Reader interface { Read(p []byte) (n int, err error) }
type Writer interface { Write(p []byte) (n int, err error) }
type ReadWriter interface { Reader; Writer }  // composition
type Closer interface { Close() error }
type Stringer interface { String() string }
type Error interface { Error() string }

// Compose interfaces from smaller ones
type ReadWriteCloser interface { Reader; Writer; Closer }

Key difference from Java: In Go, interfaces are defined by the consumer, not the implementor. Define the interface where you need it, not where you implement it. This enables loose coupling without a shared package for every abstraction.

// Good Go — interface defined at the call site
package httphandler

type UserStore interface {  // defined here, where it's needed
    GetUser(id int) (*User, error)
}

func NewHandler(store UserStore) *Handler { ... }

// PostgresUserStore in the db package satisfies this automatically
// No shared "interfaces" package needed
// No cyclic imports

Accept interfaces, return concrete types: Functions should accept interfaces (flexible, testable) but return concrete types (specific, informative). Returning an interface hides information and forces callers to type-assert if they need more specific behavior.

Embedding promotes the methods and fields of the embedded type to the outer type — syntactic sugar for delegation. It is NOT inheritance — there is no subtype relationship.

type Base struct { ID int }
func (b *Base) Describe() string { return fmt.Sprintf("ID=%d", b.ID) }

type Widget struct {
    Base            // embedded — promotes Base.ID and Base.Describe
    Name string
}

w := Widget{Base: Base{ID: 1}, Name: "button"}
w.ID             // promoted field access (equivalent to w.Base.ID)
w.Describe()     // promoted method (equivalent to w.Base.Describe())
w.Base.Describe() // explicit — same result

// Embedding interfaces — implement interface via delegation
type Logger interface { Log(msg string) }

type Service struct {
    Logger                   // embed interface
    db *DB
}
// Any Logger passed to Service is automatically promoted
// Useful for optional dependencies with nil-safety

Gotcha 1 — method set of pointer vs value:

type Animal struct{}
func (a *Animal) Breathe() {}   // pointer receiver

type Dog struct { Animal }      // value embed

var d Dog
d.Breathe()    // OK — Go auto-dereferences: (&d.Animal).Breathe()
var i interface{ Breathe() } = &d  // OK
var i interface{ Breathe() } = d   // COMPILE ERROR — Dog doesn't have Breathe, *Dog does

Gotcha 2 — outer methods shadow embedded:

func (w *Widget) Describe() string { return "Widget: " + w.Name }
// Now w.Describe() calls Widget.Describe, NOT Base.Describe
// Base.Describe is still accessible explicitly: w.Base.Describe()

Interface embedding — composing interfaces:

type ReadWriteCloser interface {
    io.Reader
    io.Writer
    io.Closer
}  // requires all three methods

Functions are first-class values in Go. Function types can be used as variables, parameters, and return values. You can even define methods on function types.

// Function type declaration
type HandlerFunc func(w http.ResponseWriter, r *http.Request)

// Method on a function type (how http.HandlerFunc works)
func (f HandlerFunc) ServeHTTP(w http.ResponseWriter, r *http.Request) { f(w, r) }

// Middleware pattern — function transforms a function
type Middleware func(http.Handler) http.Handler

func LoggingMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        start := time.Now()
        next.ServeHTTP(w, r)  // call the wrapped handler
        log.Printf("%s %s %v", r.Method, r.URL.Path, time.Since(start))
    })
}

func chain(h http.Handler, middlewares ...Middleware) http.Handler {
    for i := len(middlewares) - 1; i >= 0; i-- {
        h = middlewares[i](h)
    }
    return h
}

handler := chain(myHandler, LoggingMiddleware, AuthMiddleware, RateLimitMiddleware)

Functional options pattern — clean, extensible API design for structs with many optional fields:

type Server struct {
    host    string
    port    int
    timeout time.Duration
    maxConn int
}

type Option func(*Server)

func WithHost(host string) Option       { return func(s *Server) { s.host = host } }
func WithPort(port int) Option          { return func(s *Server) { s.port = port } }
func WithTimeout(d time.Duration) Option { return func(s *Server) { s.timeout = d } }

func NewServer(opts ...Option) *Server {
    s := &Server{host: "localhost", port: 8080, timeout: 30 * time.Second}
    for _, opt := range opts { opt(s) }
    return s
}

// Usage — clean, self-documenting
srv := NewServer(
    WithHost("0.0.0.0"),
    WithPort(9090),
    WithTimeout(60 * time.Second),
)

io.Reader and io.Writer are Go's most important interfaces. Their simplicity makes them universally composable.

type Reader interface { Read(p []byte) (n int, err error) }
type Writer interface { Write(p []byte) (n int, err error) }

// Read contract:
// - reads up to len(p) bytes into p
// - returns bytes read and any error
// - returns io.EOF when no more data
// - may return n > 0 with err != nil simultaneously

Why they're powerful — everything implements them:

// All of these implement io.Reader:
os.File, *bytes.Buffer, *strings.Reader,
net.Conn, http.Request.Body, gzip.Reader,
crypto/cipher.StreamReader, io.LimitedReader

// Compose them transparently:
f, _ := os.Open("data.gz")
gz, _ := gzip.NewReader(f)            // wraps file reader
buf := bufio.NewReader(gz)            // wraps gzip reader
io.Copy(os.Stdout, buf)               // stream: file→gunzip→buffer→stdout
// No intermediate allocations of the full content!

// Your function works with ALL readers
func process(r io.Reader) error {
    scanner := bufio.NewScanner(r)
    for scanner.Scan() { handle(scanner.Text()) }
    return scanner.Err()
}
// Call with file, HTTP body, in-memory buffer, test string — same code

Key helpers:

io.Copy(dst Writer, src Reader) (int64, error)  // stream copy
io.ReadAll(r Reader) ([]byte, error)             // read entire reader (careful with size)
io.LimitReader(r, n)                             // limit to n bytes
io.TeeReader(r, w)                               // read AND write to w
io.MultiWriter(writers...)                       // write to multiple destinations
io.MultiReader(readers...)                       // chain readers sequentially
io.Pipe()                                        // synchronous in-memory pipe

// TYPE DEFINITION — creates a new, distinct type
type Celsius float64
type Fahrenheit float64
type UserID int64

var c Celsius = 100
var f Fahrenheit = 100
// c = f  // COMPILE ERROR — different types, even though both are float64

// New type has NO methods from underlying type (except operators)
// Must re-define methods or convert explicitly
func (c Celsius) String() string { return fmt.Sprintf("%.1f°C", float64(c)) }

// TYPE ALIAS — just another name for the same type
type MyHandler = http.HandlerFunc  // MyHandler IS http.HandlerFunc, identical
type rune = int32                  // built-in alias
type byte = uint8                  // built-in alias

// Alias has all methods of the original type
// Used for: gradual code migration, creating shorter names,
// exposing types from another package

When to use type definitions:

• Domain modeling — UserID, OrderID, Celsius prevent accidental mixing of semantically different values
• Adding methods to types you don't own: type StringSlice []string; func (s StringSlice) Sort()
• Enum-like constants: type Status int; const (Active Status = iota; Inactive)

When to use aliases:

• Large-scale refactoring — move a type to a new package while keeping backward compat
• Re-exporting types: package api; type User = models.User
• Rarely needed in application code — type definitions are almost always what you want

import "encoding/json"

type User struct {
    ID        int       `json:"id"`
    Name      string    `json:"name"`
    Email     string    `json:"email,omitempty"`  // omit if empty string
    Password  string    `json:"-"`                 // always omit
    CreatedAt time.Time `json:"created_at"`
    internal  string    // unexported — always omitted
}

// Marshal — struct to JSON
u := User{ID: 1, Name: "Alice"}
data, err := json.Marshal(u)
// {"id":1,"name":"Alice","created_at":"0001-01-01T00:00:00Z"}
// Note: Email omitted (empty + omitempty), Password omitted (-)

// Unmarshal — JSON to struct
var u User
err = json.Unmarshal(data, &u)  // must pass pointer

// Streaming — for large data or HTTP
json.NewEncoder(w).Encode(u)      // write to io.Writer
json.NewDecoder(r.Body).Decode(&u) // read from io.Reader

Custom marshal/unmarshal:

type Money struct { Amount int64; Currency string }

func (m Money) MarshalJSON() ([]byte, error) {
    return json.Marshal(fmt.Sprintf("%d %s", m.Amount, m.Currency))
}
func (m *Money) UnmarshalJSON(data []byte) error {
    var s string
    if err := json.Unmarshal(data, &s); err != nil { return err }
    _, err := fmt.Sscanf(s, "%d %s", &m.Amount, &m.Currency)
    return err
}

Common pitfalls:

• Unmarshaling into a nil pointer panics — always pass an initialized pointer
• Numbers in JSON unmarshal to float64 when target is any — use json.Number or decode into a typed struct
• interface{} fields get map[string]interface{} for objects — hard to work with
• Unexported fields are silently ignored — a common source of confusion
• For performance-critical code, use github.com/json-iterator/go or github.com/bytedance/sonic (drop-in replacements, 3-5x faster)

The reflect package allows inspecting and manipulating types and values at runtime. It's how encoding/json, database/sql, and many frameworks work under the hood.

import "reflect"

// Inspect type and value
x := 42
t := reflect.TypeOf(x)   // reflect.Type: "int"
v := reflect.ValueOf(x)  // reflect.Value: 42
t.Kind()                  // reflect.Int
v.Int()                   // 42

// Inspect struct fields
type User struct {
    Name string `json:"name" validate:"required"`
    Age  int    `json:"age"`
}
u := User{Name: "Alice", Age: 30}
t = reflect.TypeOf(u)
for i := 0; i < t.NumField(); i++ {
    field := t.Field(i)
    value := reflect.ValueOf(u).Field(i)
    fmt.Printf("%s: %v (json:%s)\n",
        field.Name, value, field.Tag.Get("json"))
}

// Modify via reflection — must pass pointer, get Elem()
v = reflect.ValueOf(&u).Elem()
v.FieldByName("Name").SetString("Bob")

// Create new instances dynamically
newUser := reflect.New(reflect.TypeOf(User{})).Interface().(*User)

When to use reflection:

• Serialization/deserialization frameworks (JSON, YAML, DB ORM)
• Dependency injection containers
• Test frameworks and mock generators
• Generic utility functions that must work with arbitrary types (before generics)

When to avoid: In normal application code. Reflection bypasses type safety, is slow (10-100x compared to direct access), and makes code hard to read. With Go 1.18+ generics, many use cases that previously required reflection can now be handled type-safely.

In Go, errors are values — any type implementing the error interface can be an error.

type error interface { Error() string }

// Functions return (result, error) tuples
func divide(a, b float64) (float64, error) {
    if b == 0 { return 0, errors.New("division by zero") }
    return a / b, nil
}

result, err := divide(10, 0)
if err != nil {
    log.Printf("failed: %v", err)
    return
}
fmt.Println(result)

Why values instead of exceptions:

• Explicit control flow: Every error must be acknowledged. No hidden jumps — you can read a Go function and know exactly which calls can fail.
• Errors as information: Errors carry rich context. You can wrap, unwrap, inspect, and compare them.
• Performance: No stack unwinding. Exception handling in Java/Python has significant overhead for the "exceptional path".
• Composition: Errors compose naturally — you can collect multiple errors, retry on specific error types, add context as they propagate up.

The criticism — "if err != nil" verbosity is real. Go 2 proposals exist but nothing has shipped. The trade-off is intentional: verbose but honest code over concise but hidden control flow.

// Custom error type with context
type ValidationError struct {
    Field   string
    Message string
}
func (e *ValidationError) Error() string {
    return fmt.Sprintf("validation failed: %s - %s", e.Field, e.Message)
}

Go 1.13 introduced error wrapping — the ability to add context to errors while preserving the original for inspection.

// Wrap with context using %w
func getUser(id int) (*User, error) {
    user, err := db.QueryUser(id)
    if err != nil {
        return nil, fmt.Errorf("getUser(%d): %w", id, err)
        // message: "getUser(42): sql: no rows in result set"
        // wrapped error is unwrappable
    }
    return user, nil
}

// Sentinel errors — comparable values
var (
    ErrNotFound   = errors.New("not found")
    ErrUnauthorized = errors.New("unauthorized")
)

// errors.Is — checks if ANY error in the chain matches
err := fmt.Errorf("db lookup: %w", ErrNotFound)
errors.Is(err, ErrNotFound)  // true — unwraps chain to find it

// errors.As — extracts a specific type from the error chain
type DBError struct { Code int; Msg string }
func (e *DBError) Error() string { return e.Msg }

err = fmt.Errorf("query failed: %w", &DBError{Code: 503, Msg: "timeout"})
var dbErr *DBError
if errors.As(err, &dbErr) {
    fmt.Println("DB error code:", dbErr.Code)  // 503
}

// errors.Unwrap — get the next error in chain
errors.Unwrap(err)  // returns the wrapped error or nil

Best practices: Add context at each layer (fmt.Errorf("operation: %w", err)); use sentinel errors for expected conditions callers check; use custom error types for rich error data; don't wrap if you're re-returning the exact same error from a thin wrapper.

panic stops normal execution, runs all deferred functions in the current goroutine, and propagates up the call stack. If not recovered, it crashes the program with a stack trace.

When to use panic:

• Programming errors that should never happen — index out of bounds, nil dereference (runtime panics), violated invariants in the same package
• Initialization failures that make the program unable to run: regexp.MustCompile("invalid[") // panics on bad regex at startup
• Impossible code paths: panic("unreachable")

Never use panic for: Expected error conditions, user input errors, network failures, file not found. These should be returned as error values.

// recover — must be called in a deferred function
func safeExecute(f func()) (err error) {
    defer func() {
        if r := recover(); r != nil {
            err = fmt.Errorf("panic: %v", r)
            // Optional: capture stack trace
            // buf := make([]byte, 4096)
            // n := runtime.Stack(buf, false)
            // err = fmt.Errorf("panic: %v\n%s", r, buf[:n])
        }
    }()
    f()
    return nil
}

// HTTP servers recover from panics per-request (net/http does this automatically)
// so one panicking handler doesn't crash the whole server

// Common pattern: library code panics, wraps in recover at package boundary
func (p *Parser) Parse(input string) (result Result, err error) {
    defer func() {
        if r := recover(); r != nil {
            err = fmt.Errorf("parse panic: %v", r)
        }
    }()
    return p.parse(input), nil  // internal parse may panic on corrupt input
}

// Multi-error — collect all errors, report all at once
import "errors"

type MultiError []error
func (m MultiError) Error() string {
    msgs := make([]string, len(m))
    for i, e := range m { msgs[i] = e.Error() }
    return strings.Join(msgs, "; ")
}

// Or use errors.Join (Go 1.20+)
err := errors.Join(err1, err2, err3)
// errors.Is and errors.As work across the joined errors

// Validation — collect all field errors
func validate(u *User) error {
    var errs []error
    if u.Name == "" { errs = append(errs, errors.New("name required")) }
    if u.Age < 0   { errs = append(errs, errors.New("age must be non-negative")) }
    if len(u.Email) == 0 { errs = append(errs, errors.New("email required")) }
    return errors.Join(errs...)
}

// Concurrent error collection — first error wins (errgroup)
g, ctx := errgroup.WithContext(ctx)
for _, item := range items {
    item := item
    g.Go(func() error { return process(ctx, item) })
}
if err := g.Wait(); err != nil { return err }

// Concurrent — collect ALL errors
var mu sync.Mutex
var errs []error
var wg sync.WaitGroup
for _, item := range items {
    wg.Add(1)
    go func(item Item) {
        defer wg.Done()
        if err := process(item); err != nil {
            mu.Lock(); errs = append(errs, err); mu.Unlock()
        }
    }(item)
}
wg.Wait()
return errors.Join(errs...)

Go's standard errors package does not capture stack traces — this is intentional (performance). For application code that benefits from traces, use a library.

// github.com/pkg/errors (classic, widely used)
import "github.com/pkg/errors"

func readConfig(path string) error {
    _, err := os.ReadFile(path)
    if err != nil {
        return errors.Wrap(err, "readConfig")  // captures stack trace here
    }
    return nil
}

// Print with stack trace
fmt.Printf("%+v", err)  // %+v triggers stack trace output

// errors.New from pkg/errors also captures stack trace
err := errors.New("something went wrong")

// golang.org/x/xerrors (predecessor to standard wrapping, less used now)

// go.uber.org/zap + go.uber.org/multierr for structured logging
logger.Error("operation failed", zap.Error(err))

// Modern approach: use standard %w wrapping + structured logging
// The log entry captures the call site automatically
// Stack traces add overhead — use only in error paths, not hot paths

// Capture stack trace manually if needed
func captureStack() string {
    buf := make([]byte, 4096)
    n := runtime.Stack(buf, false)
    return string(buf[:n])
}

Recommendation: For most production Go applications, use fmt.Errorf("context: %w", err) for wrapping and structured logging with zap or slog (Go 1.21+). The log entry includes file/line. Full stack traces are a debugging tool — capture them only at boundary points (HTTP handlers, async workers) if needed.

import "log/slog"

// Default logger (goes to stderr, text format)
slog.Info("user created", "id", 42, "name", "Alice")
slog.Error("db query failed", "error", err, "query", sql)
slog.Warn("high latency", "ms", 523, "endpoint", "/api/users")

// JSON handler for production (machine-parseable)
logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
    Level: slog.LevelInfo,
    AddSource: true,  // adds file:line to each log entry
}))
slog.SetDefault(logger)

// Structured logging with typed attributes (more efficient)
slog.Info("request processed",
    slog.Int("status", 200),
    slog.String("method", "GET"),
    slog.String("path", "/users"),
    slog.Duration("latency", 23*time.Millisecond),
)

// Logger with persistent context (request ID, trace ID)
reqLogger := logger.With(
    slog.String("request_id", rid),
    slog.String("trace_id", traceID),
)
reqLogger.Info("handling request")
reqLogger.Error("handler failed", slog.Any("error", err))

// Custom handler — filter, enrich, forward to multiple sinks
// slog.Handler interface: Enabled, Handle, WithAttrs, WithGroup

// slog vs zap/zerolog performance:
// slog is ~3x slower than zap for structured logging
// For extreme throughput, use zerolog (fastest) or zap
// For most services, slog is perfectly adequate

// HTTP server
mux := http.NewServeMux()

mux.HandleFunc("GET /users/{id}", func(w http.ResponseWriter, r *http.Request) {
    id := r.PathValue("id")      // Go 1.22 — path parameters!
    user, err := store.Get(r.Context(), id)
    if err != nil {
        http.Error(w, err.Error(), http.StatusNotFound)
        return
    }
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(user)
})

mux.HandleFunc("POST /users", func(w http.ResponseWriter, r *http.Request) {
    var req CreateUserRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }
    user, err := store.Create(r.Context(), req)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    w.WriteHeader(http.StatusCreated)
    json.NewEncoder(w).Encode(user)
})

srv := &http.Server{
    Addr:         ":8080",
    Handler:      mux,
    ReadTimeout:  5 * time.Second,
    WriteTimeout: 10 * time.Second,
    IdleTimeout:  120 * time.Second,
}
log.Fatal(srv.ListenAndServe())

// Graceful shutdown
quit := make(chan os.Signal, 1)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
<-quit
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
srv.Shutdown(ctx)

// HTTP client — always configure timeouts and use context
client := &http.Client{
    Timeout: 10 * time.Second,
    Transport: &http.Transport{
        MaxIdleConns:        100,
        MaxIdleConnsPerHost: 10,
        IdleConnTimeout:     90 * time.Second,
    },
}
req, _ := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
req.Header.Set("Authorization", "Bearer "+token)
resp, err := client.Do(req)
if err != nil { return err }
defer resp.Body.Close()
// ALWAYS read and close body even if you don't need it
io.Copy(io.Discard, resp.Body)

import (
    "database/sql"
    _ "github.com/lib/pq"  // postgres driver — blank import for side effects
)

db, err := sql.Open("postgres", os.Getenv("DATABASE_URL"))
if err != nil { log.Fatal(err) }
defer db.Close()

// Connection pool settings — crucial for production
db.SetMaxOpenConns(25)                  // max simultaneous connections
db.SetMaxIdleConns(10)                  // max idle connections in pool
db.SetConnMaxLifetime(5 * time.Minute)  // recycle connections
db.SetConnMaxIdleTime(1 * time.Minute)  // close idle connections

// Always verify connectivity at startup
if err = db.PingContext(ctx); err != nil { log.Fatal(err) }

// Query — always use context, always use parameterized queries
rows, err := db.QueryContext(ctx,
    "SELECT id, name, email FROM users WHERE active = $1 AND age > $2",
    true, 18)
if err != nil { return err }
defer rows.Close()

for rows.Next() {
    var u User
    if err := rows.Scan(&u.ID, &u.Name, &u.Email); err != nil { return err }
    users = append(users, u)
}
if err = rows.Err(); err != nil { return err }  // check iteration error

// Single row
var u User
err = db.QueryRowContext(ctx, "SELECT id, name FROM users WHERE id = $1", id).
    Scan(&u.ID, &u.Name)
if errors.Is(err, sql.ErrNoRows) { return ErrNotFound }

// Exec — INSERT/UPDATE/DELETE
result, err := db.ExecContext(ctx,
    "INSERT INTO users(name, email) VALUES($1, $2)", name, email)
lastID, _ := result.LastInsertId()    // not supported by all drivers
affected, _ := result.RowsAffected()

// Transaction
tx, err := db.BeginTx(ctx, &sql.TxOptions{Isolation: sql.LevelReadCommitted})
if err != nil { return err }
defer tx.Rollback()  // no-op if already committed
_, err = tx.ExecContext(ctx, "UPDATE accounts SET balance = balance - $1 WHERE id = $2", amount, fromID)
if err != nil { return err }
_, err = tx.ExecContext(ctx, "UPDATE accounts SET balance = balance + $1 WHERE id = $2", amount, toID)
if err != nil { return err }
return tx.Commit()

// Read entire file — small files only
data, err := os.ReadFile("config.json")

// Write entire file atomically (write to temp, rename)
tmp, err := os.CreateTemp("", "config-*.json")
if err != nil { return err }
if _, err = tmp.Write(data); err != nil {
    os.Remove(tmp.Name()); return err
}
tmp.Close()
os.Rename(tmp.Name(), "config.json")  // atomic on most OS/filesystems

// Stream large files — line by line
f, err := os.Open("large.log")
if err != nil { return err }
defer f.Close()

scanner := bufio.NewScanner(f)
scanner.Buffer(make([]byte, 1024*1024), 1024*1024)  // 1MB max line length
for scanner.Scan() {
    line := scanner.Text()
    process(line)
}
if err = scanner.Err(); err != nil { return err }

// Write with buffered writer — fewer syscalls
f, _ := os.Create("output.txt")
defer f.Close()
bw := bufio.NewWriter(f)
for _, record := range records {
    fmt.Fprintln(bw, record)
}
bw.Flush()  // ALWAYS flush buffered writer before close!

// Walk directory tree
err = filepath.WalkDir(".", func(path string, d fs.DirEntry, err error) error {
    if err != nil { return err }
    if d.IsDir() { return nil }
    if filepath.Ext(path) == ".go" { process(path) }
    return nil
})

// embed files in binary (Go 1.16+)
import "embed"
//go:embed templates/*.html
var templates embed.FS

tmpl, err := templates.ReadFile("templates/index.html")

// Working with time
now := time.Now()                        // local time
utc := time.Now().UTC()                  // always prefer UTC in backend code
t := time.Date(2024, time.January, 15, 12, 0, 0, 0, time.UTC)

// Duration arithmetic
d := 5*time.Second + 30*time.Millisecond
time.Sleep(100 * time.Millisecond)

// Time arithmetic
tomorrow := now.Add(24 * time.Hour)
diff := tomorrow.Sub(now)               // time.Duration
since := time.Since(start)             // time.Since(t) = now.Sub(t)
until := time.Until(deadline)

// Parsing and formatting — Go uses reference time: Mon Jan 2 15:04:05 MST 2006
t, err := time.Parse("2006-01-02", "2024-01-15")
t, err = time.Parse(time.RFC3339, "2024-01-15T12:00:00Z")
formatted := t.Format("2006-01-02 15:04:05")
formatted = t.Format(time.RFC3339)

// Ticker and Timer
ticker := time.NewTicker(5 * time.Second)
defer ticker.Stop()  // ALWAYS stop to avoid goroutine leak
for {
    select {
    case t := <-ticker.C: doWork(t)
    case <-ctx.Done(): return
    }
}

timer := time.NewTimer(10 * time.Second)
defer timer.Stop()  // stop if not fired yet
select {
case <-timer.C: fmt.Println("timeout")
case <-done: timer.Stop()
}

Common bugs:

• time.After leaks: In a loop, time.After creates a new timer each iteration that isn't GC'd until it fires. Use time.NewTimer + Reset instead.
• Comparing time.Time with ==: Use t1.Equal(t2) — the == operator compares the monotonic clock and timezone as well, which can surprise you when parsing from JSON.
• Not using UTC: Store and compare times in UTC. Display in local time only at presentation layer.
• Duration overflow: Large durations stored as int64 nanoseconds overflow around 290 years — not a practical issue but good to know.

// JSON streaming — for large payloads, avoid loading all into memory
func handleLargeJSON(w http.ResponseWriter, r *http.Request) {
    dec := json.NewDecoder(r.Body)
    dec.DisallowUnknownFields()  // strict mode — error on unexpected fields
    var req []Item
    if err := dec.Decode(&req); err != nil {
        http.Error(w, err.Error(), 400); return
    }
    // Stream response
    enc := json.NewEncoder(w)
    w.Header().Set("Content-Type", "application/json")
    for _, result := range process(req) {
        enc.Encode(result)  // writes immediately, no buffering
    }
}

// JSON with raw messages — delay decoding
type Event struct {
    Type    string          `json:"type"`
    Payload json.RawMessage `json:"payload"`  // keep as raw JSON
}
var e Event
json.Unmarshal(data, &e)
switch e.Type {
case "order": var o Order; json.Unmarshal(e.Payload, &o)
case "user":  var u User; json.Unmarshal(e.Payload, &u)
}

// CSV — reading
f, _ := os.Open("data.csv")
defer f.Close()
r := csv.NewReader(f)
r.FieldsPerRecord = -1  // variable fields per row
records, err := r.ReadAll()
// Or row by row:
for { record, err := r.Read(); if err == io.EOF { break } }

// CSV — writing
w := csv.NewWriter(os.Stdout)
w.Write([]string{"Name", "Age", "Email"})
for _, u := range users { w.Write([]string{u.Name, strconv.Itoa(u.Age), u.Email}) }
w.Flush()
if err := w.Error(); err != nil { return err }

// Protocol Buffers (more common than XML in Go microservices)
// import "google.golang.org/protobuf/proto"
// data, err := proto.Marshal(msg)
// err = proto.Unmarshal(data, &msg)

// html/template — auto-escapes to prevent XSS (ALWAYS use for HTML)
import "html/template"

const tmplText = `



  
Hello, {{.Name}}!
  {{range .Items}}
    
{{.Title}} - {{.Price | printf "%.2f"}}
  {{end}}
  {{if .IsAdmin}}Admin{{end}}

`

// Parse once at startup, execute many times
tmpl := template.Must(template.New("page").Funcs(template.FuncMap{
    "upper": strings.ToUpper,
}).Parse(tmplText))

data := struct {
    Name    string
    Items   []Item
    IsAdmin bool
}{"Alice", items, true}

if err := tmpl.Execute(w, data); err != nil {
    log.Printf("template error: %v", err)
}

// Load from files
tmpl, err := template.ParseGlob("templates/*.html")
tmpl, err  = template.ParseFS(embeddedFS, "templates/*.html")  // embedded

// text/template — for non-HTML output (emails, config files, code gen)
// Does NOT escape — use only when output is not HTML
import "text/template"
tmpl := template.Must(template.New("email").Parse(`
Dear {{.Name}},
Your order {{.OrderID}} has been {{.Status}}.
`))

// template.HTML — mark trusted HTML to skip escaping
func safeHTML(s string) template.HTML { return template.HTML(s) }
// Use VERY carefully — only for content you've already sanitized

Key security rules: Always use html/template for HTML output — never text/template. Never use template.HTML(userInput). Parse templates at startup with template.Must to fail fast on syntax errors. Cache compiled templates — parsing is expensive.

// Table-driven tests — idiomatic Go testing
func TestDivide(t *testing.T) {
    tests := []struct {
        name    string
        a, b    float64
        want    float64
        wantErr bool
    }{
        {"normal division", 10, 2, 5, false},
        {"divide by zero", 10, 0, 0, true},
        {"negative result", -10, 2, -5, false},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {  // subtest — runs independently
            got, err := divide(tt.a, tt.b)
            if (err != nil) != tt.wantErr {
                t.Errorf("divide() error = %v, wantErr %v", err, tt.wantErr)
                return
            }
            if got != tt.want {
                t.Errorf("divide() = %v, want %v", got, tt.want)
            }
        })
    }
}

// Run specific subtest: go test -run TestDivide/divide_by_zero

// Test helpers — call t.Helper() so failures report the caller's line
func assertEqual(t *testing.T, got, want interface{}) {
    t.Helper()
    if got != want {
        t.Errorf("got %v, want %v", got, want)
    }
}

// t.Cleanup — runs after test (and subtests) complete
func TestWithCleanup(t *testing.T) {
    db := setupTestDB(t)
    t.Cleanup(func() { db.Close() })
    // test body...
}

// Parallel tests — run tests in parallel
func TestParallel(t *testing.T) {
    t.Parallel()  // this test can run in parallel with others
    // ...
}

// Short mode — skip slow tests
func TestSlow(t *testing.T) {
    if testing.Short() { t.Skip("skipping in short mode") }
    // ...
}
// go test -short

// Benchmark function — must start with Benchmark, take *testing.B
func BenchmarkJoin(b *testing.B) {
    words := []string{"hello", "world", "foo", "bar"}
    b.ResetTimer()  // exclude setup from measurement
    for i := 0; i < b.N; i++ {  // b.N adjusted by framework
        strings.Join(words, ",")
    }
}

// Benchmark with allocation tracking
func BenchmarkAlloc(b *testing.B) {
    b.ReportAllocs()  // report allocs/op and B/op
    for i := 0; i < b.N; i++ {
        _ = fmt.Sprintf("hello %d", i)
    }
}

// Sub-benchmarks — compare sizes, strategies
func BenchmarkSort(b *testing.B) {
    for _, size := range []int{100, 1000, 10000} {
        b.Run(fmt.Sprintf("size=%d", size), func(b *testing.B) {
            data := makeRandomSlice(size)
            b.ResetTimer()
            for i := 0; i < b.N; i++ {
                b.StopTimer(); tmp := append([]int{}, data...); b.StartTimer()
                sort.Ints(tmp)
            }
        })
    }
}

// Run benchmarks:
go test -bench=. -benchmem -benchtime=5s ./...
go test -bench=BenchmarkJoin -count=5  // run 5 times for stability

// Output:
// BenchmarkJoin-8   5000000   234 ns/op   48 B/op   2 allocs/op
// name       CPUs   runs    time/op  mem/op  allocs/op

// Compare with benchstat:
go test -bench=. -count=5 > old.txt
# make change
go test -bench=. -count=5 > new.txt
benchstat old.txt new.txt

Rules: Use b.N always as the loop count — the framework adjusts it. Call b.ResetTimer() after expensive setup. Use b.StopTimer()/b.StartTimer() around per-iteration setup. Prevent compiler from optimizing away results by assigning to a package-level variable: var result = expensive().

// Pattern 1: Interface + manual mock (idiomatic Go)
type UserStore interface {
    GetUser(ctx context.Context, id int64) (*User, error)
    CreateUser(ctx context.Context, u *User) error
}

type MockUserStore struct {
    GetUserFn    func(ctx context.Context, id int64) (*User, error)
    CreateUserFn func(ctx context.Context, u *User) error
    Calls        []string
}

func (m *MockUserStore) GetUser(ctx context.Context, id int64) (*User, error) {
    m.Calls = append(m.Calls, "GetUser")
    return m.GetUserFn(ctx, id)
}

func TestHandler(t *testing.T) {
    mock := &MockUserStore{
        GetUserFn: func(ctx context.Context, id int64) (*User, error) {
            return &User{ID: id, Name: "Alice"}, nil
        },
    }
    h := NewHandler(mock)
    // test h...
    if len(mock.Calls) != 1 || mock.Calls[0] != "GetUser" {
        t.Error("expected GetUser to be called once")
    }
}

// Pattern 2: testify/mock (most popular mock library)
import "github.com/stretchr/testify/mock"

type MockStore struct { mock.Mock }

func (m *MockStore) GetUser(ctx context.Context, id int64) (*User, error) {
    args := m.Called(ctx, id)
    return args.Get(0).(*User), args.Error(1)
}

func TestWithTestify(t *testing.T) {
    ms := new(MockStore)
    ms.On("GetUser", mock.Anything, int64(42)).Return(&User{ID: 42}, nil)
    // use ms...
    ms.AssertExpectations(t)
}

// Pattern 3: uber-go/mock (mockgen code generation)
// go install go.uber.org/mock/mockgen@latest
// mockgen -source=store.go -destination=mock_store.go
// Generated mock has full control over call expectations

import (
    "github.com/testcontainers/testcontainers-go"
    "github.com/testcontainers/testcontainers-go/modules/postgres"
)

func TestWithRealPostgres(t *testing.T) {
    ctx := context.Background()

    // Start a real Postgres container
    pgContainer, err := postgres.RunContainer(ctx,
        testcontainers.WithImage("postgres:16"),
        postgres.WithDatabase("testdb"),
        postgres.WithUsername("test"),
        postgres.WithPassword("test"),
        testcontainers.WithWaitStrategy(
            wait.ForLog("database system is ready to accept connections").
                WithOccurrence(2).WithStartupTimeout(30*time.Second),
        ),
    )
    if err != nil { t.Fatal(err) }
    t.Cleanup(func() { pgContainer.Terminate(ctx) })

    connStr, _ := pgContainer.ConnectionString(ctx, "sslmode=disable")
    db, err := sql.Open("postgres", connStr)
    if err != nil { t.Fatal(err) }
    defer db.Close()

    // Run migrations
    runMigrations(t, db)

    // Test with real DB
    repo := NewUserRepository(db)
    user, err := repo.Create(ctx, &User{Name: "Alice", Email: "a@b.com"})
    if err != nil { t.Fatal(err) }
    if user.ID == 0 { t.Error("expected non-zero ID") }
}

// httptest — test HTTP handlers without starting a real server
func TestHTTPHandler(t *testing.T) {
    handler := NewHandler(store)
    req := httptest.NewRequest(http.MethodGet, "/users/1", nil)
    w := httptest.NewRecorder()
    handler.ServeHTTP(w, req)
    resp := w.Result()
    if resp.StatusCode != http.StatusOK {
        t.Errorf("want 200, got %d", resp.StatusCode)
    }
    var user User
    json.NewDecoder(resp.Body).Decode(&user)
    if user.ID != 1 { t.Errorf("want ID 1, got %d", user.ID) }
}

Go 1.18 introduced native fuzzing — the testing framework generates random inputs to find crashes, panics, and unexpected behaviors in your code.

// Fuzz test — file: parser_test.go
func FuzzParseURL(f *testing.F) {
    // Seed corpus — known interesting inputs
    f.Add("https://example.com/path?q=1")
    f.Add("http://user:pass@host:8080/path")
    f.Add("")  // edge case: empty string
    f.Add("not-a-url")

    f.Fuzz(func(t *testing.T, input string) {
        // Must not panic for any input
        result, err := ParseURL(input)
        if err != nil { return }  // errors are OK

        // Properties that must always hold:
        if result != nil {
            if result.Host == "" && result.Path == "" {
                t.Errorf("parsed URL with empty host and path: %q", input)
            }
        }

        // Round-trip property: parse then serialize = original (if valid)
        if err == nil {
            serialized := result.String()
            reparsed, err2 := ParseURL(serialized)
            if err2 != nil {
                t.Errorf("re-parse failed for %q: %v", serialized, err2)
            }
            _ = reparsed
        }
    })
}

// Run fuzzer:
go test -fuzz=FuzzParseURL -fuzztime=60s ./...

// Run with existing corpus only (like a regular test):
go test -run=FuzzParseURL ./...

// Failing inputs are saved in testdata/fuzz/FuzzParseURL/
// Committed to repo — replayed in future test runs

When to fuzz: Parsers (JSON, XML, binary formats, URLs); cryptographic functions; any code handling untrusted input; compression/decompression; data structure operations. Fuzzing has found real bugs in Go's standard library.

// Build tags — control which files are compiled
// At the top of a file, before package declaration

//go:build integration

package store_test

// Run only unit tests (files without 'integration' tag):
go test ./...

// Run integration tests:
go test -tags=integration ./...

// Multiple tags:
//go:build integration && !ci  // integration AND not ci

// Common patterns:
//go:build unit         // explicit unit tag
//go:build integration  // requires external services
//go:build e2e          // full end-to-end tests
//go:build !race        // skip when -race is enabled (too slow)

// Environment variable approach — simpler for many teams
func TestIntegration(t *testing.T) {
    if os.Getenv("INTEGRATION_TESTS") == "" {
        t.Skip("set INTEGRATION_TESTS=1 to run")
    }
    // ... integration test
}

// TestMain — global setup and teardown for a package's tests
func TestMain(m *testing.M) {
    // setup
    db = setupTestDatabase()
    loadFixtures(db)

    code := m.Run()  // run all tests in package

    // teardown
    db.Close()
    os.Exit(code)
}

// Recommended test structure
// myapp/
// ├── store/
// │   ├── store.go
// │   ├── store_test.go          // unit tests, no build tag
// │   └── store_integration_test.go  // //go:build integration
// └── handler/
//     ├── handler.go
//     └── handler_test.go

import (
    "github.com/stretchr/testify/assert"   // non-fatal — continues test
    "github.com/stretchr/testify/require"  // fatal — stops test immediately
    "github.com/stretchr/testify/suite"
)

func TestUser(t *testing.T) {
    user, err := createUser("Alice", "alice@example.com")

    require.NoError(t, err)         // stop if error (can't continue)
    require.NotNil(t, user)         // stop if nil

    assert.Equal(t, "Alice", user.Name)
    assert.Equal(t, "alice@example.com", user.Email)
    assert.NotZero(t, user.ID)
    assert.WithinDuration(t, time.Now(), user.CreatedAt, time.Second)

    // Slice assertions
    assert.Contains(t, user.Roles, "viewer")
    assert.Len(t, user.Roles, 1)
    assert.ElementsMatch(t, []string{"a","b"}, []string{"b","a"})  // order-independent

    // Error type checking
    _, err = createUser("", "")
    require.Error(t, err)
    assert.ErrorIs(t, err, ErrValidation)
    var ve *ValidationError
    assert.ErrorAs(t, err, &ve)
    assert.Equal(t, "name", ve.Field)
}

// Test suite — shared setup/teardown
type UserSuite struct {
    suite.Suite
    db   *sql.DB
    repo *UserRepository
}

func (s *UserSuite) SetupTest() {  // runs before EACH test
    s.db = openTestDB()
    s.repo = NewUserRepository(s.db)
}

func (s *UserSuite) TearDownTest() { s.db.Close() }

func (s *UserSuite) TestCreate() {
    u, err := s.repo.Create(context.Background(), &User{Name: "Alice"})
    s.Require().NoError(err)
    s.Equal("Alice", u.Name)
}

func TestUserSuite(t *testing.T) { suite.Run(t, new(UserSuite)) }

// 1. Define proto (user.proto)
syntax = "proto3";
option go_package = "github.com/myorg/myapp/gen/user/v1;userv1";

service UserService {
    rpc GetUser(GetUserRequest) returns (GetUserResponse);
    rpc ListUsers(ListUsersRequest) returns (stream User);     // server streaming
    rpc CreateUsers(stream CreateUserRequest) returns (CreateUsersResponse); // client streaming
    rpc SyncUsers(stream User) returns (stream User);          // bidirectional
}
message GetUserRequest  { int64 id = 1; }
message GetUserResponse { User user = 1; }
message User { int64 id = 1; string name = 2; string email = 3; }

// 2. Generate code
// buf generate  (preferred, uses buf.gen.yaml)
// OR: protoc --go_out=. --go-grpc_out=. user.proto

// 3. Implement server
type userServer struct {
    userv1.UnimplementedUserServiceServer  // forward-compatible embed
    store UserStore
}

func (s *userServer) GetUser(ctx context.Context, req *userv1.GetUserRequest) (*userv1.GetUserResponse, error) {
    user, err := s.store.Get(ctx, req.Id)
    if err != nil {
        return nil, status.Errorf(codes.NotFound, "user %d not found: %v", req.Id, err)
    }
    return &userv1.GetUserResponse{User: userToProto(user)}, nil
}

func (s *userServer) ListUsers(req *userv1.ListUsersRequest, stream userv1.UserService_ListUsersServer) error {
    users, err := s.store.List(stream.Context())
    if err != nil { return status.Error(codes.Internal, err.Error()) }
    for _, u := range users {
        if err := stream.Send(userToProto(u)); err != nil { return err }
    }
    return nil
}

// 4. Start server with middleware
func main() {
    lis, _ := net.Listen("tcp", ":50051")
    srv := grpc.NewServer(
        grpc.ChainUnaryInterceptor(
            otelgrpc.UnaryServerInterceptor(),  // tracing
            grpc_recovery.UnaryServerInterceptor(),  // recover panics
            grpc_auth.UnaryServerInterceptor(authenticate),
        ),
        grpc.ChainStreamInterceptor(
            otelgrpc.StreamServerInterceptor(),
        ),
    )
    userv1.RegisterUserServiceServer(srv, &userServer{store: store})
    reflection.Register(srv)  // enables grpcurl and API explorers
    log.Fatal(srv.Serve(lis))
}

// Unary interceptor — for request/response RPCs
func loggingInterceptor(
    ctx context.Context,
    req interface{},
    info *grpc.UnaryServerInfo,
    handler grpc.UnaryHandler,
) (interface{}, error) {
    start := time.Now()
    resp, err := handler(ctx, req)  // call the actual handler
    slog.Info("rpc call",
        slog.String("method", info.FullMethod),
        slog.Duration("latency", time.Since(start)),
        slog.Any("error", err),
    )
    return resp, err
}

// Auth interceptor
func authInterceptor(ctx context.Context, req interface{},
    info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) {

    md, ok := metadata.FromIncomingContext(ctx)
    if !ok { return nil, status.Error(codes.Unauthenticated, "missing metadata") }

    tokens := md.Get("authorization")
    if len(tokens) == 0 { return nil, status.Error(codes.Unauthenticated, "missing token") }

    claims, err := validateJWT(strings.TrimPrefix(tokens[0], "Bearer "))
    if err != nil { return nil, status.Error(codes.Unauthenticated, err.Error()) }

    ctx = context.WithValue(ctx, claimsKey{}, claims)
    return handler(ctx, req)
}

// Stream interceptor — wraps streaming RPCs
func loggingStreamInterceptor(srv interface{}, ss grpc.ServerStream,
    info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
    start := time.Now()
    err := handler(srv, ss)
    slog.Info("stream rpc", slog.String("method", info.FullMethod),
        slog.Duration("duration", time.Since(start)), slog.Any("error", err))
    return err
}

// Client-side interceptor
conn, err := grpc.NewClient("localhost:50051",
    grpc.WithTransportCredentials(insecure.NewCredentials()),
    grpc.WithChainUnaryInterceptor(
        func(ctx context.Context, method string, req, reply interface{},
            cc *grpc.ClientConn, invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error {
            ctx = metadata.AppendToOutgoingContext(ctx, "authorization", "Bearer "+token)
            return invoker(ctx, method, req, reply, cc, opts...)
        },
    ),
)

// Service discovery via Kubernetes DNS — simplest approach
// Services get stable DNS: service-name.namespace.svc.cluster.local
conn, _ := grpc.NewClient("user-service.default.svc.cluster.local:50051",
    grpc.WithDefaultServiceConfig(`{"loadBalancingConfig": [{"round_robin":{}}]}`),
)

// go-kit/kit microservices — service mesh in code
// github.com/sony/gobreaker — circuit breaker
import "github.com/sony/gobreaker"

cb := gobreaker.NewCircuitBreaker(gobreaker.Settings{
    Name:        "payment-service",
    MaxRequests: 3,               // max requests in half-open
    Interval:    10 * time.Second, // reset counts interval
    Timeout:     30 * time.Second, // open→half-open after this
    ReadyToTrip: func(counts gobreaker.Counts) bool {
        return counts.ConsecutiveFailures > 5
    },
})

result, err := cb.Execute(func() (interface{}, error) {
    return paymentClient.Charge(ctx, req)
})
if errors.Is(err, gobreaker.ErrOpenState) {
    return nil, status.Error(codes.Unavailable, "payment service unavailable")
}

// Retry with exponential backoff
import "github.com/cenkalti/backoff/v4"

operation := func() error {
    _, err := userClient.GetUser(ctx, req)
    return err
}
b := backoff.NewExponentialBackOff()
b.MaxElapsedTime = 30 * time.Second
if err := backoff.RetryNotify(operation, b, func(err error, d time.Duration) {
    slog.Warn("retrying", slog.Any("error", err), slog.Duration("backoff", d))
}); err != nil { return err }

import (
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/attribute"
    "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc"
    "go.opentelemetry.io/otel/sdk/trace"
)

// Setup — called at startup
func initTracing(ctx context.Context) (func(), error) {
    exp, err := otlptracegrpc.New(ctx,
        otlptracegrpc.WithEndpoint("otel-collector:4317"),
        otlptracegrpc.WithInsecure(),
    )
    if err != nil { return nil, err }

    tp := trace.NewTracerProvider(
        trace.WithBatcher(exp),
        trace.WithSampler(trace.TraceIDRatioBased(0.1)),  // 10% sampling
        trace.WithResource(resource.NewWithAttributes(
            semconv.SchemaURL,
            semconv.ServiceName("user-service"),
            semconv.ServiceVersion("1.2.3"),
        )),
    )
    otel.SetTracerProvider(tp)
    otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(
        propagation.TraceContext{}, propagation.Baggage{},
    ))
    return func() { tp.Shutdown(ctx) }, nil
}

// Instrument your code
var tracer = otel.Tracer("user-service")

func (s *userServer) GetUser(ctx context.Context, req *userv1.GetUserRequest) (*userv1.GetUserResponse, error) {
    ctx, span := tracer.Start(ctx, "GetUser")
    defer span.End()

    span.SetAttributes(attribute.Int64("user.id", req.Id))

    user, err := s.store.Get(ctx, req.Id)
    if err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, err.Error())
        return nil, status.Error(grpccodes.NotFound, err.Error())
    }
    span.SetAttributes(attribute.String("user.name", user.Name))
    return &userv1.GetUserResponse{User: userToProto(user)}, nil
}

// Auto-instrument HTTP and gRPC with middleware
mux := http.NewServeMux()
handler := otelhttp.NewHandler(mux, "http-server")  // wraps all routes

grpc.NewServer(grpc.ChainUnaryInterceptor(otelgrpc.UnaryServerInterceptor()))

// Kafka with franz-go (recommended modern Go Kafka client)
import "github.com/twmb/franz-go/pkg/kgo"

// Producer
client, _ := kgo.NewClient(
    kgo.SeedBrokers("kafka:9092"),
    kgo.DefaultProduceTopic("orders"),
)
defer client.Close()

event := OrderCreatedEvent{OrderID: "ord-123", UserID: 42, Total: 99.99}
data, _ := json.Marshal(event)
record := &kgo.Record{Key: []byte(event.OrderID), Value: data}
if err := client.ProduceSync(ctx, record).FirstErr(); err != nil {
    return fmt.Errorf("produce: %w", err)
}

// Consumer — with consumer group
client, _ := kgo.NewClient(
    kgo.SeedBrokers("kafka:9092"),
    kgo.ConsumeTopics("orders"),
    kgo.ConsumerGroup("order-processor"),
)
for {
    fetches := client.PollFetches(ctx)
    if fetches.IsClientClosed() { return }
    fetches.EachRecord(func(r *kgo.Record) {
        var event OrderCreatedEvent
        json.Unmarshal(r.Value, &event)
        if err := processOrder(ctx, event); err != nil {
            slog.Error("processing failed", slog.Any("error", err))
            // decide: retry, DLQ, or continue
        }
    })
}

// NATS — lightweight, fast pub/sub
import "github.com/nats-io/nats.go"

nc, _ := nats.Connect("nats://localhost:4222")
defer nc.Drain()

// Publish
nc.Publish("orders.created", data)

// Subscribe with JetStream (persistent, at-least-once)
js, _ := nc.JetStream()
js.Subscribe("orders.*", func(msg *nats.Msg) {
    var event OrderEvent
    json.Unmarshal(msg.Data, &event)
    if err := handle(ctx, event); err != nil {
        msg.Nak()  // negative ack — will be redelivered
        return
    }
    msg.Ack()  // acknowledge — won't be redelivered
}, nats.Durable("order-processor"), nats.AckExplicit())

// Recommended structure for a Go microservice:
// user-service/
// ├── cmd/
// │   └── server/
// │       └── main.go           ← entry point, wires everything
// ├── internal/                 ← not importable by other modules
// │   ├── domain/               ← core business types (no external deps)
// │   │   ├── user.go
// │   │   └── user_repository.go  ← interface
// │   ├── service/              ← business logic
// │   │   └── user_service.go
// │   ├── store/                ← DB implementation
// │   │   └── postgres_user_store.go
// │   └── transport/            ← HTTP/gRPC handlers
// │       ├── grpc/
// │       │   └── user_server.go
// │       └── http/
// │           └── user_handler.go
// ├── gen/                      ← generated protobuf code
// │   └── user/v1/
// ├── migrations/               ← SQL migration files
// ├── config/
// │   └── config.go             ← typed config from env vars
// ├── docker-compose.yml
// ├── Dockerfile
// ├── Makefile
// └── go.mod

// Key principles:
// - cmd/ — multiple binaries (server, cli, migration tool)
// - internal/ — prevents accidental external import
// - Domain types in domain/ have no framework dependencies
// - Repository interface defined in domain/, implemented in store/
// - Dependency injection wired in main.go (composition root)

// config/config.go — typed config
type Config struct {
    DatabaseURL    string        `env:"DATABASE_URL,required"`
    Port           int           `env:"PORT" envDefault:"8080"`
    GRPCPort       int           `env:"GRPC_PORT" envDefault:"50051"`
    JWTSecret      string        `env:"JWT_SECRET,required"`
    MaxDBConns     int           `env:"MAX_DB_CONNS" envDefault:"25"`
    OTLPEndpoint   string        `env:"OTLP_ENDPOINT"`
    LogLevel       slog.Level    `env:"LOG_LEVEL" envDefault:"info"`
}
// Using github.com/caarlos0/env for parsing

// Health check endpoints — Kubernetes liveness and readiness
type HealthChecker struct {
    db    *sql.DB
    ready atomic.Bool  // set to true after warmup
}

func (h *HealthChecker) RegisterRoutes(mux *http.ServeMux) {
    // Liveness — "am I alive?" — restart if fails
    mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {
        w.WriteHeader(http.StatusOK)
        w.Write([]byte("ok"))
    })
    // Readiness — "am I ready to serve traffic?" — remove from LB if fails
    mux.HandleFunc("GET /readyz", func(w http.ResponseWriter, r *http.Request) {
        if !h.ready.Load() {
            http.Error(w, "not ready", http.StatusServiceUnavailable)
            return
        }
        ctx, cancel := context.WithTimeout(r.Context(), 2*time.Second)
        defer cancel()
        if err := h.db.PingContext(ctx); err != nil {
            http.Error(w, "db unhealthy: "+err.Error(), http.StatusServiceUnavailable)
            return
        }
        w.WriteHeader(http.StatusOK)
        json.NewEncoder(w).Encode(map[string]string{"status": "ok"})
    })
}

// Graceful shutdown — stop accepting new, finish in-flight
func run(ctx context.Context) error {
    srv := &http.Server{Addr: ":8080", Handler: handler}

    // Start server in goroutine
    go func() {
        if err := srv.ListenAndServe(); !errors.Is(err, http.ErrServerClosed) {
            slog.Error("server error", slog.Any("error", err))
        }
    }()

    // Wait for signal
    quit := make(chan os.Signal, 1)
    signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
    select {
    case <-quit:
        slog.Info("shutdown signal received")
    case <-ctx.Done():
    }

    // Mark not ready — stop receiving new traffic
    healthChecker.ready.Store(false)

    // Give LB time to remove us from rotation
    time.Sleep(5 * time.Second)

    // Shutdown with timeout — wait for in-flight requests
    shutCtx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
    defer cancel()
    return srv.Shutdown(shutCtx)
}

Go is uniquely suited for containers: static binaries need no runtime, resulting in tiny images and fast cold starts.

# Multi-stage Dockerfile — production-ready
FROM golang:1.22-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download                # cache dependencies
COPY . .
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build \
    -ldflags="-w -s -X main.version=$(git rev-parse --short HEAD)" \
    -o /app/server ./cmd/server   # -w -s strips debug info (~30% smaller)

# Minimal runtime image
FROM gcr.io/distroless/static-debian12  # ~2MB, no shell, no package manager
# OR: FROM scratch  — truly empty, for fully static binaries
COPY --from=builder /app/server /server
COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/  # for HTTPS
USER nonroot:nonroot  # don't run as root
EXPOSE 8080
ENTRYPOINT ["/server"]

Why Go is ideal for containers:

• No runtime dependency: Just the binary — no JVM, no interpreter, no shared libraries (with CGO_ENABLED=0)
• Tiny images: distroless + Go binary can be <20MB. JVM apps are typically 200MB+
• Fast startup: Milliseconds vs seconds. Critical for horizontal scaling and serverless
• Low memory baseline: Go services often run comfortably under 50MB RSS
• Cross-compilation: Build for any target OS/arch on your dev machine: GOOS=linux GOARCH=arm64

// Build flags for size reduction
CGO_ENABLED=0            // disable CGo — fully static binary
-ldflags="-w -s"         // strip DWARF debug info and symbol table
-trimpath                // remove build paths from binary (reproducibility)
// Typical Go binary: 10-20MB unstripped, 5-10MB stripped, 2-5MB upx-compressed

// 1. Pre-size slices and maps — avoid repeated resizing
users := make([]User, 0, expectedCount)  // len=0, cap=expectedCount
index := make(map[string]User, expectedCount)

// 2. Reuse allocations with sync.Pool (covered earlier)

// 3. Avoid interface conversions in hot paths
// Interface boxing allocates — use concrete types in tight loops
func processItems(items []interface{}) { ... }  // allocates for each item
func processUsers(users []User) { ... }          // no boxing

// 4. String builder — avoid repeated concatenation
var sb strings.Builder
sb.Grow(estimatedSize)  // pre-allocate
for _, s := range parts { sb.WriteString(s) }
result := sb.String()

// 5. Avoid defer in hot loops — it has per-call overhead (pre-1.14)
// Modern Go: open-coded defers mostly eliminate this, but still avoid in critical loops

// 6. Use buffered I/O
bw := bufio.NewWriterSize(w, 64*1024)  // 64KB buffer
// ... many small writes ...
bw.Flush()  // single syscall for all buffered data

// 7. Struct field ordering — minimize padding
// Bad: 1+8+1+8 = 26 bytes → padded to 32
type Bad struct { a byte; b int64; c byte; d int64 }
// Good: 8+8+1+1 = 18 bytes → padded to 24
type Good struct { b int64; d int64; a byte; c byte }
// Use fieldalignment linter: go vet -fieldalignment

// 8. Copy vs pointer for small structs
// Passing a 3-field struct by value is often faster than by pointer
// (avoids pointer indirection and potential GC scan)
func process(u User) { ... }   // copy — may be faster for small structs
func process(u *User) { ... }  // pointer — avoids copy for large structs

// 9. Reduce allocations with value types
type Point struct { X, Y float64 }  // value type — stack allocated
pts := make([]Point, 1000)          // one allocation for 1000 points
vs.
pts := make([]*Point, 1000)         // 1001 allocations!

// Standard library encoding/json — uses reflection, correct but slow
// For high-throughput services, use faster alternatives:

// 1. github.com/bytedance/sonic — fastest general-purpose JSON for Go
// Drop-in replacement for encoding/json
import "github.com/bytedance/sonic"
data, err := sonic.Marshal(v)
err = sonic.Unmarshal(data, &v)
// 3-5x faster than std, uses JIT compilation on amd64

// 2. github.com/json-iterator/go — another fast drop-in
import jsoniter "github.com/json-iterator/go"
var json = jsoniter.ConfigCompatibleWithStandardLibrary
data, err := json.Marshal(v)

// 3. Code generation — easiest.json, ffjson
// Generate MarshalJSON/UnmarshalJSON that avoid reflection
// go generate runs: ffjson ./...

// 4. Custom MarshalJSON — avoid reflection for hot types
type Order struct {
    ID    int64
    Total float64
    Items []Item
}

func (o Order) MarshalJSON() ([]byte, error) {
    var b bytes.Buffer
    b.Grow(256)
    fmt.Fprintf(&b, `{"id":%d,"total":%.2f,"items":[`, o.ID, o.Total)
    for i, item := range o.Items {
        if i > 0 { b.WriteByte(',') }
        fmt.Fprintf(&b, `{"id":%d,"name":%q}`, item.ID, item.Name)
    }
    b.WriteString("]}")
    return b.Bytes(), nil
}

// 5. Avoid allocation in hot path — reuse encoder
pool := sync.Pool{New: func() any { return json.NewEncoder(nil) }}
enc := pool.Get().(*json.Encoder)
defer pool.Put(enc)
var buf bytes.Buffer
enc.Reset(&buf)
enc.Encode(response)

// Go's compiler auto-vectorizes some loops — check with:
go build -gcflags="-d=ssa/prove/debug=1" ./...

// For manual SIMD, Go supports assembly for hot functions
// Function declared in Go:
// math/add_amd64.go
package math
func addSlices(a, b []float64) []float64

// Implementation in assembly:
// math/add_amd64.s
#include "textflag.h"
TEXT ·addSlices(SB),NOSPLIT,$0
    // ... AVX2 SIMD instructions ...

// More practical: use libraries that wrap SIMD
// gonum.org/v1/gonum — linear algebra, highly optimized
import "gonum.org/v1/gonum/blas/blas64"

// github.com/klauspost/compress — fast compression with CPU-specific paths
// github.com/klauspost/cpuid — detect CPU features

// Practical Go optimizations before SIMD:
// 1. Profile first — find the actual bottleneck
// 2. Reduce allocations (biggest win in most Go programs)
// 3. Improve data locality (structure of arrays vs array of structures)
// 4. Use uint8/uint16 instead of int where values fit — cache efficiency

// unsafe for zero-copy conversions (carefully!)
import "unsafe"

func bytesToString(b []byte) string {
    return *(*string)(unsafe.Pointer(&b))  // zero-copy! read-only
}
func stringToBytes(s string) []byte {
    sh := (*reflect.StringHeader)(unsafe.Pointer(&s))
    return *(*[]byte)(unsafe.Pointer(&reflect.SliceHeader{
        Data: sh.Data, Len: sh.Len, Cap: sh.Len,
    }))
}
// WARNING: The resulting string/slice must not be mutated
// Use only in performance-critical paths where you control the data

// Tuned HTTP server
srv := &http.Server{
    Addr:              ":8080",
    Handler:           mux,
    ReadTimeout:       5 * time.Second,    // time to read request headers
    ReadHeaderTimeout: 2 * time.Second,    // time for just headers (prevents slowloris)
    WriteTimeout:      10 * time.Second,   // time to write response
    IdleTimeout:       120 * time.Second,  // keep-alive timeout
    MaxHeaderBytes:    1 << 20,            // 1MB max header size
}

// For maximum performance, use fasthttp (not standard net/http)
// github.com/valyala/fasthttp — zero-allocation HTTP server
// 5-10x fewer allocations than net/http
import "github.com/valyala/fasthttp"
fasthttp.ListenAndServe(":8080", func(ctx *fasthttp.RequestCtx) {
    ctx.SetContentType("application/json")
    ctx.SetStatusCode(200)
    ctx.WriteString(`{"status":"ok"}`)
})
// Note: incompatible with net/http middleware ecosystem

// Connection pooling for DB and downstream services
// Already covered in database/sql section

// Response caching
var cache = sync.Map{}
mux.HandleFunc("GET /expensive", func(w http.ResponseWriter, r *http.Request) {
    key := r.URL.String()
    if cached, ok := cache.Load(key); ok {
        w.Write(cached.([]byte)); return
    }
    data := computeExpensive()
    cache.Store(key, data)
    w.Write(data)
})

// Avoid per-request allocations
var responsePool = sync.Pool{New: func() any { return make([]byte, 0, 4096) }}
mux.HandleFunc("GET /data", func(w http.ResponseWriter, r *http.Request) {
    buf := responsePool.Get().([]byte)
    defer func() { responsePool.Put(buf[:0]) }()
    buf = buildResponse(buf)
    w.Write(buf)
})

// GOMAXPROCS — match to container CPU limit
// Go 1.21+ automatically reads cgroups to set GOMAXPROCS correctly
// For older versions, use automaxprocs:
import _ "go.uber.org/automaxprocs"  // auto-sets GOMAXPROCS from cgroup limit
// Prevents over-scheduling: a 2-CPU container shouldn't use GOMAXPROCS=32

// GOMEMLIMIT — prevent OOM kills
// Set to ~90% of container memory limit
// In Kubernetes: containerMemoryLimit * 0.9
GOMEMLIMIT=450MiB  # for 500Mi container limit

// Or in code at startup:
import "runtime/debug"
func init() {
    if limit := os.Getenv("GOMEMLIMIT"); limit != "" {
        if bytes, err := humanize.ParseBytes(limit); err == nil {
            debug.SetMemoryLimit(int64(bytes))
        }
    }
}

// GC tuning for latency-sensitive services
GOGC=off       # disable GC, rely on GOMEMLIMIT (Go 1.19+, low-latency)
GOGC=200       # GC less frequently, use more memory

// Kubernetes resource configuration
resources:
  requests:
    cpu: "500m"      # 0.5 CPU
    memory: "256Mi"
  limits:
    cpu: "1000m"     # 1 CPU — GOMAXPROCS will see 1
    memory: "512Mi"  # GOMEMLIMIT should be ~460MiB

// Monitor in production
go func() {
    for range time.Tick(30 * time.Second) {
        var ms runtime.MemStats
        runtime.ReadMemStats(&ms)
        metrics.Gauge("go.heap.alloc", float64(ms.HeapAlloc))
        metrics.Gauge("go.gc.pause_ns", float64(ms.PauseNs[(ms.NumGC+255)%256]))
        metrics.Gauge("go.goroutines", float64(runtime.NumGoroutine()))
    }
}()

// Direct OpenAI API integration
import "github.com/sashabaranov/go-openai"

type LLMClient struct {
    client *openai.Client
    model  string
}

func NewLLMClient(apiKey, model string) *LLMClient {
    return &LLMClient{client: openai.NewClient(apiKey), model: model}
}

// Non-streaming
func (c *LLMClient) Complete(ctx context.Context, prompt string) (string, error) {
    resp, err := c.client.CreateChatCompletion(ctx, openai.ChatCompletionRequest{
        Model: c.model,
        Messages: []openai.ChatCompletionMessage{
            {Role: openai.ChatMessageRoleSystem, Content: "You are a helpful assistant."},
            {Role: openai.ChatMessageRoleUser, Content: prompt},
        },
        MaxTokens:   2048,
        Temperature: 0.7,
    })
    if err != nil { return "", fmt.Errorf("completion: %w", err) }
    return resp.Choices[0].Message.Content, nil
}

// Streaming — returns tokens as they arrive
func (c *LLMClient) Stream(ctx context.Context, prompt string, out io.Writer) error {
    stream, err := c.client.CreateChatCompletionStream(ctx, openai.ChatCompletionRequest{
        Model: c.model,
        Messages: []openai.ChatCompletionMessage{
            {Role: openai.ChatMessageRoleUser, Content: prompt},
        },
        Stream: true,
    })
    if err != nil { return fmt.Errorf("stream: %w", err) }
    defer stream.Close()

    for {
        chunk, err := stream.Recv()
        if errors.Is(err, io.EOF) { return nil }
        if err != nil { return fmt.Errorf("stream recv: %w", err) }
        if len(chunk.Choices) > 0 {
            fmt.Fprint(out, chunk.Choices[0].Delta.Content)
        }
    }
}

// SSE streaming to HTTP client
func (h *Handler) Chat(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "text/event-stream")
    w.Header().Set("Cache-Control", "no-cache")
    w.Header().Set("Connection", "keep-alive")
    flusher := w.(http.Flusher)

    h.llm.Stream(r.Context(), prompt, &sseWriter{w: w, flusher: flusher})
}

import (
    "github.com/sashabaranov/go-openai"
    "github.com/pgvector/pgvector-go"
    "github.com/jackc/pgx/v5"
)

type RAGService struct {
    llm  *openai.Client
    db   *pgx.Conn
}

// Embed text using OpenAI
func (s *RAGService) embed(ctx context.Context, text string) ([]float32, error) {
    resp, err := s.llm.CreateEmbeddings(ctx, openai.EmbeddingRequestStrings{
        Model: openai.AdaEmbeddingV2,
        Input: []string{text},
    })
    if err != nil { return nil, err }
    floats := make([]float32, len(resp.Data[0].Embedding))
    for i, v := range resp.Data[0].Embedding { floats[i] = float32(v) }
    return floats, nil
}

// Ingest document chunks into pgvector
func (s *RAGService) Ingest(ctx context.Context, docID, content string) error {
    chunks := chunkText(content, 500, 50)  // size, overlap
    for i, chunk := range chunks {
        embedding, err := s.embed(ctx, chunk)
        if err != nil { return err }
        _, err = s.db.Exec(ctx,
            `INSERT INTO doc_chunks(doc_id, chunk_index, content, embedding)
             VALUES($1, $2, $3, $4)`,
            docID, i, chunk, pgvector.NewVector(embedding),
        )
        if err != nil { return err }
    }
    return nil
}

// Retrieve most similar chunks
func (s *RAGService) Retrieve(ctx context.Context, query string, topK int) ([]string, error) {
    embedding, err := s.embed(ctx, query)
    if err != nil { return nil, err }
    rows, err := s.db.Query(ctx,
        `SELECT content FROM doc_chunks
         ORDER BY embedding <=> $1  -- cosine distance (pgvector)
         LIMIT $2`,
        pgvector.NewVector(embedding), topK,
    )
    if err != nil { return nil, err }
    defer rows.Close()
    var chunks []string
    for rows.Next() {
        var c string
        rows.Scan(&c)
        chunks = append(chunks, c)
    }
    return chunks, rows.Err()
}

// RAG query
func (s *RAGService) Query(ctx context.Context, question string) (string, error) {
    chunks, err := s.Retrieve(ctx, question, 5)
    if err != nil { return "", err }
    context := strings.Join(chunks, "\n\n---\n\n")
    prompt := fmt.Sprintf("Context:\n%s\n\nQuestion: %s\n\nAnswer:", context, question)
    return s.Complete(ctx, prompt)
}

// Option 1: Ollama HTTP API — run models locally, call via HTTP
// Start: ollama run llama3.2
import "github.com/ollama/ollama/api"

client, _ := api.ClientFromEnvironment()  // OLLAMA_HOST env var

// Chat completion
err := client.Chat(ctx, &api.ChatRequest{
    Model: "llama3.2",
    Messages: []api.Message{
        {Role: "user", Content: "Explain Go interfaces"},
    },
}, func(resp api.ChatResponse) error {
    fmt.Print(resp.Message.Content)  // streaming tokens
    return nil
})

// Generate embeddings
resp, err := client.Embeddings(ctx, &api.EmbeddingRequest{
    Model:  "nomic-embed-text",
    Prompt: "text to embed",
})
embedding := resp.Embedding  // []float64

// Option 2: llama.go — native Go bindings for llama.cpp
// github.com/go-skynet/go-llama.cpp
import llama "github.com/go-skynet/go-llama.cpp"
l, err := llama.New("./models/llama-3.2.gguf",
    llama.SetContext(2048), llama.SetGPULayers(32))
output, err := l.Predict("Explain goroutines:", llama.SetTemperature(0.7))

// Option 3: ONNX Runtime — for smaller models (embedding, classification)
// github.com/yalue/onnxruntime_go
// Great for embedding models: all-MiniLM-L6-v2 (22MB ONNX)
session, err := onnxruntime.NewSession[float32](
    "model.onnx",
    []string{"input_ids", "attention_mask"},
    []string{"last_hidden_state"},
)

// Option 4: OpenAI-compatible local API (llamafile, vLLM, LM Studio)
// They serve the OpenAI API format — just change the base URL
client := openai.NewClientWithConfig(openai.ClientConfig{
    BaseURL: "http://localhost:8080/v1",  // local server
    APIKey:  "not-needed",
})

type Tool struct {
    Name        string
    Description string
    Parameters  map[string]any   // JSON Schema
    Execute     func(ctx context.Context, args map[string]any) (string, error)
}

type Agent struct {
    client *openai.Client
    tools  []Tool
    model  string
}

func (a *Agent) Run(ctx context.Context, userMsg string) (string, error) {
    messages := []openai.ChatCompletionMessage{
        {Role: openai.ChatMessageRoleUser, Content: userMsg},
    }

    // Convert tools to OpenAI format
    var toolDefs []openai.Tool
    for _, t := range a.tools {
        toolDefs = append(toolDefs, openai.Tool{
            Type: openai.ToolTypeFunction,
            Function: &openai.FunctionDefinition{
                Name:        t.Name,
                Description: t.Description,
                Parameters:  t.Parameters,
            },
        })
    }

    // Agent loop
    for range 10 {  // max iterations
        resp, err := a.client.CreateChatCompletion(ctx, openai.ChatCompletionRequest{
            Model: a.model, Messages: messages, Tools: toolDefs,
        })
        if err != nil { return "", err }

        choice := resp.Choices[0]
        messages = append(messages, choice.Message)

        if choice.FinishReason == openai.FinishReasonStop { // done
            return choice.Message.Content, nil
        }

        // Execute tool calls
        for _, tc := range choice.Message.ToolCalls {
            var args map[string]any
            json.Unmarshal([]byte(tc.Function.Arguments), &args)

            tool := a.findTool(tc.Function.Name)
            result, err := tool.Execute(ctx, args)
            if err != nil { result = "error: " + err.Error() }

            messages = append(messages, openai.ChatCompletionMessage{
                Role: openai.ChatMessageRoleTool, ToolCallID: tc.ID, Content: result,
            })
        }
    }
    return "", errors.New("max iterations reached")
}

// Example tools
weatherTool := Tool{
    Name: "get_weather",
    Description: "Get current weather for a city",
    Parameters: map[string]any{"type":"object","properties":map[string]any{
        "city":{"type":"string","description":"City name"},
    },"required":[]string{"city"}},
    Execute: func(ctx context.Context, args map[string]any) (string, error) {
        city := args["city"].(string)
        return fetchWeather(ctx, city)
    },
}

// TorchScript / ONNX model serving via CGo bindings
// github.com/yalue/onnxruntime_go — ONNX Runtime bindings

import ort "github.com/yalue/onnxruntime_go"

type InferenceServer struct {
    session  *ort.Session[float32]
    inputBuf []float32
    mu       sync.Mutex  // protect if sharing session
}

func NewInferenceServer(modelPath string) (*InferenceServer, error) {
    ort.SetSharedLibraryPath("/usr/lib/libonnxruntime.so")
    if err := ort.InitializeEnvironment(); err != nil { return nil, err }

    session, err := ort.NewSession[float32](modelPath,
        []string{"input"}, []string{"output"})
    return &InferenceServer{session: session}, err
}

func (s *InferenceServer) Predict(ctx context.Context, features []float32) ([]float32, error) {
    input, err := ort.NewTensor(ort.NewShape(1, int64(len(features))), features)
    if err != nil { return nil, err }
    defer input.Destroy()

    output, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 10))
    if err != nil { return nil, err }
    defer output.Destroy()

    if err = s.session.Run([]ort.ArbitraryTensor{input}, []ort.ArbitraryTensor{output}); err != nil {
        return nil, err
    }
    return output.GetData(), nil
}

// Request batching — accumulate requests, batch inference
type BatchPredictor struct {
    server    *InferenceServer
    batchSize int
    timeout   time.Duration
    requests  chan batchRequest
}

type batchRequest struct {
    features []float32
    reply    chan batchResult
}

func (bp *BatchPredictor) worker() {
    ticker := time.NewTicker(bp.timeout)
    var batch []batchRequest
    for {
        select {
        case req := <-bp.requests:
            batch = append(batch, req)
            if len(batch) >= bp.batchSize { bp.flush(batch); batch = nil }
        case <-ticker.C:
            if len(batch) > 0 { bp.flush(batch); batch = nil }
        }
    }
}

// Setup pgvector in Postgres
// CREATE EXTENSION vector;
// CREATE TABLE embeddings (
//   id BIGSERIAL PRIMARY KEY,
//   content TEXT NOT NULL,
//   metadata JSONB,
//   embedding vector(1536)  -- OpenAI ada-002 dimensions
// );
// CREATE INDEX ON embeddings USING hnsw (embedding vector_cosine_ops)
//   WITH (m = 16, ef_construction = 64);

import (
    "github.com/jackc/pgx/v5/pgxpool"
    "github.com/pgvector/pgvector-go"
    pgxvector "github.com/pgvector/pgvector-go/pgx"
)

func setupPool(ctx context.Context, dsn string) (*pgxpool.Pool, error) {
    config, err := pgxpool.ParseConfig(dsn)
    if err != nil { return nil, err }
    // Register pgvector type with pgx
    config.AfterConnect = func(ctx context.Context, conn *pgx.Conn) error {
        return pgxvector.RegisterTypes(ctx, conn)
    }
    return pgxpool.NewWithConfig(ctx, config)
}

// Upsert with conflict handling (idempotent ingestion)
func (s *Store) Upsert(ctx context.Context, id, content string, embedding []float32) error {
    _, err := s.pool.Exec(ctx, `
        INSERT INTO embeddings(id, content, embedding)
        VALUES($1, $2, $3)
        ON CONFLICT(id) DO UPDATE
            SET content = excluded.content, embedding = excluded.embedding`,
        id, content, pgvector.NewVector(embedding),
    )
    return err
}

// Hybrid search — combine vector similarity + full-text search
func (s *Store) HybridSearch(ctx context.Context, query string, embedding []float32, topK int) ([]Result, error) {
    rows, err := s.pool.Query(ctx, `
        WITH semantic AS (
            SELECT id, content,
                   1 - (embedding <=> $1) AS semantic_score
            FROM embeddings
            ORDER BY embedding <=> $1
            LIMIT $2 * 2
        ),
        keyword AS (
            SELECT id, content,
                   ts_rank(to_tsvector('english', content), plainto_tsquery($3)) AS keyword_score
            FROM embeddings
            WHERE to_tsvector('english', content) @@ plainto_tsquery($3)
            LIMIT $2 * 2
        )
        SELECT COALESCE(s.id, k.id),
               COALESCE(s.content, k.content),
               COALESCE(s.semantic_score, 0) * 0.7 + COALESCE(k.keyword_score, 0) * 0.3 AS score
        FROM semantic s FULL OUTER JOIN keyword k ON s.id = k.id
        ORDER BY score DESC LIMIT $2`,
        pgvector.NewVector(embedding), topK, query,
    )
    // ... scan rows
}

import (
    "golang.org/x/time/rate"
    "golang.org/x/sync/singleflight"
    "crypto/sha256"
)

type LLMService struct {
    client    *openai.Client
    limiter   *rate.Limiter
    sfg       singleflight.Group
    cache     *ttlcache.Cache[string, string]
    metrics   *prometheus.CounterVec
}

func NewLLMService(client *openai.Client) *LLMService {
    return &LLMService{
        client:  client,
        limiter: rate.NewLimiter(rate.Limit(50), 10),  // 50 RPM, burst 10
        cache:   ttlcache.New[string, string](ttlcache.WithTTL[string,string](1*time.Hour)),
    }
}

func (s *LLMService) Complete(ctx context.Context, prompt string) (string, error) {
    // 1. Rate limit
    if err := s.limiter.Wait(ctx); err != nil { return "", err }

    // 2. Cache key (SHA256 of prompt)
    key := fmt.Sprintf("%x", sha256.Sum256([]byte(prompt)))
    if item := s.cache.Get(key); item != nil {
        s.metrics.WithLabelValues("cache_hit").Inc()
        return item.Value(), nil
    }

    // 3. Deduplicate concurrent identical requests
    result, err, shared := s.sfg.Do(key, func() (any, error) {
        resp, err := s.client.CreateChatCompletion(ctx, openai.ChatCompletionRequest{
            Model:     openai.GPT4oMini,  // use cheaper model when possible
            Messages:  []openai.ChatCompletionMessage{{Role: "user", Content: prompt}},
            MaxTokens: 1024,
        })
        if err != nil { return "", err }
        text := resp.Choices[0].Message.Content
        s.cache.Set(key, text, ttlcache.DefaultTTL)  // cache result
        s.metrics.WithLabelValues("api_call").Inc()
        return text, nil
    })
    _ = shared
    if err != nil { return "", err }
    return result.(string), nil
}

// Retry on rate limit errors
func withRetry(ctx context.Context, fn func() error) error {
    b := backoff.NewExponentialBackOff()
    b.MaxElapsedTime = 60 * time.Second
    return backoff.Retry(func() error {
        err := fn()
        var apiErr *openai.APIError
        if errors.As(err, &apiErr) && apiErr.HTTPStatusCode == 429 {
            return err  // retryable
        }
        return backoff.Permanent(err)  // non-retryable
    }, backoff.WithContext(b, ctx))
}

LLM APIs:

• github.com/sashabaranov/go-openai — OpenAI and compatible APIs (Groq, Together, local). Most popular Go LLM client.
• github.com/anthropics/anthropic-sdk-go — Official Anthropic SDK for Claude models.
• github.com/ollama/ollama/api — Local models via Ollama. Great for dev and privacy-sensitive use cases.

Frameworks:

• github.com/tmc/langchaingo — LangChain port. Chains, agents, vector stores, memory. Good for complex pipelines.
• github.com/philippgille/chromem-go — Lightweight embedded vector DB. No external dependencies for small-scale RAG.

Vector databases:

• github.com/pgvector/pgvector-go — pgvector driver. Best when you're already on Postgres.
• github.com/qdrant/go-client — Qdrant client. Good for filtered vector search at scale.
• github.com/weaviate/weaviate-go-client — Weaviate client. Schema-based, hybrid search.

ML inference:

• github.com/yalue/onnxruntime_go — ONNX Runtime bindings. Run embedding models, classifiers locally.
• gorgonia.org/gorgonia — Tensor library for Go. Building and running neural nets natively.
• gonum.org/v1/gonum — Scientific computing, linear algebra. Not ML-specific but foundational.

Go's AI/ML ecosystem vs Python: Python dominates training. Go excels at serving — low latency, high throughput, simple deployment. The common pattern: train in Python, export to ONNX or TorchScript, serve in Go. For LLM applications (RAG, agents, tool calling), Go's ecosystem is mature enough for production.