7 Caching with generics
This chapter uses a practical cache to introduce Go generics and show when caching helps or hurts. It contrasts stable versus time-sensitive data to frame good caching choices, then walks through building a generic Cache[K comparable, V any] that wraps a map. Along the way it recaps generics essentials: parameterized functions and types, type inference on inputs, constraints like any and comparable, and defining union constraints (including the tilde form for underlying types). The initial cache exposes a small, predictable API—New to initialize, Read to return (value, found), Upsert to insert or overwrite, and Delete as an idempotent removal—while discussing trade-offs like storing values versus pointers and a testing mindset that favors behavior over implementation details.
The chapter then pivots to concurrency fundamentals to prepare for thread safety. It clarifies concurrency versus parallelism, shows how to launch goroutines, and demonstrates three ways to coordinate them: raw channels (including directional channels and close semantics), sync.WaitGroup for simple synchronization, and errgroup for synchronized execution with error propagation and optional limits. With those tools in hand, the naive cache is stress-tested using goroutines and t.Parallel, and race detection via go test -race reveals data races during concurrent writes, proving the first design is not thread-safe.
To harden the cache, a mutex is added next to the protected map; methods lock and defer unlock to serialize access, with a discussion of when (and when not) to use RWMutex for read-heavy paths. The cache is then extended with practical features: per-entry TTL by wrapping values with an expiration timestamp (set on Upsert and validated on Read, removing expired entries on access), and a maximum size with eviction by age. The latter uses a slice to track insertion order and small helper functions to keep the map and ordering in sync, evicting the oldest when capacity is reached and updating recency on overwrite. The chapter closes with guidance and pitfalls: benchmark before preferring RWMutex, use channels only when you truly need inter-goroutine communication, size buffered channels thoughtfully, align concurrency with workload type (CPU vs IO bound), and always ensure goroutines finish cleanly.
Summary
- A cache is a key-value storage facility. They are commonly used when getting the value associated with a key is costly (timewise or in the amount of resources) and when getting a previously retrieved ro computed value is OK.
- When writing a library, one question that should always be raised is “is this library thread-safe ?”. The answer is either “yes, and I know why”, or “No, it’s not”. There is no middle ground - the worst case scenario is usually also the most dangerous.
- Go implements genericity with, well, generics. Structures, variables, and functions can be declared using generics.
- A constraint is a requirement for a generic type. Common constraints are any (quite explicit), comparable, which allows the use of == between two values, or golang.org/x/exp/constraints.Ordered., which allows the use of >, <, >=, <= between two values.
- Constraints are passed in square brackets after the name of the generic entity:
- type fieldWithName[T any] struct { value T, name string }
- var hashIndex[T myConstraint] map[uint]T
- func sortSlice[T constraints.Ordered](t []T) ([]T, error)
- Constraints can be omitted in the declaration of functions when the compiler can infer which type it should be using: sortSlice([]int{1,4,3,2}).
- Go uses goroutines for concurrency. Goroutines are similar to what other languages usually call threads, except that they’re not. Threads live at an OS-level, while goroutines live farther from your silicon - they exist in the runtime environment of Go.
- In Go, goroutines are launched with the keyword go: go do() runs the do function in a new goroutine.
- Channels to communicate data between different goroutines. When passing a channel to a function, make it explicit in the signature that the function will either read from the channel, with the syntax func f(c <- chan string), or that it will write to the channel, with the syntax func f(c chan <- string). If you need to both read from and write to a channel in a single function, there is probably a design flaw.
- Two types are commonly used when we need to synchronise goroutines: sync.WaitGroup and errgroup.Group. When using sync.WaitGroup, start by calling Add(n) with the number of goroutines that will be executed. Each goroutine is in charge of calling Done. The synchronisation is achieved by calling Wait(). When using errgroup.Group, start by setting a maximum number of parallel goroutines with SetLimit(n). Launch each goroutine with a call to Go(...). The synchronisation is achieved with a call to Wait(), which returns an error if one of the goroutines returns an error.
- The choice of sync.WaitGroup or errgroup.Group is often driven by the necessity to check for errors in at least one goroutine: use sync.WaitGroup when errors don’t need a specific treatment. Use errgroup.Group if you want to handle errors.
- Mutexes are used whenever we want to protect a variable from concurrent writing - or reading. In Go, we can create a mutex variable by using the sync.Mutex type: var myMutex sync.Mutex. A mutex should never be exposed - instead, use it in exposed functions. A mutex should always be written close to the variable or field it protects.
- You can call mu.Lock() on a mutex, but we highly recommend immediately following this with defer mu.Unlock(). Debugging locked mutexes is a pain.
- Use t.Parallel() in your tests to let the framework know that a test is not blocking for the execution of other tests.
- Use the -race flag when testing to try and detect data races - but remember, the failure to detect a data race doesn’t mean there are no data races.
- Use the -trimpath flag when testing to only output paths relatively to the root of the module.
- A switch/case statement will only execute the first valid case - any subsequent case will be ignored.
FAQ
What is a cache, and when should I use or avoid one?
A cache is a key–value store for reusing results that were already computed. Use it when the same expensive function is called repeatedly with identical inputs and the output is stable (for example, “longest city name”). Avoid it for frequently changing data where staleness is harmful (for example, live FX rates). It can be acceptable for slowly changing values where last week’s value is “good enough” (for example, population estimates).How do generics help implement a cache in Go?
Generics let you define a reusable cache type like “Cache[K comparable, V any]” so it works with any comparable key type and any value type. You implement “New”, “Read”, “Upsert”, and “Delete” once, and reuse the same cache API across many key/value combinations without boilerplate.Why must cache keys be comparable?
Go maps require keys to be comparable, so a generic cache that uses a map must constrain “K” with “comparable”. Comparable types support “==” and “!=” (bools, numbers, strings, pointers, channels, comparable arrays/structs). Slices, maps, and functions are not comparable, so they cannot be used as map keys.How do I make the cache concurrency-safe?
- First, prove you have a race: exercise writes in parallel (goroutines or “t.Parallel()”) and run “go test -race”.- Protect shared state with a “sync.Mutex” (or “sync.RWMutex” when appropriate). Lock early, “defer Unlock()”, and use pointer receivers so the mutex isn’t copied. After adding locks, re-run with “-race” to confirm the fix.
What’s the difference between concurrency and parallelism in Go?
Concurrency is a way to structure programs so independent tasks can make progress without blocking each other; parallelism is simultaneous execution on multiple CPU cores. Goroutines give you concurrency; whether they run in parallel depends on the scheduler and available cores. As the proverb says: “Concurrency is not parallelism.”How can I coordinate goroutines finishing: channels vs WaitGroup vs errgroup?
- Channels: send a “done” signal (often “struct{}”) and read from it to wait. Flexible but more boilerplate.- sync.WaitGroup: call “Add(n)”, “go func(){ defer Done() }()”, then “Wait()”. Simple, no error propagation.
- errgroup.Group: “Go(func() error)”, then “Wait()” returns an error from a goroutine; “SetLimit(n)” bounds concurrent work. Great when you need error handling.
How do I detect data races in tests?
Run “go test -race”. The race detector instruments your code and reports concurrent conflicting accesses (for example, two goroutines writing the same map entry). Optional “--trimpath” makes paths shorter in the output. Fix races with proper synchronization and re-run “-race”.When should I use sync.Mutex vs sync.RWMutex in the cache?
- Prefer “sync.Mutex” by default: simpler and hard to misuse.- Consider “sync.RWMutex” only when reads dominate and profiling shows contention; use “RLock/RUnlock” for read-only sections and “Lock/Unlock” for writes. Note: if “Read()” might delete expired items (TTL), it writes to the map, so you must use full “Lock/Unlock”. Always benchmark to justify “RWMutex”.
How do I add a TTL (time to live) to cache entries?
Wrap values in an “entryWithTimeout” that stores the value and an “expires time.Time”. Have “New” accept a “time.Duration” TTL. On “Upsert”, set “expires = time.Now() + TTL”. On “Read”, if the key is present but expired, delete it and return “not found”; otherwise return the value. This evicts on access; it avoids a background GC goroutine but may keep some expired entries until read.How can I cap cache size and evict the oldest entry?
Track insertion order with a slice of keys (for example, “chronologicalKeys”). Store “maxSize” and maintain both the map and the slice via helpers like “addKeyValue” and “deleteKeyValue”. On “Upsert”:- If key exists, remove it and re-add to move it to the tail (freshest).
- If at capacity and key is new, remove the oldest key (slice[0]) and insert the new one.
You can use “slices.DeleteFunc” (Go 1.21) to remove keys from the slice.
Learn Go with Pocket-Sized Projects ebook for free
