Posted By
Go is built for concurrency. Its lightweight goroutines, type-safe channels, and composable primitives make it one of the best languages available for building concurrent systems. But understanding what is concurrency in Go is just the beginning — knowing which golang concurrency patterns to apply, when, and why is what separates robust production code from fragile, leaky goroutine soup.
This go concurrency patterns guide walks through every major pattern with working code examples: from the basics of confinement and for-select loops to pipelines with cancellation, fan-out/fan-in, worker pools, tee channels, bridge channels, and the modern context package. Whether you are building a high-throughput data pipeline, a concurrent API client, or a distributed backend service, these patterns will help you write Go that is safe, readable, and leak-free.
If you are also exploring how these patterns fit into larger distributed systems, Opcito's software product engineering practice covers the full spectrum of building production-grade Go applications at scale.
The Go Concurrency Model
Go's approach to concurrency is grounded in CSP (communicating sequential processes). Instead of relying on shared memory and locks, Go goroutines communicate by passing values through channels. This model makes many concurrency problems easier to reason about — and it is the foundation for all the golang concurrency patterns in this guide.
Goroutines
A goroutine is a lightweight thread managed by the Go runtime. You start one by prefixing any function call with the go keyword:
go doWork()
The Go runtime multiplexes thousands of goroutines onto a smaller number of OS threads using an internal scheduler. This makes goroutines extremely cheap to create — but it also means unmanaged goroutines can quietly accumulate into a resource leak if you are not careful.
Channels
Channels are the primary mechanism for safe communication between goroutines in Go concurrency. They eliminate the need for explicit locks or shared-memory coordination:
ch := make(chan int)
go func() {
ch <- 42 // send
}()
fmt.Println(<-ch) // receive
Understanding go channel patterns is central to writing idiomatic Go. Buffered channels add capacity and decouple send/receive timing; unbuffered channels enforce synchronization.
Select Statement
The select statement lets a goroutine wait on multiple channel operations simultaneously — like a switch that works on channels:
select {
case msg := <-ch1:
fmt.Println("Received", msg)
case <-time.After(time.Second):
fmt.Println("Timeout")
}
select is foundational for timeouts, cancellation, and multiplexing in concurrency patterns in golang.
Go Concurrency patterns you need to know
The rest of this golang concurrency tutorial covers each major pattern in depth. Each section includes a working example and a real-world use case so you can apply them directly to your own systems.
Confinement
Confinement keeps data accessible to only one goroutine at a time. By giving one goroutine exclusive ownership of a data structure, you eliminate race conditions without any locks.
In Go, this typically means one goroutine owns and writes to a channel, while all other goroutines only read from it. Because only the owner writes or closes the channel, there is no concurrent access to protect against.
func producer() <-chan int {
out := make(chan int, 5)
go func() {
defer close(out)
for i := 0; i < 5; i++ {
out <- i
}
}()
return out
}
func main() {
// The main goroutine is the consumer; the producer confines the writes.
for v := range producer() {
fmt.Println(v) // safely receives 0,1,2,3,4
}
}
The producer goroutine writes to its own channel and closes it when done. No other goroutine writes to that channel, so no locks are needed. This pattern is well-suited for feeding data from a sensor, network connection, or batch job into a channel that worker goroutines safely consume. Confinement also extends to slices and maps where each goroutine is assigned its own index or partition, keeping access sequential and lock-free.
The for-select Loop
The for-select loop is idiomatic Go for any long-lived goroutine that must wait on channels or respond to a stop condition. It pairs a for loop with select to check a cancellation channel on each iteration:
func worker(done <-chan struct{}) {
for {
select {
case <-done:
// Stop the loop and exit.
return
default:
}
// Perform non-blocking work here.
fmt.Println("working")
time.Sleep(100 * time.Millisecond)
}
}
func main() {
done := make(chan struct{})
go worker(done)
time.Sleep(500 * time.Millisecond)
close(done) // signal worker to exit
}
Here, the goroutine loops indefinitely but uses a select to check <-done each time. If a value (or a close) is received on done, the goroutine returns. This pattern shows up throughout Go. You’ll see two common variants:
- Minimal select with default: keep the select as short as possible (case <-done and default), then do long-running work after. Example above.
- Select around work: put the work inside select cases (e.g. case msg := <-workCh: vs case <-done:).
Either way, the for-select loop cleanly handles waiting on multiple channels or an exit signal. It’s fundamental for patterns like rate-limited loops, worker goroutines, and pipelines.
Preventing goroutine leaks
A goroutine leak happens when a goroutine never terminates because it’s stuck waiting (perhaps on a channel that no one will ever send on). To prevent leaks, always ensure goroutines can exit when they should. A common approach is to use a done channel (or better, a context.Context) that signals cancellation. For example:
func doWork(done <-chan struct{}) {
for {
select {
case <-done:
fmt.Println("doWork canceled")
return
default:
}
// Simulate work
time.Sleep(200 * time.Millisecond)
}
}
func main() {
done := make(chan struct{})
go doWork(done)
// Cancel after 1 second to avoid leak
time.Sleep(1 * time.Second)
close(done)
// Wait a bit for doWork to print cancelation message
time.Sleep(100 * time.Millisecond)
}
Here, the doWork loop checks <-done and exits when done is closed. The key insight is always arrange for your goroutines to terminate. Using a cancellation channel (or context) ensures you “join” goroutines instead of leaving them running forever, avoiding resource leaks.
The or-channel pattern
An or-channel (sometimes called the orDone pattern) merges multiple cancellation channels into one. It returns a single channel that closes as soon as any of its inputs close — effectively a logical OR across cancellation signals:
// OrChannel combines multiple channels into one that closes when any input closes.
func OrChannel(channels ...<-chan interface{}) <-chan interface{} {
switch len(channels) {
case 0:
return nil
case 1:
return channels[0]
}
orDone := make(chan interface{})
go func() {
defer close(orDone)
switch len(channels) {
case 2:
select {
case <-channels[0]:
case <-channels[1]:
}
default:
// Recursively wait on others
select {
case <-channels[0]:
case <-channels[1]:
case <-channels[2]:
case <-OrChannel(append(channels[3:], orDone)...):
}
}
}()
return orDone
}
This function starts a goroutine that waits on any of the input channels. As soon as any channel receives or closes, it closes orDone, which signals all consumers. For instance, you might use it to wait for the first of several timeouts or events. This “closes if any of its component channels close”. A real-world use case: waiting for the first response from multiple servers or cancelling a task if any of several error signals occur.
Error handling in concurrent Go programs
In concurrent code, parent goroutines need a reliable way to receive errors from child goroutines. The idiomatic approach is to bundle both the result and the error into a single struct and send it over a channel:
type Result struct {
Err error
Response *http.Response
}
// checkStatus queries multiple URLs and reports each Response or error.
func checkStatus(done <-chan interface{}, urls ...string) <-chan Result {
results := make(chan Result)
go func() {
defer close(results)
for _, url := range urls {
resp, err := http.Get(url)
select {
case <-done:
return
case results <- Result{Err: err, Response: resp}:
}
}
}()
return results
}
func main() {
done := make(chan interface{})
defer close(done)
urls := []string{"https://example.com", "https://golang.org", "https://doesnot.exist"}
errCount := 0
for res := range checkStatus(done, urls...) {
if res.Err != nil {
fmt.Printf("error: %v\n", res.Err)
errCount++
if errCount >= 2 {
fmt.Println("too many errors, exiting")
break
}
continue
}
fmt.Printf("Status: %d\n", res.Response.StatusCode)
}
}
Here, the Result struct carries both the HTTP response and any error. The parent loop reads from the channel and inspects res.Err. Real-world scenarios: contacting multiple services and handling whichever fail, or aggregating errors from parallel tasks. The parent can decide when to abort (as shown by breaking out after too many errors) while child goroutines simply send back their results via the shared channel.
Pipelines
A pipeline is a series of stages connected by channels, where each stage runs in its own goroutine (or set of goroutines) and passes data to the next stage. Think of it like Unix pipes but in Go. Each stage receives from an inbound channel, does work (filtering, transformation, etc.), and sends results downstream. When done, each stage closes its outbound channel.
For example, a simple pipeline with two stages – “generator” and “squarer” – can be written as follows (in idiomatic Go using range loops):
// gen emits the list of ints onto a channel and then closes it.
func gen(nums ...int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for _, n := range nums {
out <- n
}
}()
return out
}
// sq reads ints from 'in', squares them, and sends them on its returned channel.
func sq(in <-chan int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for n := range in {
out <- n * n
}
}()
return out
}
func main() {
// Set up the pipeline: gen -> sq -> sq
for n := range sq(sq(gen(2, 3))) {
fmt.Println(n) // prints 16 then 81
}
}
Each stage closes its channel when complete. Real-world usage: data processing pipelines, streaming transformations, log processing stages, etc.
Best practices: Always have a defer close(channel) in each stage. Also, it’s important to allow early exit. In practice, we include a “done” or context.Context to cancel a pipeline if some stage fails. For instance, we can modify stages to take a done <-chan struct{} or ctx context.Context and return early if canceled. This avoids goroutine leaks when downstream stages stop reading. For brevity, the example above omits cancellation, but in production code you’d typically pass a context to each stage to allow terminating the pipeline early.
Handy generators for pipelines
Generators are utility functions that produce streams of values to feed pipelines. Two common ones are Repeat (emits values endlessly) and Take (reads only the first N values):
- Repeat: repeatedly emits a set of values (e.g. 1,2,3,1,2,3,…) on a channel until stopped.
- Take: reads the first N values from a channel and then closes.
Example using a done channel to stop:
// repeat sends the given values endlessly on a channel until done is closed.
func repeat(done <-chan struct{}, values ...int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for {
for _, v := range values {
select {
case <-done:
return
case out <- v:
}
}
}
}()
return out
}
// take reads up to 'n' values from 'in' and then closes.
func take(done <-chan struct{}, in <-chan int, n int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for i := 0; i < n; i++ {
select {
case <-done:
return
case out <- <-in:
}
}
}()
return out
}
func main() {
done := make(chan struct{})
defer close(done)
// Repeat 1,2,3 until taken 5 values.
for v := range take(done, repeat(done, 1, 2, 3), 5) {
fmt.Print(v, " ") // output: 1 2 3 1 2
}
}
It’s handy for testing examples: you can generate infinite streams of data and then constrain them. Updated idiomatic tip: in modern Go you might use a context.Context instead of a done channel, and you could use type parameters (generics) to avoid int vs interface{} issues.
Fan-Out, Fan-In
Fan-Out means starting multiple goroutines (workers) to process items from the same channel. Fan-In means merging results from multiple channels back into one. This is used to parallelize work. For example, we might send the same input to several workers (fan-out) and then merge their outputs (fan-in).
The Go blog demonstrates this by running two sq stages in parallel:
in := gen(2, 3)
// Two parallel workers for sq (fan-out).
c1 := sq(in)
c2 := sq(in)
// fan-in: merge results from c1 and c2.
out := merge(c1, c2)
for n := range out {
fmt.Println(n) // prints 4 and 9 (order not guaranteed)
}
The merge function (aka fan-in) uses a sync.WaitGroup to copy from each input channel to one output channel, then closes the output when done:
func merge(chs ...<-chan int) <-chan int {
var wg sync.WaitGroup
out := make(chan int)
output := func(c <-chan int) {
defer wg.Done()
for v := range c {
out <- v
}
}
wg.Add(len(chs))
for _, c := range chs {
go output(c)
}
// Close out when all copies are done
go func() {
wg.Wait()
close(out)
}()
return out
}
Fan-out is useful when you have CPU-bound or I/O-bound tasks and want to spread work across cores or handle multiple requests. Fan-in (merge) is essential to collect all results into one place. The pattern is widely used in pipelines for parallelism.
Worker Pool
A worker pool limits concurrent work to a fixed number of goroutines. Instead of spawning a goroutine per task (which can flood memory/FDs/CPUs), you create N workers that pull tasks from a queue channel, process them, and send results back. This yields steady throughput with predictable resource usage and natural backpressure (the task queue fills when producers outpace workers).
Where it shines
- Hitting an external API with rate or connection limits
- CPU-bound jobs (hashing, image transforms) where you want to match GOMAXPROCS
- Disk/network scraping where too much parallelism hurts more than it helps
Minimal Worker Pool: Tasks, results, done, and fan-In
The example below is a self-contained, production-ready pattern using:
tasks <-chan Tas the input queue (read-only to workers)results chan<- Ras the output sinkdone <-chan struct{}to cancel/stopsync.WaitGroupto join workers- fan-in close: only close results after all workers finish
Example scenario
We’ll “process” integers by squaring them, but you can imagine each task being an HTTP call, DB query, or CPU job.
package main
import (
"fmt"
"math/rand"
"sync"
"time"
)
// Task and Result types (can be any domain-specific payloads)
type Task struct {
ID int
Val int
}
type Result struct {
TaskID int
Out int
Err error
}
// worker consumes tasks and produces results.
// It must: respect 'done', never panic, and never leak.
func worker(
id int,
done <-chan struct{},
tasks <-chan Task,
results chan<- Result,
) {
for {
select {
case <-done:
// Cooperative cancellation.
return
case t, ok := <-tasks:
if !ok {
// No more tasks; clean exit.
return
}
// Simulate variable work (I/O/CPU/etc.)
time.Sleep(time.Duration(rand.Intn(150)+50) * time.Millisecond)
// Example "processing":
out := t.Val * t.Val
// Send the result OR honor cancellation.
select {
case <-done:
return
case results <- Result{TaskID: t.ID, Out: out, Err: nil}:
}
}
}
}
// startWorkerPool launches 'n' workers and returns a function to wait for them.
// The caller is responsible for: closing 'tasks' when done submitting, and
// closing 'results' AFTER all workers finish (we do that here via a helper goroutine).
func startWorkerPool(
n int,
done <-chan struct{},
tasks <-chan Task,
) (<-chan Result, *sync.WaitGroup) {
results := make(chan Result)
var wg sync.WaitGroup
wg.Add(n)
for i := 0; i < n; i++ {
go func(id int) {
defer wg.Done()
worker(id, done, tasks, results)
}(i + 1)
}
// Close results after all workers exit.
go func() {
wg.Wait()
close(results)
}()
return results, &wg
}
func main() {
rand.Seed(time.Now().UnixNano())
done := make(chan struct{}) // signal to stop early
tasks := make(chan Task, 8) // buffered queue -> backpressure control
const workers = 4 // pool size
results, _ := startWorkerPool(workers, done, tasks)
// Producer: enqueue tasks (could be another goroutine reading a file/DB/etc.)
go func() {
defer close(tasks) // signal "no more tasks"
for i := 1; i <= 20; i++ {
select {
case <-done:
return
case tasks <- Task{ID: i, Val: i}:
// enqueued
}
}
}()
// Consumer: drain results. In real systems, this might be writing to a DB, another stage, etc.
for r := range results {
if r.Err != nil {
fmt.Printf("task %d error: %v\n", r.TaskID, r.Err)
continue
}
fmt.Printf("task %02d -> %d\n", r.TaskID, r.Out)
// Example early-stop condition (optional):
// if r.TaskID == 10 { close(done) }
}
fmt.Println("all done")
}
Why this worker pool design works
- Backpressure: The buffered tasks channel controls enqueue rate. When full, producers block—preventing unbounded memory growth.
- Bounded concurrency: Exactly workers goroutines execute tasks at once.
- Clean shutdown: Closing tasks lets workers exit naturally; closing done cancels immediately.
- No leaks: Workers select on done and exit; results is closed only after workers finish.
Tee Channel
A tee-channel duplicates a stream: it reads from one input channel and sends each item to two output channels. Each output gets every value from the input (like teeing a garden hose). This allows, for instance, logging and processing the same events in parallel. Example:
func tee(done <-chan interface{}, in <-chan interface{}) (<-chan interface{}, <-chan interface{}) {
out1 := make(chan interface{})
out2 := make(chan interface{})
go func() {
defer close(out1)
defer close(out2)
for val := range orDone(done, in) {
// Try sending to out1 and out2, but if one sends successfully,
// disable that channel to avoid double-sending to it.
c1, c2 := out1, out2
for i := 0; i < 2; i++ {
select {
case c1 <- val:
c1 = nil
case c2 <- val:
c2 = nil
}
}
}
}()
return out1, out2
}
// Example usage:
done := make(chan struct{})
in := make(chan interface{})
go func() {
defer close(in)
for i := 0; i < 5; i++ {
in <- i
}
}()
c1, c2 := tee(done, in)
for x := range c1 {
fmt.Println("c1:", x)
if x == 2 {
close(done) // stop tee after some values
}
}
for x := range c2 {
fmt.Println("c2:", x)
}
In this code, each value from in is sent once to both out1 and out2. The trick of setting c1 = nil or c2 = nil ensures each value is sent exactly once to each channel. Use case: splitting a stream of events to two consumers (e.g. one for logging, one for business logic).
Bridge Channel
A bridge channel flattens a stream of channels into a single stream of values. Imagine you have a channel that emits other channels (like <-chan <-chan T). The bridge will take each inner channel in turn and emit its values on one output channel. This is useful when you have dynamic pipelines that produce more channels. For example:
func bridge(done <-chan interface{}, chanStream <-chan (<-chan interface{})) <-chan interface{} {
valStream := make(chan interface{})
go func() {
defer close(valStream)
for {
var stream <-chan interface{}
select {
case maybeStream, ok := <-chanStream:
if !ok {
return
}
stream = maybeStream
case <-done:
return
}
// Drain values from the current inner channel.
for v := range orDone(done, stream) {
select {
case valStream <- v:
case <-done:
}
}
}
}()
return valStream
}
// Usage:
// chanStream := make(chan (<-chan interface{}))
// ... send some channels on chanStream ...
// The bridge will output all values from those channels in order.
This code takes a chanStream (a channel of channels). Each time a new channel is received, it consumes that channel fully (or until done) and emits all its values on valStream. Use case: You might use this if an API returns multiple channels of results and you want to process them in one loop.
Queuing with buffered channels as semaphores
The Queuing pattern uses a buffered channel as a semaphore to limit the number of concurrent tasks. This decouples task submission from execution and throttles concurrency. For example, to process up to N tasks in parallel (say N workers), you can use a channel of size N:
func main() {
const numWorkers = 3
const numTasks = 5
var wg sync.WaitGroup
sem := make(chan struct{}, numWorkers)
for i := 1; i <= numTasks; i++ {
wg.Add(1)
// Acquire a slot.
sem <- struct{}{}
go func(taskID int) {
defer wg.Done()
// Do the work.
fmt.Println("Start task", taskID)
time.Sleep(500 * time.Millisecond)
fmt.Println("Done task", taskID)
// Release the slot.
<-sem
}(i)
}
wg.Wait()
}
Here, sem is a buffered channel of capacity numWorkers. Each goroutine sends a dummy struct to sem before working (blocking if sem is full) and receives from sem (releasing) when done. This ensures at most numWorkers tasks run concurrently. The Go Patterns site describes this: “A buffered channel is commonly used as a semaphore to throttle the number of active goroutines,” providing backpressure and preventing resource exhaustion. Use case: limiting concurrent network calls, database queries, or CPU-heavy jobs in a server.
The context Package
In modern Go, many cancellation and timeout patterns use context.Context instead of raw channels. The context package provides deadlines, cancellation signals, and value propagation across API boundaries. For example:
func work(ctx context.Context) {
for i := 0; i < 5; i++ {
select {
case <-time.After(1 * time.Second):
fmt.Println("Work step", i)
case <-ctx.Done():
fmt.Println("Work canceled at step", i)
return
}
}
}
func main() {
// Create a context that is canceled after 2 seconds.
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
go work(ctx)
// Wait for work to finish or be canceled.
time.Sleep(3 * time.Second)
}
This example uses context.WithTimeout to automatically cancel the context after 2 seconds. The work function checks ctx.Done() (like a done channel) to know when to stop. Cox-Buday’s chapter emphasizes that context can be used “to add timeouts and cancellations” to patterns. In short, use context to manage lifecycles instead of custom channels when possible: it’s idiomatic, integrates with many libraries, and makes cancellation explicit.
Choosing the right Go concurrency pattern
With so many golang concurrency patterns available, the decision comes down to what you are trying to protect against and what your data flow looks like. Here is a practical guide to pattern selection:
- Confinement: Use when you can give each goroutine exclusive ownership of the data it needs (no sharing). Great for simple pipelines and one-writer scenarios.
- For-Select Loop: Use this as the basic structure for any long-lived goroutine that needs to wait on channels or implement timeouts (select on <-time.After, <-done, etc.).
- Done / Context: Always incorporate a cancellation mechanism (a done channel or, better, context.Context) in long-running goroutines and pipelines, to avoid leaks.
- Pipelines: Use when you have staged processing (e.g. transforming a stream of data step-by-step). They allow composability and parallelism.
- Fan-Out/Fan-In: Use when you need to parallelize independent work (fan-out) and then combine results (fan-in). Good for parallelizing CPU-bound tasks or doing multiple I/O calls concurrently.
- Generators: Handy for creating test data or streaming values when you need infinite or repeatable inputs.
- Queuing: Use when you must limit concurrency (e.g. to N workers). A buffered channel as semaphore is a simple and effective strategy.
- Advanced Patterns (or-channel, or-done, tee, bridge): Use when combining or splitting streams in complex ways. For most everyday code, these are less common, but they can simplify code for broadcasting to multiple consumers or merging streams of streams.
Putting it all together — concurrency patterns in Go in practice
The patterns in this guide are not mutually exclusive. A real production system might use a pipeline of stages, where each stage runs a worker pool, fan-in merges the results, a tee channel splits output to a logger and a database writer, and context propagates cancellation from the top-level request handler all the way through.
Go's CSP-based model gives you composable primitives that scale from simple concurrent loops to complex distributed pipelines. The key design principles that run through all golang concurrency patterns are the same: communicate through channels rather than shared memory, ensure every goroutine can terminate, and make cancellation explicit.
When these patterns are applied consistently, Go code becomes predictable, race-condition-free, and easy to test. The -race flag remains your best tool for catching data races during development, and context.Context is your best tool for managing goroutine lifetimes in production.
If you are building Go-based systems and want to discuss how these concurrency patterns fit into your architecture or CI/CD pipeline, get in touch with Opcito's Go experts — whether you are starting a new product, modernizing an existing service, or looking to improve the reliability of a system already in production.
