The Inbox & Outbox Pattern: Reliable Events with Kafka 📬
How to stop losing (and duplicating) events when your service writes to a database and Kafka at the same time.
When hand-wiring main.go stops scaling, and what Fx's graph resolution and lifecycle ordering buy you in return.
Go doesn't have a built-in DI container, and for a long time that felt like a feature, not a gap. Most Go services wire their dependencies by hand in main.go, and for a small service that's genuinely the right call. But once you're running several services, each with a handful of adapters (database, cache, message broker, background job queue), manual wiring starts to show cracks. This post walks through what dependency injection actually buys you in Go, and where a framework like Uber Fx fits in.
A typical financial backend service — say, a payment service — ends up depending on Postgres, Kafka, Redis, and a handful of internal ports and adapters. Wired by hand, main.go looks like this:
db := connectPostgres(cfg)
repo := NewTransactionRepo(db)
kafkaWriter := NewKafkaWriter(cfg)
publisher := NewEventPublisher(kafkaWriter)
redisClient := NewRedisClient(cfg)
asynqClient := NewAsynqClient(redisClient)
enqueuer := NewJobEnqueuer(asynqClient)
useCase := NewPaymentUseCase(repo, publisher, enqueuer)
handler := NewPaymentHandler(useCase)
server := NewHTTPServer(handler)
server.Start()This works fine at this size. The trouble starts when you need to also start a background worker, register graceful shutdown for five different resources in the correct order, and repeat a similar pattern across three or four services that share common infrastructure code. The wiring becomes the least interesting and most error-prone part of the codebase.
Dependency Injection is simply: a component receives its dependencies from the outside, instead of constructing them itself.
flowchart LR
subgraph Without DI
A["PaymentUseCase"] -->|creates directly| B["PostgresRepo"]
A -->|creates directly| C["KafkaWriter"]
end
subgraph With DI
D["PaymentUseCase"] -.->|depends on interface| E["TransactionRepository"]
F["PostgresRepo"] -->|implements| E
G["Constructor injects concrete adapter"] -.-> D
endIn Go, this pairs naturally with small interfaces defined by the consumer, not the implementer — the standard idiom behind Clean Architecture's Ports & Adapters pattern:
// application layer — defines what it needs, knows nothing about Postgres
type TransactionRepository interface {
Save(ctx context.Context, tx *domain.Transaction) error
}
type PaymentUseCase struct {
repo TransactionRepository
}
func NewPaymentUseCase(repo TransactionRepository) *PaymentUseCase {
return &PaymentUseCase{repo: repo}
}PaymentUseCase never imports the postgres package. Something else — a human writing main.go, or a DI framework — decides which concrete adapter satisfies that interface at startup. That decision point is called the composition root.
Fx doesn't change this design — it automates the composition root. You register constructors, and Fx inspects their function signatures via reflection to resolve a dependency graph.
flowchart TB
CFG["Config"] --> DB["*sql.DB"]
CFG --> KAFKA["*kafka.Writer"]
DB --> REPO["TransactionRepository"]
KAFKA --> PUB["EventPublisher"]
REPO --> UC["PaymentUseCase"]
PUB --> UC
UC --> HANDLER["HTTP Handler"]
HANDLER --> SERVER["HTTP Server"]fx.New(
fx.Provide(
NewConfig,
NewPostgresDB,
NewTransactionRepo, // returns application.TransactionRepository
NewKafkaWriter,
NewEventPublisher, // returns application.EventPublisher
NewPaymentUseCase,
NewPaymentHandler,
NewHTTPServer,
),
fx.Invoke(RegisterRoutes),
).Run()Two primitives do all the work:
fx.Provide — "here's how to build a T." Lazy: only invoked if something in the graph needs a T.fx.Invoke — "run this now, resolving its arguments from the graph." Eager: this is where real side effects happen (starting a server, registering routes).Nothing executes until .Run() is called — Fx builds the entire graph first, so a missing or ambiguous dependency fails fast at startup, before any traffic is served.
Manual wiring makes you write shutdown logic by hand, in the correct reverse order, for every resource. Miss the order and you risk closing a database connection while a request is still mid-write. Fx's fx.Lifecycle solves this structurally: any constructor can register OnStart/OnStop hooks, and Fx runs OnStop hooks in reverse dependency order automatically.
func NewHTTPServer(lc fx.Lifecycle, handler http.Handler) *http.Server {
srv := &http.Server{Handler: handler}
lc.Append(fx.Hook{
OnStart: func(ctx context.Context) error {
ln, err := net.Listen("tcp", ":8080")
if err != nil {
return err
}
go srv.Serve(ln)
return nil
},
OnStop: func(ctx context.Context) error {
return srv.Shutdown(ctx) // drains in-flight requests first
},
})
return srv
}On SIGTERM, the HTTP server stops accepting new work and drains active requests before Fx closes the gRPC client it depends on, which in turn closes before the database connection. This ordering falls directly out of the same graph used for injection — no hand-maintained shutdown sequence to get wrong.
A common real case: two adapters implement the same port, and you need to tell them apart. fx.Annotate with named values solves this cleanly:
fx.Provide(
fx.Annotate(
NewKafkaNotifier,
fx.As(new(NotificationPort)),
fx.ResultTags(`name:"async_notifier"`),
),
fx.Annotate(
NewSMSNotifier,
fx.As(new(NotificationPort)),
fx.ResultTags(`name:"sync_notifier"`),
),
)Consumers request the specific one they need via a matching tag on the field, so the graph stays unambiguous even with duplicate interface implementations.
The same graph that wires production makes tests cheap. Build a throwaway app that provides a fake for the one adapter you're isolating, pull the value you want to assert on with fx.Populate, and let Fx wire the rest. fxtest even runs the lifecycle, so OnStart/OnStop hooks fire exactly as they do in production.
func TestPaymentFlow(t *testing.T) {
var useCase *PaymentUseCase
app := fxtest.New(t,
fx.Provide(
func() TransactionRepository { return newFakeRepo() }, // fake, not Postgres
NewEventPublisher,
NewPaymentUseCase,
),
fx.Populate(&useCase), // hand the built value back to the test
)
app.RequireStart()
defer app.RequireStop()
// useCase is now wired with the in-memory repo — assert against it
}When you'd rather override a single provider inside a larger shared module than re-list constructors, fx.Decorate replaces one T while leaving the rest of the graph intact. Either way, you never hand-wire the dependencies a given test doesn't care about.
| Benefit | Cost |
|---|---|
| Wiring stays flat as services grow | Missing dependency is a runtime error, not a compile-time one |
| Shutdown ordering is automatic and correct | Constructor-to-consumer links are implicit — harder to grep than explicit wiring |
Easy to swap fakes in tests (fxtest, fx.Populate) | Reflection-based — a learning curve for the error messages |
For a two-service prototype, manual wiring is often still the pragmatic choice — the cost of Fx isn't justified yet. It starts paying off once you have several services sharing common provider packages (config, logging, database pools) and enough adapters that the composition root would otherwise become a multi-hundred-line function.
DI in Go isn't about a container — it's a design discipline: depend on interfaces defined by the consumer, and decide the concrete wiring in one place. Uber Fx doesn't introduce that discipline; it automates the mechanical part of it — graph resolution and lifecycle ordering — once manual wiring becomes the bottleneck rather than the business logic.
How to stop losing (and duplicating) events when your service writes to a database and Kafka at the same time.
What makes Kafka a distributed log rather than a queue, and when to reach for RabbitMQ or Redis instead.
Practical patterns for retries, idempotency, and observability in Lambda-backed product APIs.