Before you read
Comfort with the command line and basic GPU terminology.
What a laptop-sized inference stack actually needs, and what it can safely skip.
The three layers
A local stack is a runtime, a serving layer, and a client. You can replace
any one of them without touching the others, and most troubleshooting is
figuring out which layer is actually failing.
What you can skip
You do not need a queue, a router, or an observability stack on day one.
You do need a pinned model version, because "it worked yesterday" is
untestable across weight updates.
Quantisation is a trade, not a default
Smaller quants run on less memory and answer differently. Pick the largest
quant your evaluation set tolerates, not the smallest your hardware allows.