RAG from First Principles · Part 12

The cost / latency / quality triangle

One RAG request, four stages, two bars each. Toggle the levers and watch a corner move.

Optimizations

Per request, by stage

latency cost baseline marker

Readout

Latency / request 2000ms

First token 2000ms

Cost / request 1.000cents

With cache (avg, 35% hit)

latency2000 ms

cost1.000 cents

Answer quality 100