Scaling LLM Inference with Disaggregated Prefill and Decode
Disaggregated prefill/decode isn’t an academic concept it’s the fix that saved my LLM serving at scale for 40M users. Here’s exactly how it works and why mon...
Recent Posts
Deep Dive into Fine Tuning a LoRA Reranker on Phi-3
I fine tuned Phi-3 as a pairwise reranker with LoRA and logged every gradient. Early layers changed 200x more than late layers, but ranking representations o...
Text Diffusion: The Idea Hiding Inside BERT All Along
Auto regressive generation is sequential and diffusion uses much fewer passes in text generation.
Cross Entropy Loss Connection to GPT Models
Cross-entropy loss isn’t a heuristic, it is maximum likelihood estimation with a sign flip. It also shows how the same math powers GPT training.
RLHF vs RLAIF vs RLVR: The Three Ways to Teach AI Models
Understanding the basics of RLHF vs RLAIF vs RLVR for AI feedback comparison