Training, inference, and retrieval
Explain what changes in training and what a model uses at inference time.
- Explain what changes in training and what a model uses at inference time.
Our bigram model counts which word follows the previous word. If red follows sky three times and blue once, their probabilities are 3/4 and 1/4. Modern neural language models instead optimize many weights to reduce prediction loss. At inference time they use those weights and the available context to calculate scores for the next token. This demo illustrates conditional prediction, not the full Transformer algorithm.
A closer look
Fine-tuning changes model weights; LoRA trains smaller adapter matrices while base weights can stay frozen. RAG retrieves material and supplies it as context without requiring weight updates. BM25 ranks lexical matches using term frequency with saturation and document-length normalization; embedding retrieval compares learned vectors. Either can miss evidence or retrieve misleading text. A chat message is not automatically a training update.
Explain it yourself
An app retrieves a new manual and adds it to the prompt. Is that necessarily fine-tuning?
Compare with an explanation
No. Retrieval adds context. Fine-tuning requires a training process that updates weights.
You mark this yourself. Nothing is graded here.