Inference is the process by which a trained AI model generates a response or prediction from a given input. When a user submits a prompt, uploads a document, or triggers an AI feature, the model runs inference: applying what it learned during training to produce an output. Training is how a model learns; inference is how it performs.
Inference happens in real time, every time the model is used. It is distinct from training, which is a separate, resource-intensive process that happens before the model is deployed. Once a model is trained, inference is what end users and applications actually interact with.
Why This Matters for Technical Writers
Inference is the process happening underneath every AI interaction a user has with a product. Understanding the distinction between training and inference helps technical writers describe AI behavior accurately. Particularly when explaining why a model does not learn from individual user interactions in real time, or why its knowledge has a cutoff date. It also surfaces when documenting latency, cost, or performance characteristics, since inference speed and resource consumption are common product considerations that affect how features are designed and documented.
Common Confusion
Inference is sometimes confused with training, but they are fundamentally different processes. Training shapes what the model knows and how it behaves. It happens once, or periodically, at significant computational cost. Inference is the model putting that knowledge to work in response to a specific input. It happens continuously, every time the model is used. A model does not retrain itself based on what users ask it during inference.
Related Terms
Large language model (LLM), token, probabilistic, hallucination, context window, foundation model, RAG