LLMOps
disciplineLLM Operations
The discipline of running language-model applications in production: versioning prompts and retrieval, evaluating output that is never the same twice, tracing every step and keeping the cost per token under control.
LLMOps is MLOps moved to ground where the thing you ship is no longer a trained model but a model plus a prompt, plus the context you feed it, plus the tools it can call and the guardrails that fence it in. Three things change: what you version (prompts and retrieval, not weights), where the cost sits (inference, not training) and how you judge quality (a suite of cases with expected outcomes, not a single closed accuracy score). In the split the market has settled on, MLOps runs predictions, LLMOps runs generated responses and AgentOps runs actions. AI Operations sits a layer above all three: which process gets automated, who answers for the result and how many human hours it actually saves. A team can have flawless LLMOps and still have no idea whether the system is worth anything to the business.
How it differs from
- MLOps
- MLOps runs trained models and their predictions. LLMOps runs a system whose main artifact is the prompt plus the context, and whose quality is probabilistic.
- AgentOps
- AgentOps adds the steps, tool calls and actions of an agent. LLMOps stops at the model call and its quality.
- AI Operations
- LLMOps is the technical layer. AI Operations is the business layer: which process, who is accountable, what it saves.
Examples
- You swap the model, one case gets better and three get worse: with no versioned evals, your customer finds out before you do.
- The successor model is chattier: quality holds, cost per conversation jumps, and no alert fires.
FAQ
- Are LLMOps and AI Operations the same thing?
- No. LLMOps is the technical operation of language-model applications: prompts, context, evaluation, traces and cost. AI Operations contains that but starts earlier and ends later: which business process gets automated, how the workflow is redesigned, who governs the agent and what saving you can defend in front of a board.
- Do I need LLMOps if I only call a vendor API?
- Yes. Calling an API does not spare you from versioning prompts, measuring quality against real cases, watching cost per interaction or planning the migration for the day the vendor retires the model. What you skip is training, not operating.