Inference is what happens when a trained AI model is put to work answering a question. Training is the earlier stage where the model learns. Inference is the everyday use that comes after. When someone on your staff types “Draft a thank-you note to our donors” into a chat assistant and gets a reply, inference produced that reply.
Why it matters
Every prompt your team sends starts inference somewhere. It runs either on the vendor’s servers or on hardware your organization runs itself. That location decides who can see what you type, including any donor names, client details or internal notes pasted into a prompt. If inference runs on a vendor’s machines, your text goes to them, and their policies control what happens to it next. Before you adopt a tool, ask the vendor three questions: where does inference run, who on their side can see prompts, and are prompts kept afterward? If you are thinking about running a model yourself, read our entries on an inference server and model weights next. Data residency is a closely related topic.
Published Updated