Data poisoning

Data poisoning is when someone tampers with the information an AI model learns from or looks things up in, so the model behaves the way that person wants. The bad data can go into the large collection a model is first trained on. It can also go into the smaller set used for fine-tuning, or into the documents a retrieval-augmented generation system searches. For example, picture an assistant that answers staff questions from your shared policy folder. If someone quietly edits one file to say that payments now go to a new bank account, the assistant may repeat that as fact. Security references such as the OWASP Top 10 for LLM Applications and MITRE ATLAS both list this kind of attack.

Why it matters

For a small organization, the most likely target is a shared knowledge base or document folder that an AI assistant answers from. Anyone who can add or change files there can change what the assistant tells your staff. Limit who can edit those folders. Check any answer about money, passwords or contact details against the original document. When you talk to a vendor, ask who can change the data their tool learns from or searches, and how they would notice tampering. Related risks include a backdoor in an AI model and an AI supply chain attack.