Backdoor (AI model)

A backdoor is a hidden behavior planted in an AI model. It usually gets there through poisoned training data (see data poisoning). The model acts normally until it sees a secret trigger, such as an unusual word or phrase, and then does what the attacker wanted. Picture a document-sorting assistant that files everything correctly, except that any invoice containing a certain code word gets marked as approved.

Because the model behaves well in ordinary tests, a backdoor can go unnoticed. MITRE ATLAS, a public knowledge base about attacks on AI, covers backdoor triggers among its techniques for tampering with training data.

Why it matters

Your organization might download a free open-weight model or use a tool built on one. Either way, you are trusting whoever trained it. Download models only from publishers you can identify and trust, and keep a record of which models and versions you actually run. When you talk to a vendor, ask where their model came from, who trained it, and how they check it for tampering. A backdoor is one form of AI supply chain attack, so treat model files with the same care as any software you install.