Data Poisoning Attacks: How Adversaries Corrupt Your ML Models from the Inside

While most ML security discussions focus on evasion attacks — where adversaries craft inputs to fool a deployed model — data poisoning attacks target a more fundamental vulnerability: the training process itself. By injecting carefully crafted malicious samples into training data, an attacker can cause a model to learn systematic errors that persist through deployment. The poisoned model appears to perform normally on standard test sets while containing hidden behaviours that the attacker can trigger at will.

The most dangerous variant is the backdoor attack. An attacker embeds a trigger pattern in a small percentage of training samples while relabelling them to a target class. The model learns to associate the trigger with the target output while maintaining high accuracy on clean data. In deployment, any input containing the trigger pattern produces the attacker-chosen output. For example, a poisoned image classifier might correctly identify all normal images but classify any image containing a specific pixel pattern as benign — enabling an attacker to bypass a security screening system at will.

The challenge of defending against data poisoning is that it exploits the fundamental mechanism by which ML models learn. Standard training procedures are designed to be influenced by training data — that is how learning works. Distinguishing between legitimate data influence and malicious poisoning requires techniques that go beyond standard ML practice: statistical outlier detection on training data, spectral analysis of learned representations, certified robustness bounds against bounded perturbations and provenance tracking for training data sources.

Organisations that rely on ML models for critical decisions — fraud detection, medical diagnosis, autonomous driving, security classification — must treat their training data pipelines with the same security rigour as their production code. This means access controls on training data repositories, integrity verification for data sourced from third parties, anomaly detection on incoming training batches and regular auditing of model behaviour for signs of backdoor activation patterns.