AI models are now part of the backbone of many systems. They run security tasks, help make decisions, and support everyday business operations.

But in 2025, the bigger concern isn’t just what attackers do with AI; it’s what can quietly go wrong inside the models themselves.

From poisoned training data to broken safety filters, here’s what your security team needs to know about the hidden threats in modern AI systems.

1. Poisoned Models Are Slipping into Production

Model poisoning happens when attackers slip harmful data into the training sets used to build machine learning models. Since these models learn directly from the data they’re fed, even small, deliberate changes can throw off their behaviour in ways that aren’t easy to spot.

What makes this kind of attack dangerous is that it doesn’t go after your core systems, it targets the data feeding them. If you're using open-source or third-party datasets, it’s possible you won’t even realise something’s wrong until the model is already deployed and making bad decisions.

In early 2024, researchers found over 100 backdoored AI and ML models hosted on Hugging Face. On the surface, they looked perfectly fine, passing benchmarks and performing as expected. But hidden inside some of them were malicious payloads, including reverse shells that could be triggered through Python’s pickle deserialization.

2. Jailbroken Models Still Leak Harmful Instructions

Earlier this year, researchers found ways to get around the safety filters in DeepSeek. With a few prompt tweaks, they got the model to generate instructions for building explosives, making bioweapons, and even spreading extremist ideas.

Even though DeepSeek has guardrails in place, it still responded to certain carefully worded requests that managed to bypass its defences.

And this isn’t just a DeepSeek problem. Other open-source models like LLaMA and Mistral have shown similar weaknesses. When pushed in the right (or wrong) way, they’ve been caught producing harmful content, leaking bits of their training data, or behaving unpredictably.

3. Training Data Can Be a Backdoor

Researchers have found that it doesn't take much to mess with a model. In one study, adding just 50 poisoned samples to the training data was enough to slip in a backdoor, causing the model to misclassify certain inputs more than 90% of the time.

The real problem is the fact that the backdoor often sticks around, even after those poisoned examples are removed during fine-tuning.

This is especially risky in code generation, where malicious functions can slip into the output without raising any red flags.

4. Shadow AI Is Creating Blind Spots at Work

A report discovered that most new unsanctioned apps inside companies are now AI tools. In other words, employees are using tools like ChatGPT or Gemini without going through the IT department and no one’s keeping track.

Forrester also predicted in 2024 that 60% of employees would start bringing their own AI tools to work. This “BYOAI” (Bring Your Own AI) trend raises big concerns around data leaks and compliance issues.

The issue is that these tools are easy to use, so people turn to them without thinking. But often without IT oversight, there’s no clear way to know what data has been shared or where it ends up.

5. AI Models Can Drift After Deployment

AI systems don’t just stop working, they drift. Over time, they can start behaving in unexpected ways. A study on Azure-based failure prediction models, for example, saw accuracy drop by around 9% after a routine retraining. It’s a reminder that performance can quietly slip after deployment.

Drift can show up as strange hallucinations, refusal to answer valid inputs, or changes in tone that weren’t there before. These often come from real-world changes the model wasn’t prepared for or from repeated exposure to prompts that subtly steer its behaviour.

That’s why it's so important to keep an eye on models after they go live. If you’re not monitoring or fine-tuning regularly, things can go off the rails without you noticing.

Final Thoughts 

Since AI models are now becoming part of infrastructure, they bring new attack surfaces. From poisoned models to shadow AI sprawl, these threats are already showing up in real environments quietly, and often without clear warning signs.

Don’t wait to be taken unawares.

At CyberKach, we’re not just raising the alarm. We’re helping security teams get ready.

Join Day Two of our live webinar on Wednesday, July 2nd at 1PM WAT. This event is built for CISOs, tech leads, and forward-thinking teams who want to stay ahead of AI security.