The Trojan Horses of AI

What your AI system trusts could be its downfall

Training data poisoning and insecure plug-in use

Following my previous post on System PLeaks, I wanted to talk about one of the key points that ensure AI systems provide accuracy and the trust that organizations and users need to have confidence in its utilization.

As we race to integrate LLMs and AI-driven workflows into enterprise systems, the threat landscape is evolving faster than most organizations can patch. Two of the most insidious - and often underestimated - attack vectors are:

1. Training Data Poisoning

We have all heard the saying "garbage in, garbage out", but what we also need to be aware of when building or employing these systems is "garbage in, compromise out".

Malicious actors are quietly seeding harmful, misleading or biased data into public datasets - particularly those scraped at scale for LLM training.

Why does it matter? Because:

• Polluted training data can lead to:

  • Model bias and toxic outputs

  • Evasion of safety and policy guardrails

  • Hallucinated trust in dangerous concepts

• Poisoning isn't just theoretical - it's happening in open-source, academic and even web-scraped data.

Bottom line - if your model can be tricked into "learning" the wrong things, it can be manipulated at scale.

2. Insecure Plug-in Use: "The plug-in that breaks your AI"

Modern LLMs now rely on plug-ins and API connectors to perform actions and fetch data. But with great extensibility comes a massive increase in the attack surface.

Where it goes wrong:

• Plug-ins that lack isolation and can expose your full AI environment.

• A single vulnerable or misconfigured plug-in can be exploited to exfiltrate data, hijack workflows or inject malicious prompts.

Key Takeaway

Plug-ins = convenience + risk

Every plug-in added expands your LLMs attack surface. If you are not securing your plug-in stack, you are leaving the backdoor wide open. As with any Trojan Horse, the danger lies in what you invite in.

Pillar Group Advisory Tip

Secure your AI like you secure your software:

• Adopt an AI SBOM (Software Bill of Materials). Know which datasets, pre-trained models and plug-ins your AI stack depends on.

• Implement a plug-in permission model. Not all plug-ins need equal access. Think "least privilege", just like in identity and access management.

• Consider the use of a Purple Team. A purple team combines the capabilities of red teams (attackers) and blue teams (defenders) working together to: validate security controls, enhance detection and response, and embed security-by-design in Agentic AI and Zero Trust architecture.

Take control of your AI ecosystem now

1. Start with an internal plug-in review

2. Build your AI SBOM

3. Lock down permissions

Your future systems won't just be powered by AI - they will be defined by how securely you build and extend them today.

We have the team and the tools to assist with this - reach out today!