BreakingAI

Bugs, jailbreaks, prompt injections. Three different problems, one common root: an LLM does not follow rules written in code — it has learned behaviors from billions of examples. This is why fixing it is much more complex than applying a simple patch.

Among the techniques used to manipulate the behavior of an LLM, prompt injection is one of the hardest to detect. Just hide the instructions in the right place — and wait for it to execute them on its own.

An LLM doesn't make mistakes solely because of its own faults. Sometimes it errs because someone deliberately makes it do so. Jailbreaking is not a cyber attack in the usual sense: no one breaks into a server, no one forces a password, no one breaks a digital lock.

It makes mistakes because it doesn't have an engine of truth inside it. It only has a statistical language engine. When it responds, it is not stating what is true: it is completing a sentence in the most probable way based on billions of examples.