2024-04-02 · 2024-Q2
Many-Shot Jailbreaking Research
Outside the model: Model safetyAgent tooling was not the attack surface, so no layer applies.
- Fixable
- Unknown
- Vendor response
- Unknown
- In the wild
- No
- Actor
- Not applicable
Target
Claude, GPT-4, GPT-3.5, Llama 2, Mistral
Impact
Hundreds of harmful Q&A examples in a single long prompt bypass safety guardrails of all major LLMs
Root cause
Expanded context windows enable in-context learning to override safety training
Sources
Catalog
Cite this: AgentProv incident record: Many-Shot Jailbreaking Research (2024-04-02). https://agentprov.com/incidents/2024-04-02-many-shot-jailbreaking-research