Security for AI: Defender for AI services threat protection, prompt injection and jailbreak alerts, Azure AI Content Safety Prompt Shields

Generative AI applications bring new attack surfaces. A large language model (LLM) follows instructions written in natural language, so an attacker can try to change its behavior with words instead of code. Two terms matter most. A direct prompt injection, often called a jailbreak, is a user prompt crafted to make the model ignore its system instructions or safety rules, for example asking it to role-play a character with no restrictions. An indirect prompt injection hides instructions inside content the model processes, such as a web page, email or document retrieved for grounding, so that the model follows the attacker's hidden text when summarizing it. Other risks include sensitive data leakage, wallet abuse (running up token costs) and misuse of connected tools.