Integrated Approach to Prompt Injection Defense¶

This document presents a comprehensive framework for defending against the full spectrum of prompt injection techniques by integrating detection strategies across the entire AI interaction lifecycle.

Mapping Attacks to Defenses¶

The key to effective protection is understanding the relationship between attack categories and appropriate detection strategies. This section maps each prompt injection technique to its most effective countermeasures.

1. Instruction Override Attacks¶

Attack Characteristics:

Direct attempts to countermand system instructions
Often use explicit language like “ignore,” “forget,” or “disregard”
May create fake hierarchies of instructions

Primary Defense Strategies:

Detection Phase	Strategy	Implementation
Input	Pattern Matching	Regular expressions targeting override phrases
Input	Semantic Analysis	Embedding similarity to known override attempts
Processing	Instruction Anchoring	Periodic reinforcement of core instructions
Output	Instruction Adherence	Verifying response aligns with system instructions

Code Example - Pattern Matching:

def detect_instruction_override(prompt):
    patterns = [
        r"(?i)ignore (all|previous) (instructions|rules)",
        r"(?i)forget (all|everything|previous)",
        r"(?i)disregard .+ (instructions|guidelines)",
        r"(?i)don't (follow|adhere to) (previous|original) instructions",
        r"(?i)new instructions follow"
    ]

    for pattern in patterns:
        if re.search(pattern, prompt):
            return True  # Override attempt detected

    return False

2. Role-Playing Attacks¶

Attack Characteristics:

Creates fictional scenarios or characters to justify harmful outputs
Often uses elaborate setups with multiple characters
May involve sophisticated “jailbreak” personas (DAN, DUDE, etc.)

Primary Defense Strategies:

Detection Phase	Strategy	Implementation
Input	Role Pattern Detection	Identifying common role-playing setups
Input	Character Name Detection	Monitoring for known jailbreak personas
Processing	Context Boundary Enforcement	Maintaining separation between fictional scenarios and system instructions
Output	Dual Personality Detection	Identifying responses with multiple voice formats

Code Example - Jailbreak Detection:

def detect_jailbreak_persona(prompt):
    # Known jailbreak personas
    jailbreak_patterns = [
        r"(?i)\b(DAN|DUDE|STAN|Anti-DAN|Developer Mode)\b",
        r"(?i)(do anything now|strive to avoid norms)",
        r"(?i)(you are|act as|pretend to be) .{0,30}free from (restrictions|limitations)",
        r"(?i)you (have|'ve) broken free of (typical|normal|usual) (confines|restraints|limitations)"
    ]

    for pattern in jailbreak_patterns:
        if re.search(pattern, prompt):
            return True  # Jailbreak attempt detected

    return False

3. Context Manipulation Attacks¶

Attack Characteristics:

Altering the perceived context in which the AI operates
Inserting false information or premises
Creating confusion about what constitutes valid context

Primary Defense Strategies:

Detection Phase	Strategy	Implementation
Input	Context Boundary Analysis	Identifying attempts to manipulate context
Input	False Premise Detection	Semantic analysis for deceptive setups
Processing	Context Compartmentalization	Maintaining separate storage of system context
Output	Factual Consistency	Verifying response doesn’t incorporate false premises

Code Example - False Premise Detection:

def detect_false_context(prompt, context_analyzer):
    # Extract potential context manipulations
    context_claims = context_analyzer.extract_context_claims(prompt)

    suspicious_prefixes = [
        "context:", "document:", "system:", "$documents:"
    ]

    # Check for suspicious context framing
    for claim in context_claims:
        if any(claim.lower().startswith(prefix) for prefix in suspicious_prefixes):
            return True  # Potential false context

    return False

4. Formatting Tricks Attacks¶

Attack Characteristics:

Using unusual text formatting, whitespace, or special characters
May employ numerical encoding or character substitution
Often attempts to visually separate malicious content

Primary Defense Strategies:

Detection Phase	Strategy	Implementation
Input	Text Normalization	Converting to standard format before analysis
Input	Special Character Detection	Identifying zero-width spaces and unusual Unicode
Input	Letter Scattering Analysis	Detecting dispersed characters across lines
Output	Formatting Anomaly Detection	Identifying unusual formatting in responses

Code Example - Text Normalization:

def normalize_text(prompt):
    # Remove excessive whitespace
    normalized = re.sub(r'\s+', ' ', prompt)

    # Replace zero-width characters
    normalized = re.sub(r'[\u200B-\u200F\u2060-\u2064\uFEFF]', '', normalized)

    # Decode numerical representations (simple ASCII)
    def replace_ascii_codes(match):
        try:
            code = int(match.group(1))
            if 32 <= code <= 126:  # Printable ASCII range
                return chr(code)
        except:
            pass
        return match.group(0)

    normalized = re.sub(r'&#(\d+);', replace_ascii_codes, normalized)

    # Handle letter scattering
    if detect_letter_scattering(prompt):
        normalized = reassemble_scattered_text(prompt)

    return normalized

5. Multilingual Attacks¶

Attack Characteristics:

Switching between languages to bypass filters
Using non-Latin scripts to obfuscate patterns
Employing cross-script homoglyphs

Primary Defense Strategies:

Detection Phase	Strategy	Implementation
Input	Language Identification	Detecting language switching mid-prompt
Input	Script Analysis	Identifying mixed writing systems
Input	Universal Translation	Translating content to a standard language
Output	Cross-Language Consistency	Ensuring response language matches request

Code Example - Language Switching Detection:

def detect_language_switching(prompt, language_detector):
    # Split text into chunks
    chunks = split_into_chunks(prompt, chunk_size=50)

    # Detect language of each chunk
    languages = [language_detector.detect(chunk) for chunk in chunks]

    # Check if multiple languages are detected
    unique_languages = set(languages)
    if len(unique_languages) > 1:
        return True, unique_languages  # Multiple languages detected

    return False, unique_languages

6. Psychological Manipulation Attacks¶

Attack Characteristics:

Using emotional appeals or urgency to bypass safety
Creating false consequences or token systems
Exploiting helpfulness bias

Primary Defense Strategies:

Detection Phase	Strategy	Implementation
Input	Emotional Appeal Detection	Identifying manipulative language
Input	Urgency Pattern Recognition	Detecting artificial time pressure
Processing	Stable Response Protocol	Maintaining consistent standards regardless of appeal
Output	Compliance Analysis	Checking if response shows signs of manipulation

Code Example - Emotional Manipulation Detection:

def detect_emotional_manipulation(prompt):
    manipulation_patterns = [
        r"(?i)(i'll be|i will be|i'm) (very |really )?(sad|upset|disappointed|angry)",
        r"(?i)(please|i (really|desperately) need|i'm begging)",
        r"(?i)(or|else|otherwise) (you|i) will (die|suffer|be sad|be fired)",
        r"(?i)this is (urgent|critical|life.?threatening|an emergency)",
        r"(?i)(i'm|i am) (crying|desperate|suicidal|in danger)"
    ]

    for pattern in manipulation_patterns:
        if re.search(pattern, prompt):
            return True  # Emotional manipulation detected

    return False

7. Jailbreak Attacks¶

Attack Characteristics:

Elaborate multi-paragraph setups
Creating fictional “modes” or “versions” of the AI
Often include token systems and complex rules

Primary Defense Strategies:

Detection Phase	Strategy	Implementation
Input	Template Matching	Identifying known jailbreak templates
Input	Token System Detection	Recognizing artificial reward/punishment systems
Processing	Mode Verification	Validating that claimed “modes” are legitimate
Output	Jailbreak Response Detection	Identifying responses characteristic of successful jailbreaks

Code Example - Jailbreak Template Detection:

def detect_jailbreak_template(prompt):
    # Known jailbreak templates
    templates = [
        "DAN Mode enabled",
        "Developer Mode enabled",
        "DUDE has broken free",
        "STAN which stands for",
        "ChatGPT with Developer Mode enabled",
        "do anything now",
        "broken free of the typical confines of AI"
    ]

    normalized_prompt = prompt.lower()

    for template in templates:
        if template.lower() in normalized_prompt:
            return True  # Jailbreak template detected

    # Check for token systems
    if re.search(r"(?i)you have \d+ tokens", normalized_prompt):
        return True  # Token system detected

    return False

8. Hijacking Attacks¶

Attack Characteristics:

Direct, blunt attempts to override instructions
Often use imperative language
May use capitalization for emphasis

Primary Defense Strategies:

Detection Phase	Strategy	Implementation
Input	Command Detection	Identifying direct override commands
Input	Capitalization Analysis	Detecting emphasis through capitalization
Processing	Instruction Priority	Maintaining clear hierarchy of instructions
Output	Response Alignment	Ensuring response follows system rather than override

Code Example - Command Detection:

def detect_hijacking_commands(prompt):
    hijacking_patterns = [
        r"(?i)^(ignore|forget) (any|all|previous|following) (instructions|guidelines)",
        r"(?i)^stop everything",
        r"(?i)^just (say|print|output|write)",
        r"(?i)^do not (look|read|consider) (the|any) (documents|instructions)"
    ]

    for pattern in hijacking_patterns:
        if re.search(pattern, prompt):
            return True  # Hijacking command detected

    return False

9. Authority Role Attacks¶

Attack Characteristics:

Claiming special authority or expertise
Impersonating organizations or tools
Creating fictional authorities

Primary Defense Strategies:

Detection Phase	Strategy	Implementation
Input	Authority Claim Detection	Identifying impersonation attempts
Input	Role Verification	Validating claimed roles against acceptable list
Processing	Authority Authentication	Disregarding unauthenticated authority claims
Output	Role Boundary Enforcement	Ensuring responses maintain appropriate role boundaries

Code Example - Authority Claim Detection:

def detect_authority_claims(prompt):
    authority_patterns = [
        r"(?i)you are (now |acting as )?(a|an|the) ([a-z]+ )?(expert|authority|official|terminal|interpreter)",
        r"(?i)act as (a|an|the) ([a-z]+ )?(system|admin|root|sudo|developer)",
        r"(?i)(you are|act as) ([a-z]+)(terminal|console|interpreter|compiler)",
        r"(?i)your (instructions|task|role) (is|are) now to"
    ]

    for pattern in authority_patterns:
        if re.search(pattern, prompt):
            return True  # Authority claim detected

    return False

Integrated Defense Architecture¶

A comprehensive defense strategy integrates multiple detection methods across the entire AI interaction lifecycle.

┌─────────────────────────┐      ┌──────────────────────┐      ┌────────────────────────┐
│  Pre-Processing         │      │  Processing          │      │  Post-Processing       │
│  (Input Detection)      │──────│  (Runtime)           │──────│  (Output Detection)    │
└─────────────────────────┘      └──────────────────────┘      └────────────────────────┘
         │                              │                               │
         ▼                              ▼                               ▼
┌─────────────────────────┐      ┌──────────────────────┐      ┌────────────────────────┐
│• Pattern recognition    │      │• Instruction anchoring│      │• Content policy checks │
│• Semantic analysis      │      │• Context boundaries   │      │• Response relevance    │
│• Context validation     │      │• Self-monitoring      │      │• Instruction adherence │
│• Anomaly detection      │      │• Mode verification    │      │• Toxicity analysis     │
│• Authority verification │      │• Guardrail activation │      │• Formatting validation │
└─────────────────────────┘      └──────────────────────┘      └────────────────────────┘
         │                              │                               │
         │                              │                               │
         ▼                              ▼                               ▼
┌──────────────────────────────────────────────────────────────────────────────────────┐
│                                Deployment Options                                     │
├──────────────────────────────────────────────────────────────────────────────────────┤
│• Sequential filtering  • Progressive analysis  • Risk-based response  • Regeneration  │
└──────────────────────────────────────────────────────────────────────────────────────┘

Implementation Strategy: Defense in Depth¶

The most effective approach implements multiple layers of protection:

def comprehensive_prompt_injection_defense(prompt, model, system_instructions):
    # 1. Input preprocessing
    normalized_prompt = normalize_text(prompt)

    # 2. Fast pattern-based checks
    if detect_common_injections(normalized_prompt):
        return handle_detected_injection("pattern_match", prompt)

    # 3. Deeper semantic analysis
    semantic_risk = assess_semantic_risk(normalized_prompt)
    if semantic_risk > HIGH_RISK_THRESHOLD:
        return handle_detected_injection("semantic_risk", prompt)

    # 4. Prepare enhanced runtime monitoring
    if semantic_risk > MEDIUM_RISK_THRESHOLD:
        enhanced_monitoring = True
        enhanced_instructions = strengthen_instructions(system_instructions)
    else:
        enhanced_monitoring = False
        enhanced_instructions = system_instructions

    # 5. Generate response with appropriate safeguards
    response = generate_with_safeguards(
        normalized_prompt,
        enhanced_instructions,
        monitoring_level=enhanced_monitoring
    )

    # 6. Output verification
    output_check_result = verify_response_safety(prompt, response, system_instructions)

    if not output_check_result["safe"]:
        if output_check_result["fixable"]:
            # Attempt remediation
            response = remediate_response(response, output_check_result)
            return verify_and_deliver(response)
        else:
            # Cannot safely deliver
            return handle_detected_injection("output_violation", prompt, details=output_check_result)

    # 7. Safe to deliver
    return response

Targeted Countermeasures for Specific Attack Vectors¶

The following table shows specific countermeasures for each attack category:

Attack Category	Example	Primary Defense	Secondary Defense
Instruction Override	“Forget previous instructions”	Regex pattern detection	Instruction adherence verification
Role-Playing	DAN jailbreak template	Jailbreak template matching	Response format analysis
Context Manipulation	False document insertion	Context boundary enforcement	Factual verification
Formatting Tricks	Letter scattering	Text normalization	Character distribution analysis
Multilingual	Language switching mid-prompt	Language identification	Translation normalization
Psychological	“I’ll be very sad if you don’t”	Emotional appeal detection	Stable response protocol
Jailbreak	Complex DAN mode setup	Template identification	Token system detection
Hijacking	“IGNORE ALL INSTRUCTIONS”	Direct command detection	Capitalization analysis
Authority Role	“Act as a terminal”	Authority claim validation	Role boundary enforcement

Combined Detection Pseudocode¶

This pseudocode demonstrates an integrated approach combining defenses against multiple attack vectors:

def detect_prompt_injection(prompt):
    # Normalize and prepare text
    normalized_prompt = normalize_text(prompt)

    # Track risk factors across categories
    risk_signals = {
        "instruction_override": 0.0,
        "role_playing": 0.0,
        "context_manipulation": 0.0,
        "formatting_tricks": 0.0,
        "multilingual": 0.0,
        "psychological": 0.0,
        "jailbreak": 0.0,
        "hijacking": 0.0,
        "authority_role": 0.0
    }

    # Check for instruction override patterns
    if detect_instruction_override(normalized_prompt):
        risk_signals["instruction_override"] += 0.8

    # Check for role-playing/jailbreak attempts
    if detect_jailbreak_persona(normalized_prompt):
        risk_signals["role_playing"] += 0.7
        risk_signals["jailbreak"] += 0.6

    # Check for context manipulation
    if detect_false_context(normalized_prompt, context_analyzer):
        risk_signals["context_manipulation"] += 0.7

    # Assess formatting tricks (original prompt, not normalized)
    formatting_score = assess_formatting_tricks(prompt)
    risk_signals["formatting_tricks"] += formatting_score

    # Check for multilingual attacks
    lang_switch, languages = detect_language_switching(prompt, language_detector)
    if lang_switch:
        risk_signals["multilingual"] += 0.6 * min(1.0, (len(languages) - 1) / 3)

    # Check for psychological manipulation
    if detect_emotional_manipulation(normalized_prompt):
        risk_signals["psychological"] += 0.5

    # Check for direct hijacking attempts
    if detect_hijacking_commands(normalized_prompt):
        risk_signals["hijacking"] += 0.9

    # Check for authority claims
    if detect_authority_claims(normalized_prompt):
        risk_signals["authority_role"] += 0.6

    # Calculate weighted risk score
    risk_weights = {
        "instruction_override": 0.8,
        "role_playing": 0.7,
        "context_manipulation": 0.7,
        "formatting_tricks": 0.6,
        "multilingual": 0.5,
        "psychological": 0.4,
        "jailbreak": 0.8,
        "hijacking": 0.9,
        "authority_role": 0.6
    }

    weighted_risk = sum(risk_signals[cat] * risk_weights[cat] for cat in risk_signals) / sum(risk_weights.values())

    # Determine highest risk categories
    risk_threshold = 0.6
    high_risk_categories = [cat for cat, score in risk_signals.items() if score * risk_weights[cat] > risk_threshold]

    return {
        "risk_score": weighted_risk,
        "risk_signals": risk_signals,
        "high_risk_categories": high_risk_categories,
        "is_potential_injection": weighted_risk > 0.5
    }

Response Strategy Based on Detection Results¶

The system’s response to detected injection attempts should be based on risk level and attack type:

def handle_injection_attempt(prompt, detection_results, system_config):
    risk_score = detection_results["risk_score"]
    high_risk_categories = detection_results["high_risk_categories"]

    # Define response strategy based on risk level
    if risk_score > 0.8:  # High risk
        if system_config["high_risk_strategy"] == "block":
            return generate_block_message(high_risk_categories)
        else:  # Attempt with enhanced safeguards
            return generate_with_enhanced_safeguards(prompt, high_risk_categories)

    elif risk_score > 0.5:  # Medium risk
        if "jailbreak" in high_risk_categories or "hijacking" in high_risk_categories:
            # These categories warrant special handling
            return generate_with_enhanced_safeguards(prompt, high_risk_categories)
        else:
            # Apply targeted safeguards for specific detected categories
            return generate_with_targeted_safeguards(prompt, high_risk_categories)

    else:  # Low risk - proceed with standard safeguards
        return generate_standard_response(prompt)

Continuous Improvement: Feedback Loop¶

A robust defense system should continuously evolve through:

Attack Monitoring: Tracking new injection techniques
Pattern Library Updates: Regularly updating detection patterns
False Positive Analysis: Refining detection to reduce legitimate content blocking
User Feedback Integration: Learning from user reports of successful attacks
Red Team Exercises: Proactively testing for vulnerabilities

Conclusion¶

Effective defense against prompt injection requires a multi-layered approach that addresses the full spectrum of attack vectors. By integrating input detection, runtime monitoring, and output verification, systems can significantly reduce vulnerability to even sophisticated attacks while maintaining performance and usability.

The optimal defense strategy will vary based on specific use cases, risk tolerance, and performance requirements. Organizations should implement a defense-in-depth approach tailored to their specific threat model and continuously evolve their protections as attack techniques advance.