CTF AI/ML

Quick reference for AI/ML CTF challenges. Each technique has a one-liner here; see supporting files for full details.

Prerequisites

Python packages (all platforms):
CODEBLOCK0

Linux (apt):
CODEBLOCK1

macOS (Homebrew):
CODEBLOCK2

Additional Resources

• - model-attacks.md - Model weight perturbation negation, model inversion via gradient descent, neural network encoder collision, LoRA adapter weight merging, model extraction via query API, membership inference attack
• adversarial-ml.md - Adversarial example generation (FGSM, PGD, C&W), adversarial patch generation, evasion attacks on ML classifiers, data poisoning, backdoor detection in neural networks
• llm-attacks.md - Prompt injection (direct/indirect), LLM jailbreaking, token smuggling, context window manipulation, tool use exploitation

When to Pivot

• - If the challenge becomes pure math, lattice reduction, or number theory with no ML component, switch to /ctf-crypto.
• If the task is reverse engineering a compiled ML model binary (ONNX loader, TensorRT engine, custom inference binary), switch to /ctf-reverse.
• If the challenge is a game or puzzle that merely uses ML as a wrapper (e.g., Python jail inside a chatbot), switch to /ctf-misc.

Quick Start Commands

CODEBLOCK3

Model Weight Analysis

• - Weight perturbation negation: Fine-tuned model suppresses behavior; recover by computing 2*W_orig - W_chal to negate the fine-tuning delta. See model-attacks.md.
• LoRA adapter merging: Merge LoRA adapter W_base + alpha * (B @ A) and inspect activations or generate output with merged weights. See model-attacks.md.
• Model inversion: Optimize random input tensor to minimize distance between model output and known target via gradient descent. See model-attacks.md.
• Neural network collision: Find two distinct inputs that produce identical encoder output via joint optimization. See model-attacks.md.

Adversarial Examples

• - FGSM: Single-step attack: x_adv = x + eps * sign(grad_x(loss)). Fast but less effective than iterative methods. See adversarial-ml.md.
• PGD: Iterative FGSM with projection back to epsilon-ball each step. Standard benchmark attack. See adversarial-ml.md.
• C&W: Optimization-based attack that minimizes perturbation norm while achieving misclassification. See adversarial-ml.md.
• Adversarial patches: Physical-world patches that cause misclassification when placed in a scene. See adversarial-ml.md.
• Data poisoning: Injecting backdoor triggers into training data so model learns attacker-chosen behavior. See adversarial-ml.md.

LLM Attacks

• - Prompt injection: Overriding system instructions via user input; both direct injection and indirect via retrieved documents. See llm-attacks.md.
• Jailbreaking: Bypassing safety filters via DAN, role play, encoding tricks, multi-turn escalation. See llm-attacks.md.
• Token smuggling: Exploiting tokenizer splits so filtered words pass through as subword tokens. See llm-attacks.md.
• Tool use exploitation: Abusing function calling in LLM agents to execute unintended actions. See llm-attacks.md.

Model Extraction & Inference

• - Model extraction: Querying a model API with crafted inputs to reconstruct its parameters or decision boundary. See model-attacks.md.
• Membership inference: Determining whether a specific sample was in the training data based on confidence score distribution. See model-attacks.md.

Gradient-Based Techniques

• - Gradient-based input recovery: Using model gradients to reconstruct private training data from shared gradients (federated learning attacks). See model-attacks.md.
• Activation maximization: Optimizing input to maximize a specific neuron's activation, revealing what the network has learned.