CTF Cryptography

Quick reference for crypto CTF challenges. Each technique has a one-liner here; see supporting files for full details with code.

Prerequisites

Python packages (all platforms):
CODEBLOCK0

Linux (apt):
CODEBLOCK1

macOS (Homebrew):
CODEBLOCK2

Manual install:

• - SageMath — Linux: apt install sagemath, macOS: INLINECODE1
• RsaCtfTool — git clone https://github.com/RsaCtfTool/RsaCtfTool (automated RSA attacks)

Note: gmpy2 requires libgmp — Linux: apt install libgmp-dev, macOS: brew install gmp.

Additional Resources

• - classic-ciphers.md - Classic ciphers: Vigenere (+ Kasiski examination), Atbash, substitution wheels, XOR variants (+ multi-byte frequency analysis), deterministic OTP, cascade XOR, book cipher, OTP key reuse / many-time pad, variable-length homophonic substitution, grid permutation cipher keyspace reduction, image-based Caesar shift ciphers, XOR key recovery via file format headers
• modern-ciphers.md - Modern cipher attacks: AES (CFB-8, ECB leakage), CBC-MAC/OFB-MAC, padding oracle, S-box collisions, GF(2) elimination, LCG partial output recovery, affine cipher over composite modulus, AES-GCM with derived keys, AES-GCM nonce reuse (forbidden attack), Ascon-like reduced-round differential cryptanalysis, custom linear MAC forgery, CBC padding oracle (full block decryption), Bleichenbacher RSA PKCS#1 v1.5 padding oracle (ROBOT), birthday attack / meet-in-the-middle, CRC32 collision signature forgery, AES key recovery via byte-by-byte zeroing oracle
• modern-ciphers-2.md - Modern cipher attacks (continued): Blum-Goldwasser bit-extension oracle, hash length extension, compression oracle (CRIME-style), hash function time reversal via cycle detection, OFB mode invertible RNG backward decryption, weak key derivation via public key hash XOR, HMAC-CRC linearity attack, DES weak keys in OFB mode, SRP protocol bypass, modified AES S-Box brute-force, square attack on reduced-round AES, AES-ECB byte-at-a-time chosen plaintext, AES-ECB cut-and-paste block manipulation, AES-CBC IV bit-flip auth bypass, Rabin LSB parity oracle, PBKDF2 pre-hash bypass, MD5 multi-collision via fastcol, custom hash state reversal, CRC32 brute-force for small payloads, noisy RSA LSB oracle error correction, sponge hash MITM collision, CBC IV forgery + block truncation, padding oracle to CBC bitflip RCE, SPN S-box intersection attack
• stream-ciphers.md - Stream cipher attacks: LFSR (Berlekamp-Massey, correlation attack, known-plaintext, Galois vs Fibonacci, Galois tap recovery via autocorrelation), RC4 second-byte bias, XOR consecutive byte correlation
• rsa-attacks.md - RSA attacks: small e (cube root), common modulus, Wiener's, Pollard's p-1, Hastad's broadcast, Hastad with linear padding (Coppersmith), Fermat/consecutive primes, multi-prime, restricted-digit, Coppersmith structured primes, Manger oracle, polynomial hash
• rsa-attacks-2.md - RSA attacks (specialized): RSA p=q validation bypass, cube root CRT gcd(e,phi)>1, factoring from phi(n) multiple, multiplicative homomorphism signature forgery, weak keygen via base representation, RSA with gcd(e,phi)>1 exponent reduction, batch GCD shared prime factoring, partial key recovery from dp/dq/qinv, RSA-CRT fault attack, homomorphic decryption oracle bypass, small prime CRT decomposition, Montgomery reduction timing attack, Bleichenbacher low-exponent signature forgery
• ecc-attacks.md - ECC attacks: small subgroup, invalid curve, Smart's attack (anomalous, with Sage code), fault injection, clock group DLP, Pohlig-Hellman, ECDSA nonce reuse, Ed25519 torsion side channel, DSA nonce reuse, DSA key recovery via MD5 collision on k-generation
• zkp-and-advanced.md - ZKP/graph 3-coloring, Z3 solver guide, garbled circuits, Shamir SSS, bigram constraint solving, race conditions, Groth16 broken setup, DV-SNARG forgery, KZG pairing oracle for permutation recovery, Shamir SSS reused polynomial coefficients
• prng.md - PRNG attacks (MT19937, MT float recovery via GF(2) magic matrix for token prediction, LCG, GF(2) matrix PRNG, V8 XorShift128+ Math.random state recovery via Z3, middle-square, deterministic RNG hill climbing, random-mode oracle, time-based seeds, C srand/rand synchronization via ctypes, password cracking, logistic map chaotic PRNG)
• historical.md - Historical ciphers (Lorenz SZ40/42, book cipher implementation)
• advanced-math.md - Advanced mathematical attacks (isogenies, Pohlig-Hellman, baby-step giant-step (BSGS) for general DLP, LLL, Merkle-Hellman knapsack via LLL, Coppersmith, quaternion RSA, GF(2)[x] CRT, S-box collision code, LWE lattice CVP attack, affine cipher over non-prime modulus, introspective CRC via GF(2) linear algebra)
• lattice-and-lwe.md - Lattice attack triage and workflow: LLL/BKZ/Babai, HNP from partial or biased nonces, truncated LCG state recovery, LWE embedding and CVP, Ring-LWE / Module-LWE recognition, orthogonal lattices, subset sum / knapsack, and common failure modes
• exotic-crypto.md - Exotic algebraic structures (braid group DH / Alexander polynomial, monotone function inversion, tropical semiring residuation, Paillier cryptosystem, Hamming code helical interleaving, ElGamal universal re-encryption, FPE Feistel brute-force, icosahedral symmetry group cipher, Goldwasser-Micali replication oracle, BB-84 QKD MITM attack)

When to Pivot

• - If the real blocker is understanding a binary, obfuscated client, or weird VM, switch to /ctf-reverse.
• If the challenge is mostly packet carving, disk recovery, or stego extraction before any decryption starts, switch to /ctf-forensics.
• If the task is just implementing an exploit against a vulnerable network service after the crypto part is solved, switch to /ctf-pwn or /ctf-web.
• If the crypto challenge involves adversarial ML, model extraction, or neural-network-based ciphers, switch to /ctf-ai-ml.
• If the challenge is really an encoding puzzle, esoteric cipher, or polyglot trick rather than true cryptanalysis, switch to /ctf-misc.

Quick Start Commands

CODEBLOCK3

Classic Ciphers

• - Caesar: Frequency analysis or brute force 26 keys
• Vigenere: Known plaintext attack with flag format prefix; derive key from (ct - pt) mod 26. Kasiski examination for unknown key length (GCD of repeated sequence distances)
• Atbash: A<->Z substitution; look for "Abashed" hints in challenge name
• Substitution wheel: Brute force all rotations of inner/outer alphabet mapping
• Multi-byte XOR: Split ciphertext by key position, frequency-analyze each column independently; score by English letter frequency (space = 0x20)
• Cascade XOR: Brute force first byte (256 attempts), rest follows deterministically
• XOR rotation (power-of-2): Even/odd bits never mix; only 4 candidate states
• Weak XOR verification: Single-byte XOR check has 1/256 pass rate; brute force with enough budget
• Deterministic OTP: Known-plaintext XOR to recover keystream; match load-balanced backends
• OTP key reuse (many-time pad): C1 XOR C2 XOR known_P = unknown_P; crib dragging when no plaintext known
• Homophonic (variable-length): Multi-character ciphertext groups map to single plaintext chars. Find n-grams with identical sub-n-gram frequencies, replace with symbols, solve as monoalphabetic. See classic-ciphers.md.
• Grid permutation cipher: 5x5 grid with independent row/column permutations collapses keyspace to 5! x 5! = 14,400; brute-force in milliseconds. See classic-ciphers.md.
• Image-based Caesar shift: Pixel rows/columns shifted by per-strip offsets; compare original vs shifted image to extract ASCII-encoded flag from shift amounts. See classic-ciphers.md.
• Polybius square cipher: 5x5 grid maps letter pairs to plaintext; digits/coordinates encode positions. See classic-ciphers.md.
• XOR key recovery via file format headers: File claims to be PDF/PNG/ZIP but file reports "data". XOR first bytes against expected magic bytes to derive repeating key; extend using trailer structures (%%EOF, IEND marker). See classic-ciphers.md.

See classic-ciphers.md for full code examples.

Modern Cipher Attacks

• - AES-ECB: Block shuffling, byte-at-a-time chosen-plaintext suffix recovery (256 queries per byte, tool: FeatherDuster ecb_cpa_decrypt); image ECB preserves visual patterns. ECB cut-and-paste: splice ciphertext blocks to forge JSON fields (e.g., is_admin: true). See modern-ciphers-2.md.
• AES-CBC: Bit flipping to change plaintext; padding oracle for decryption without key. IV bit-flip: flip specific bits in the IV to change first plaintext block (requires no MAC). See modern-ciphers-2.md.
• CBC IV forgery + block truncation: XOR IV bytes to change decrypted block 0; strip trailing ciphertext blocks (no length integrity in CBC). Forges authenticated tokens when MAC is embedded in the ciphertext. See modern-ciphers-2.md.
• Padding oracle to CBC bitflip RCE: Chain padding oracle (recover plaintext) with CBC bitflipping (inject shell metacharacters) for command injection via encrypted parameters. See modern-ciphers-2.md.
• AES-CFB-8: Static IV with 8-bit feedback allows state reconstruction after 16 known bytes
• CBC-MAC/OFB-MAC: XOR keystream for signature forgery: INLINECODE18
• S-box collisions: Non-permutation S-box (len(set(sbox)) < 256) enables 4,097-query key recovery
• GF(2) elimination: Linear hash functions (XOR + rotations) solved via Gaussian elimination over GF(2)
• Padding oracle: Byte-by-byte decryption by modifying previous block and testing padding validity
• LFSR stream ciphers: Berlekamp-Massey recovers feedback polynomial from 2L keystream bits; correlation attack breaks combined generators with biased combining functions
• Galois LFSR tap recovery: XOR known file header (PNG/PDF/ZIP) with ciphertext to get keystream; split into N-bit windows, compute (state >> 1) XOR next_state for LSB=1 transitions to directly recover tap mask. Autocorrelation sliding finds correct length. See stream-ciphers.md.
• OFB with invertible RNG: Known plaintext in any block leaks RNG state; if state transition is bijective, run RNG backwards to decrypt all blocks. See modern-ciphers-2.md.
• Weak key derivation (public key hash XOR): AES key derived from SHA256(public_key) XOR seed is fully recoverable without private key; "hybrid" RSA+AES provides no security. See modern-ciphers-2.md.
• HMAC-CRC linearity: CRC is linear over GF(2), so HMAC-CRC key is recoverable from a single message-MAC pair via polynomial arithmetic. See modern-ciphers-2.md.
• DES weak keys in OFB: 4 DES weak keys make encryption self-inverse; OFB keystream cycles with period 2, reducing to 16-byte repeating XOR. See modern-ciphers-2.md.
• Square attack (reduced-round AES): 4-round AES broken by integral cryptanalysis: 256-plaintext lambda set, guess last round key bytes via XOR-sum = 0 distinguisher. See modern-ciphers-2.md.
• AES-GCM nonce reuse (forbidden attack): Same nonce = CTR keystream reuse + GHASH authentication key recovery via polynomial factoring over GF(2^128). Tool: nonce-disrespect. See modern-ciphers.md.
• SRP protocol bypass: Send A = 0 or A = n to force shared secret to 0, bypassing password verification entirely. See modern-ciphers-2.md.
• Modified AES S-Box brute force: Custom S-Box with only 16 unique outputs reduces key entropy; brute-force feasible key bytes per round. See modern-ciphers-2.md.
• Rabin LSB parity oracle: Rabin ciphertext c = m^2 mod n with LSB oracle enables binary search plaintext recovery in log2(n) queries via multiplicative homomorphism (c * 4 mod n doubles plaintext). See modern-ciphers-2.md.
• Noisy RSA LSB oracle error correction: When LSB oracle has sporadic errors, run standard attack then inspect output charset. Flip oracle results at error positions to correct remaining decryption. See modern-ciphers-2.md.
• PBKDF2 pre-hash bypass: HMAC pre-hashes keys > 64 bytes (SHA-1/SHA-256 block size). Login with SHA1(password) instead of password when original exceeds 64 bytes. See modern-ciphers-2.md.
• MD5 multi-collision (fastcol): Chain fastcol runs to produce 2^k files with identical MD5. Merkle-Damgard composition: collisions propagate through appended suffixes. See modern-ciphers-2.md.
• Custom hash state reversal: When iterative hash leaks intermediate states, isolate per-block hash values by inverting the state update equation, then brute-force each 4-byte block independently. See modern-ciphers-2.md.
• CRC32 brute-force (small payloads): ZIP CRC32 headers are unencrypted; brute-force content of small files (≤ 6 bytes) by checking all printable strings against stored CRC32. See modern-ciphers-2.md.

See modern-ciphers.md and modern-ciphers-2.md for full code examples.

RSA Attacks

• - Small e with small message: Take eth root
• Common modulus: Extended GCD attack
• Wiener's attack: Small d
• Fermat factorization: p and q close together
• Pollard's p-1: Smooth p-1
• Hastad's broadcast: Same message, multiple e=3 encryptions
• Consecutive primes: q = nextprime(p); find first prime below sqrt(N)
• Multi-prime: Factor N with sympy; compute phi from all factors
• Restricted-digit primes: Digit-by-digit factoring from LSB with modular pruning
• Coppersmith structured primes: Partially known prime; f.small_roots() in SageMath
• Manger oracle (simplified): Phase 1 doubling + phase 2 binary search; ~128 queries for 64-bit key
• Manger on RSA-OAEP (timing): Python or short-circuit skips expensive PBKDF2 when Y != 0, creating fast/slow timing oracle. Full 3-step attack (~1024 iterations for 1024-bit RSA). Calibrate timing bounds with known-fast/known-slow samples.
• Polynomial hash (trivial root): g(0) = 0 for polynomial hash; craft suffix for msg = 0 (mod P), signature = 0
• Polynomial CRT in GF(2)[x]: Collect ~20 remainders r = flag mod f, filter coprime, CRT combine
• Affine over composite modulus: CRT in each prime factor field; Gauss-Jordan per prime
• RSA p=q validation bypass: Set p=q so server computes wrong phi=(p-1)^2 instead of p*(p-1); test decryption fails, leaking ciphertext
• RSA cube root CRT (gcd(e,phi)>1): When all primes ≡ 1 mod e, compute eth roots per-prime via nthroot_mod, enumerate CRT combinations (3^k feasible for small k)
• Factoring from phi(n) multiple: Any multiple of phi(n) (e.g., e*d-1) enables factoring via Miller-Rabin square root technique; succeeds with prob ≥ 1/2 per attempt
• Weak keygen via base representation: Primes p = kp*B + tp with small kp create mixed-radix structure in n; brute-force kp*kq (2^24) to factor
• RSA with gcd(e,phi)>1 (exponent reduction): Reduce e' = e/g, compute d' = e'^(-1) mod phi, partial decrypt to m^g, then take g-th root over integers
• RSA partial key recovery (dp/dq/qinv): CRT exponents from partial PEM leak allow O(e) prime recovery: iterate k, check if (dp*e-1)/k+1 is prime. See rsa-attacks-2.md.
• RSA-CRT fault attack: Single faulty CRT signature leaks factor via gcd(s^e - m, n) (Bellcore attack). See rsa-attacks-2.md.
• RSA homomorphic decryption bypass: Multiplicative homomorphism lets you decrypt c by querying oracle with c * r^e mod n, then dividing result by r. See rsa-attacks-2.md.
• RSA small prime CRT decomposition: When n has many small prime factors, factor with trial division, solve m mod p_i per prime, CRT combine. See rsa-attacks-2.md.
• Hastad broadcast with linear padding (Coppersmith): When each of e recipients applies a known affine transform a_i*m+b_i before encryption, CRT + Coppersmith smallroots recovers m. See rsa-attacks.md.
• RSA Montgomery reduction timing attack: Leaked extra-subtraction counts in Montgomery multiplication reveal private key bits MSB-to-LSB via statistical correlation. See rsa-attacks-2.md.
• Bleichenbacher low-exponent signature forgery: With e=3, forge PKCS#1 v1.5 signatures by computing cube root of a value with correct padding prefix; trailing garbage absorbs the remainder. See rsa-attacks-2.md.

See rsa-attacks.md and advanced-math.md for full code examples.

Elliptic Curve Attacks

• - Small subgroup: Check curve order for small factors; Pohlig-Hellman + CRT
• Invalid curve: Send points on weaker curves if validation missing
• Singular curves: Discriminant = 0; DLP maps to additive/multiplicative group
• Smart's attack: Anomalous curves (order = p); p-adic lift solves DLP in O(1)
• Baby-step giant-step (BSGS): General DLP in O(sqrt(n)) time/space. Combined with Pohlig-Hellman for smooth-order groups (all factors of p-1 or curve order are small). Sage: discrete_log(Mod(h,p), Mod(g,p)). See advanced-math.md.
• Fault injection: Compare correct vs faulty output; recover key bit-by-bit
• Clock group (x^2+y^2=1): Order = p+1 (not p-1!); Pohlig-Hellman when p+1 is smooth
• Isogenies: Graph traversal via modular polynomials; pathfinding via LCA
• ECDSA nonce reuse: Same r in two signatures leaks nonce k and private key d via modular arithmetic. Check for repeated r values
• Braid group DH: Alexander polynomial is multiplicative under braid concatenation — Eve computes shared secret from public keys. See exotic-crypto.md
• Ed25519 torsion side channel: Cofactor h=8 leaks secret scalar bits when key derivation uses key = master * uid mod l; query powers of 2, check y-coordinate consistency
• Tropical semiring residuation: Tropical (min-plus) DH is broken — residual b* = max(Mb[i] - M[i][j]) recovers shared secret directly from public matrices
• FPE Feistel brute-force: Format-preserving encryption with 16-bit round key is brute-forceable; remaining affine GF(2) mixing layer solved via Gaussian elimination. See exotic-crypto.md
• Icosahedral symmetry cipher: Dodecahedron face permutations form order-120 group; build lookup table of all permutations via API probing, match visible face patterns. See exotic-crypto.md
• Goldwasser-Micali replication oracle: GM encrypts one bit per ciphertext; replaying a single ciphertext value N times as an N-bit key forces all-zero or all-one key, distinguishable via hash oracle. 128 queries recover full AES key. See exotic-crypto.md
• DSA nonce reuse: Same r in two DSA signatures leaks private key via same formula as ECDSA nonce reuse. See ecc-attacks.md.
• DSA limited k brute force: When nonce k is small (e.g., 20-bit), brute-force all k values and check which yields the known r. See ecc-attacks.md.
• ECC shared prime GCD: Multiple ECC curves sharing a prime factor in their modulus; gcd(n1, n2) reveals the shared prime. See ecc-attacks.md.
• DSA key recovery via MD5 collision on k-generation: When nonce k derives from MD5(prefix+counter), use fastcoll to produce MD5 prefix collision forcing nonce reuse, then standard private key recovery. See ecc-attacks.md.
• BB-84 QKD MITM: Simulated BB-84 without authenticated classical channels allows full MITM -- independently negotiate keys with both parties, force constant value to one side. See exotic-crypto.md.

See ecc-attacks.md, advanced-math.md, and exotic-crypto.md for full code examples.

Lattice / LWE Attacks

• - Quick triage: If the challenge gives modular linear equations plus a promise that the hidden quantity is small, sparse, biased, or only partially leaked, treat it as a lattice candidate first. See lattice-and-lwe.md.
• LLL / BKZ / Babai: Start with LLL, move to BKZ when LLL almost works, and use Babai after reduction for approximate CVP. See lattice-and-lwe.md.
• HNP from partial nonce leakage: Partial or biased ECDSA/Schnorr nonces often reduce to Hidden Number Problem lattices; normalize equations, isolate bounded error, reduce, then brute-force the last few bits if needed. See lattice-and-lwe.md.
• Truncated LCG state recovery: High-bit or low-bit leakage from affine recurrences is often just HNP in disguise; write each state as observed * 2^t + hidden and solve for the small hidden corrections. See lattice-and-lwe.md.
• LWE via CVP (Babai): Construct lattice from [q*I | 0; A^T | I], use fpylll CVP.babai to find closest vector, project to ternary {-1,0,1}. Watch for endianness mismatches between server description and actual encoding.
• Ring-LWE / Module-LWE recognition: Polynomial or negacyclic structure often looks scary but many CTFs weaken it with tiny coefficients, buggy representations, or enough leakage to flatten back into plain LWE. See lattice-and-lwe.md.
• Orthogonal lattices: Hidden subset or hidden subspace problems may need you to recover an orthogonal lattice first, then reconstruct the actual binary or short basis from its complement. See lattice-and-lwe.md.
• LLL for approximate GCD: Short vector in lattice reveals hidden factors
• Subset sum / knapsack: Binary knapsack and low-density subset-sum instances are still classic lattice territory; build the standard basis and look for a reduced row with a zero final coordinate. See lattice-and-lwe.md.
• Multi-layer challenges: Geometry → subspace recovery → LWE → AES-GCM decryption chain

See advanced-math.md for worked LWE solving code and lattice-and-lwe.md for attack selection, embeddings, and failure-mode triage.

ZKP & Constraint Solving

• - ZKP cheating: For impossible problems (3-coloring K4), find hash collisions or predict PRNG salts
• Graph 3-coloring: INLINECODE73
• Z3 solver: BitVec for bit-level, Int for arbitrary precision; BPF/SECCOMP filter solving
• Garbled circuits (free XOR): XOR three truth table entries to recover global delta
• Bigram substitution: OR-Tools CP-SAT with automaton constraint for known plaintext structure
• Trigram decomposition: Positions mod n form independent monoalphabetic ciphers
• Shamir SSS (deterministic coefficients): One share + seeded RNG = univariate equation in secret
• Race condition (TOCTOU): Synchronized concurrent requests bypass counter < N checks
• Groth16 broken setup (delta==gamma): Trivially forge: A=alpha, B=beta, C=-vkx. Always check verifier constants first
• Groth16 proof replay: Unconstrained nullifier + no tracking = infinite replays from setup tx
• DV-SNARG forgery: With verifier oracle access, learn secret v values from unconstrained pairs, forge via CRS entry cancellation
• Shamir SSS reused polynomial coefficients: When same random coefficients are used for every secret byte, subtracting shares cancels all randomness, leaving only plaintext differences. See zkp-and-advanced.md.