Regex

A regular expression is a pattern language for matching text. Parentheses create a capture group, allowing only one part of the complete match to be stored. Python’s re.search() returns the first match object or None, while re.findall() returns every non-overlapping match.

Capture between fixed markers

[^<] means any character except <, and * permits zero or more of those characters. The surrounding parentheses store that part as capture group 1. group(0) would contain the complete matched text, while group(1) contains only the text captured by the first pair of parentheses.

import re
match = re.search(r"Results:</b><br><br>([^<]*)</center>", r.text)
value = None
if match:
    captured_value = match.group(1)
    captured_value = captured_value.strip()
    if captured_value:
        value = captured_value

Sample response fragment

<center><b>Results:</b><br><br>VALUE</center>

Example output

value -> "VALUE"

_{Find by: regex, re.search, capture group, extract, between markers, parse response, scrape without bs4, group1, pattern, csrf, token, flag · Source: CWEE/XPath in-band}

Non-greedy capture across newlines (re.DOTALL)

. normally matches any character except a newline. re.DOTALL changes it to include newline characters, allowing one capture group to span several lines. *? is a non-greedy repetition: it stops at the first following end marker that allows the complete pattern to match. \s* accepts any surrounding whitespace.

import re
text = "Name: uid=0(root) gid=0(root) groups=0(root)\nVaccination Status: Complete"
match = re.search(r"Name:\s*(.*?)\s*Vaccination\s+Status:", text, re.DOTALL)
value = None
if match:
    value = match.group(1)
    value = value.strip()
print(value)

Sample text (the two labels sit on different lines)

Name: uid=0(root) gid=0(root) groups=0(root)
Vaccination Status: Complete

group(1)

uid=0(root) gid=0(root) groups=0(root)

_{Find by: regex, re.search, DOTALL, non greedy, lazy, multiline, newline, dot matches newline, capture group, group0 group1, whitespace, extract from pdf text, command output · Source: CWEE/PDF RCE}

Every match at once (re.findall)

re.findall() returns a list containing every non-overlapping match in input order. With one capture group, each list item is the captured string. With two or more capture groups, each item is a tuple containing those captured strings. No matches produce an empty list rather than None.

import re

id_pattern = r"/user/(\d+)"
ids = re.findall(id_pattern, html)

cookie_pattern = r"(\w+)=([^;]+)"
cookie_pairs = re.findall(cookie_pattern, cookie)

Example output

ids -> ['1', '2', '3']
cookie_pairs -> [('sid', 'abc'), ('role', 'admin')]

_{Find by: regex, re.findall, all matches, list, multiple, tuples, groups, iterate matches, extract all ids, enumerate, scrape list, idor sweep, no none guard · Source: CWEE/many}

HTTP Encodings