Regex basics
A regex is built from literal characters plus metacharacters that mean "a class of characters" or "a repetition." Character classes: \d is any digit, \w any word character (letter, digit, underscore), \s whitespace, and . any character; square brackets define a custom class like [A-Z]. Quantifiers control repetition: + means one or more, * zero or more, ? optional, and {2,4} a specific count range. Anchors ^ and $ tie the match to the start or end of a line. Parentheses create capture groups - the part of the match you actually want to pull out. So price:\s*\$(\d+\.\d{2}) finds "price: $19.99" and captures just "19.99".
How regex fits into web scraping
The clean division of labor is: use CSS selectors or XPath to navigate the HTML and grab the right element, then use regex on that element's text to extract the precise value. You select the .price span, then regex pulls the number out of "Only $19.99 today!". Regex also shines at cleanup - stripping whitespace, normalizing phone formats, splitting a combined string - and at scanning free-form text where there is no element boundary at all, like finding every email or URL in a blob of page copy. Used this way, on text rather than on markup, regex is fast, dependency-free, and exactly the right tool.
When not to use regex
The classic mistake is trying to parse HTML structure with regex - matching tags, walking nested elements, extracting an attribute by pattern-matching the raw markup. HTML is not a regular language: it nests arbitrarily, tags can be malformed, attributes reorder, and whitespace varies, so a regex that works on one page silently breaks on the next. Use a real HTML parser (which builds a proper tree) for structure, and reserve regex for the leaf-level text inside the elements that parser hands you. If your regex is matching <div or href=, that is the signal to switch to a selector. Keeping regex on the text side and selectors on the structure side keeps both robust.