Cleanup Patterns

Common cleanups, ready to lift

Text normalization is where regex shines. Each of these is a one-liner you can copy.

Collapse whitespace

re.sub(r'\s+', ' ', text) — replace any run of whitespace (spaces, tabs, newlines) with a single space.

Strip non-printable characters

re.sub(r'[^\x20-\x7E\n]', '', text) — keep only printable ASCII plus newline. Useful for cleaning corrupted text.

Remove HTML tags (loose, not parsing)

re.sub(r'<[^>]+>', '', text) — strip everything inside angle brackets. For ad-hoc text extraction, fine. For real HTML, use a parser.

Trim trailing whitespace per line

re.sub(r'[ \t]+$', '', text, flags=re.MULTILINE) — remove trailing spaces and tabs from each line.

Normalize newlines

re.sub(r'\r\n?', '\n', text) — convert Windows (\r\n) and old Mac (\r) newlines to Unix (\n).

Strip surrounding quotes

re.sub(r'^["\']+|["\']+$', '', text) — remove leading and trailing quotes.

Mask sensitive patterns

re.sub(r'\b\d{4}-\d{4}-\d{4}-\d{4}\b', '****-****-****-****', text) — mask credit-card-shaped strings. Same pattern, different replacement, for SSNs etc.

Slugify a title

Three-step: lowercase, replace non-alphanumeric with hyphen, collapse multiple hyphens.

Code

Cleanup patterns·python

import re

# Collapse whitespace
re.sub(r'\s+', ' ', 'too\tmany\n\nspaces')
# 'too many spaces'

# Strip non-printable
re.sub(r'[^\x20-\x7E\n]', '', 'hello\x00world\x07')
# 'helloworld'

# Remove HTML tags
re.sub(r'<[^>]+>', '', '<p>Hello <b>world</b></p>')
# 'Hello world'

# Trim trailing whitespace
re.sub(r'[ \t]+$', '', 'line1   \nline2 \t', flags=re.MULTILINE)
# 'line1\nline2'

# Normalize newlines
re.sub(r'\r\n?', '\n', 'win\r\nmac\rmix\n')
# 'win\nmac\nmix\n'

# Mask credit card
re.sub(r'\b\d{4}-\d{4}-\d{4}-\d{4}\b', '****-****-****-****', 'card 1234-5678-9012-3456')
# 'card ****-****-****-****'

# Slugify
def slugify(text):
    text = text.lower()
    text = re.sub(r'[^a-z0-9]+', '-', text)
    text = re.sub(r'-+', '-', text)
    return text.strip('-')

slugify('Hello, World! Pippa #1')
# 'hello-world-pippa-1'