sort + uniq

The pair that solves frequency questions

Whenever you ask "what's the most common X in this file/log/output?" the answer is sort | uniq -c | sort -rn. Memorize that incantation; it solves a class of questions.

sort flags

• -n — numeric sort (10 after 9, not after 1).
• -r — reverse.
• -h — human-numeric (1K, 5M, 2G correctly).
• -u — unique (skip duplicates inline; lighter than uniq).
• -k 2 — sort by the second whitespace-separated field.
• -t ',' — set delimiter to comma (CSV).

uniq flags

• -c — prefix each line with its count.
• -d — only show duplicates.
• -u — only show non-duplicates.

Big gotcha: uniq only deduplicates adjacent lines. You almost always need sort | uniq, not just uniq alone.

The frequency one-liner

cat access.log | awk '{print $1}' | sort | uniq -c | sort -rn | head

Reads: take the first field of each log line (the IP), group identical ones together, count, sort by count descending, show top 10. That's a production-grade analytics query in one pipe.