The character that changed computing
The pipe character | connects the stdout of the command on the left to the stdin of the command on the right. Doug McIlroy introduced the pipe at Bell Labs in 1973, making composition between small programs a central Unix pattern.
The mental model
Each pipe stage is a process. They run in parallel, with the kernel buffering data between them. cmd1 | cmd2 | cmd3 launches three processes; each one reads as it can and produces as it goes. There is no intermediate file — everything is in-memory streams.
Reading pipes left-to-right
A pipe is a recipe in order: "do A, feed result to B, feed result to C." ps aux | grep python | grep -v grep | wc -l reads as: "list every process, keep only python ones, drop the grep itself, count the lines." Once you can write English in pipe form, you've internalized the model.
Pipe exit codes
By default a pipeline's exit code is the last command's. cat missing | wc -l exits 0 because wc succeeded. set -o pipefail changes the rule: any failed stage fails the whole pipeline. Always set this in scripts (we cover the full set -euo pipefail pattern in scripting).
When pipes are the wrong tool
Pipes stream byte by byte. Tools that need the whole input upfront — sort, jq with certain filters, anything random access — buffer it before producing output. That's fine for typical sizes but for multi-GB streams, prefer named pipes (FIFOs), or process substitution for parallel branches.
A pipe does not store intermediate results
Output flows through buffers at the consumer's pace. If a downstream command such as head exits early, the producer may receive SIGPIPE as a normal consequence. pipefail does not make every nonzero status the same failure.
Long pipelines erase observation points
When several transformations are joined, intermediate formats disappear. Break a surprising pipeline into stages and inspect samples and statuses until you find the first broken contract.