Linux › Module 5 › Lesson 1
Pipes and Redirection (|, >, >>)
Chain commands with pipes and save output using redirection
Opening
One command is good. A chain is power.
Unix was designed as a workshop of small, sharp tools. ls lists. wc counts. grep searches. None of them tries to do everything. The magic is composition: a pipe (|) sends one command's output into the next. Redirection (>, >>, 2>) parks that output in a file — or catches errors so they do not flood your screen. This is how analysts filter millions of log lines in seconds without opening a GUI.
1. Three streams every command uses
Every process inherits three standard streams. stdin (file descriptor 0) is input — usually your keyboard, or the output of the previous command in a pipe. stdout (1) is normal results. stderr (2) is diagnostics and errors. They are separate on purpose: you can save a report and still see a failure, or hide "Permission denied" noise while keeping real matches.
stdin (0)
What the program reads. In a pipe, this is the previous command's stdout — not a file you typed by hand.
stdout (1)
Successful output. ls, cat, and grep write matches here. > and >> redirect this stream by default.
stderr (2)
Warnings and errors. A missing file, a denied directory, a bad flag. Redirect with 2> so it does not mix into your report.
2. The pipe: why Unix tools compose
The pipe symbol | connects stdout of the left command to stdin of the right. Read left to right: produce, then filter, then count. You do not write a custom program to "list files and count them" — you join ls and wc. That philosophy (do one thing well, then combine) is why Linux still wins at log analysis, incident response, and automation.
You can chain more than two commands. Each stage shrinks or transforms the stream. A defender might later write: cat auth.log | grep Failed | wc -l — but first you must understand that the pipe never copies the file into memory as a spreadsheet. It streams text. That is why a 2 GB log is still searchable.
3. Overwrite vs append: > and >>
A single > sends stdout to a file. If the file exists, it is truncated — old contents are gone. That is a feature when you want a fresh report. It is a disaster when you meant to add a line to notes you spent an hour writing.
Double >> appends. The file is created if missing, and new bytes are added at the end if it already exists. Incident notes, timestamps, and "scan started / scan done" lines belong on >> so you keep history. Treat > like a factory reset for that filename.
4. Catching errors with 2>
command 2> errors.txt writes stderr into a file and leaves stdout on the screen. command > out.txt 2> err.txt splits both streams. Some shells also support &> all.txt to merge them. When you search from / later in this module, 2>/dev/null discards permission-denied noise — it does not grant access, and it does not delete the errors from the system. It only hides them from your terminal.
5. Wrong vs right: overwriting the only copy
Worked failure mode — ">" on a log you meant to keep:
Wrong
You spend an hour collecting notes in status.txt, then run echo "scan started" > status.txt. The file now contains only those two words. The investigation timeline is gone. Same trap: command > auth.log when you thought you were saving a filtered copy under a new name.
Right
Use >> to append timestamps and status lines. When you need a fresh report, write to a new filename (failed-root.txt) instead of clobbering the source log. Confirm with cat or wc -l before you assume the file is intact.
6. Practical: pipes and redirection on your machine
Run these only in a folder you create under your home directory. Do not redirect over system logs in /var/log. This lab proves > vs >> and a tiny pipe — not an attack on anyone else's host.
Safe local pipes and redirection (your machine)
# Work only in YOUR practice folder mkdir -p "$HOME/cyberlium-lab" cd "$HOME/cyberlium-lab" # > creates or OVERWRITES echo "scan started" > status.txt cat status.txt # >> APPENDS — history survives date >> status.txt echo "scan done" >> status.txt cat status.txt # Pipe: stdout of printf becomes stdin of wc printf 'alpha beta gamma ' | wc -l # expect 3 # stderr only — missing path is an error, not a crash ls o/such/cyberlium-path 2> errors.txt cat errors.txt # DO NOT: echo anything > /var/log/auth.log # DO NOT: run these against files you do not own
Mission: prove overwrite vs append
In $HOME/cyberlium-lab, write a line with >, then append date and a second line with >>. Pipe three lines into wc -l and confirm the count is 3. Save one stderr sample with 2>. Write one sentence: "> would have destroyed my notes; >> kept them."
Stuck? Ask Cyberlium AI Mentor
If > vs >> or stdin/stdout/stderr feels abstract, ask Cyberlium AI Mentor for a hint — not a destructive one-liner. Try: "Hint only: which operator appends without erasing my notes file?" Or: "Hint only: stdout vs stderr — which one does 2> catch?"
You can name the three streams, compose tools with |, and choose > versus >> on purpose. Next — grep, head, tail, and wc — search and slice text so you never open a giant log in an editor just to find one line.
Knowledge Check
APPLY: You have investigation notes in status.txt. You run echo "checkpoint" > status.txt. What happened?
Multiple choice
Knowledge Check
APPLY: Why do defenders chain small Unix tools with | instead of writing one giant custom program?
Multiple choice
Knowledge Check
APPLY: True or False: command 2> errors.txt redirects stderr, leaving stdout on the terminal unless you also redirect it.
True or False