Log Parser Script

A text log is a sequence of lines. A parser is a loop that asks a yes/no question of each line and sometimes pulls a slice. with open(path, encoding="utf-8", errors="ignore") as f keeps the handle from leaking if you hit a bad byte — logs are messy; errors="ignore" is a teaching choice so a single corrupt character does not abort the count. for line in f reads lazily. You do not f.read() a multi-gigabyte file into one string on a laptop as the first move. if "Failed password" in line is a substring filter: cheap, readable, good enough until two different messages share the words. count += 1 is the report. print the total. That is a parser. Everything fancier (regex groups, Counter, CSV output) is the same loop with more memory of what it saw.

Work on a copy. The only copy of a log is evidence and an operational file; a parser bug that truncates on write is how people destroy the only trail. In this course you never had production. You create the sample. If you later have authorization to export, you export, you parse the export, you leave the live file alone. You do not open the live path with mode "w". You do not delete lines that matched to “clean up.” You do not email the full log to a public list to get help. Counts and top-IP lists can go in chmod 600 notes. Full dumps of real users do not go on GitHub.

Failed password lines are a teaching stand-in for “auth failure.” Error and panic strings are the same idea with a different needle. Suspicious IPs in this course means “addresses that appear often in MY sample,” ranked so you see a spray-shaped pattern in fiction. It does not mean geo-lookup, WHOIS harassment, or knocking on the address with Module 2’s scanner. A documentation IP that failed twenty times in your sample is a Counter lesson, not a person. Do not treat RFC 5737 / TEST-NET addresses as attackers to pursue. Do not paste real customer IPs from a job into the lab file “to make it realistic.” Invent the lines. Keep usernames fictional (root, admin, alice) so you are not practicing on a roster.

Password spraying in a real SOC is a pattern: many usernames, few passwords, many failures from one source. Your parser can count failures per IP once the lab extracts the address. That count is a hint for a human, not a green light to lock a production account or to scan the source. False positives exist: a misconfigured printer, your own lab VM, a user who forgot a rotation. The script does not know. You already learned that automation reports; it does not isolate. Keep that split when the numbers look exciting.

Needle in line fails when the interesting field moves or when Failed password appears in a comment. Regex (Module 4) extracts with groups: the IP after from , a status code, a username. You will still test on five to ten sample lines you own, not on a firehose of other people’s traffic. Do not overfit a regex to one vendor’s format and then declare all logs parsed. Do not use a parser as cover to netcat the IP you just extracted. Extraction is not authorization to connect. Module 2 already limited sockets to localhost; that limit does not expire because a log line named 203.0.113.10.

The file below is teaching fiction. Type it yourself (or redirect it) into cyberlium-lab. Run the counter. Record the number in notes. Do not replace the lines with a log from work. Do not add real coworker names. chmod 600 the sample and the notes — even fiction can sit next to other lab files that are more sensitive.

# FICTIONAL log YOU create. Do NOT scrape production.
# IPs are documentation ranges — not people to deanonymize or scan.

mkdir -p "$HOME/cyberlium-lab"
cd "$HOME/cyberlium-lab"

cat > sample_auth.log << 'LOG'
sshd: Failed password for root from 203.0.113.10 port 44022
sshd: Accepted password for alice from 192.0.2.10 port 51111
sshd: Failed password for admin from 203.0.113.10 port 44023
sshd: Failed password for root from 198.51.100.8 port 51234
sshd: Failed password for root from 203.0.113.10 port 44024
sshd: Accepted password for alice from 192.0.2.10 port 51112
sshd: Failed password for ubuntu from 198.51.100.8 port 51235
sshd: Failed password for root from 203.0.113.10 port 44025
LOG

cat > count_failures.py << 'PY'
path = "sample_auth.log"
needle = "Failed password"
count = 0

with open(path, encoding="utf-8", errors="ignore") as f:
    for line in f:
        if needle in line:
            count += 1

print("failures:", count)
# Next lesson: Counter of source IPs on this same fictional file.
PY

python3 count_failures.py

NOTES="$HOME/cyberlium-lab/log-parse-notes.txt"
{
  echo "=== LOG PARSE NOTES ==="
  echo "source: fictional sample_auth.log I created (not production)"
  echo "method: line-by-line; substring Failed password"
  echo "failures_count: (fill after run)"
  echo "ethics: no scrape, no scan of extracted IPs, no real user dump in git"
} > "$NOTES"

chmod 600 sample_auth.log count_failures.py "$NOTES"
# Windows without chmod: WSL/Git Bash, or restrict the files in your profile.

# NEVER: copy /var/log/auth.log from a job into this folder without authorization
# NEVER: nmap the from-addresses
# NEVER: treat 203.0.113.10 as a person

You now treat a log as lines you loop, a needle you test, and a count you report — on a copy you created. Substring first. IPs are labels, not targets. Production stays behind authorization. Next — Lab: Build a Log Analyzer — the same fictional file, plus Counter of source IPs, a short summary, and chmod 600 on everything in cyberlium-lab.