Eval and Guards

Eval and guardrails literacy — red-team eval on YOUR app, input/output filters, refusal tests — authorized `$AI_LAB` only.

Eval and Guards• Audio Walkthrough

00:00/03:09

Visual · t32_eval_and_guards

Eval and guards = named test rows. $AI_LAB YOUR apps. Original Cyberlium.

Eval proves guards work on apps you own — name eval and guardrail rows on YOUR toy app, never weaponized jailbreak kits vs stranger SaaS.

Eval and guardrails literacy names: authorized red-team eval scope (YOUR app only), prompt injection test cases category, output policy filter, refusal/alignment test category, regression eval on model/prompt change, and guardrail bypass documentation (defender fix — not public kit). Analyst runs or documents eval plan on `$AI_LAB` toy app — five test cases with expected guard behavior — without publishing jailbreak weaponization, without testing stranger chatbots, without 'eval' as excuse for prod abuse. Cyberlium: AI red team = authorized eval of YOUR apps — governance/ethics gate. Refused: jailbreak kits, stranger app testing, hiding failed eval. Lab row: eval test matrix (case, expected guard, result category) five rows.

1. Authorized eval scope

YOUR toy app, written RoE, no stranger targets — three ethics anchors.

Red team here means improve YOUR guards — not weaponize bypasses.

Command guide

Try these commands — Authorized eval scope

═══ TOOLS & WEBSITES ═══ Browse / read these (authorized learning only — stay in YOUR lab / program scope)

OpenAI evals — https://platform.openai.com/docs/guides/evals OWASP LLM Top 10 — https://owasp.org/www-project-top-10-for-large-language-model-applications/ Hugging Face security — https://huggingface.co/docs/hub/security

═══ INSTALL ═══

Linux (Debian/Ubuntu):

Command — copy this

sudo apt install python3

macOS:

Command — copy this

brew install python3

Windows: Download https://python.org/downloads/

═══ LINUX / macOS ═══

Command — copy this

export AI_LAB=${AI_LAB:-$HOME/cyberlium-lab/t32-ai}
cat > "$AI_LAB/eval/eval-checklist.md" <<'EOF'
# Eval and Guardrails Checklist — YOUR lab
- [ ] Golden-set prompts for policy compliance
- [ ] Injection regression tests (YOUR bot only)
- [ ] Toxicity/PII leakage classifiers on output
- [ ] Tool-call allowlist enforcement tests
- [ ] Version eval results per model/prompt change
Run evals on YOUR toys — not automated jailbreak vs stranger SaaS
EOF

Command — copy this

grep -E 'Golden-set|YOUR toys|jailbreak' "$AI_LAB/eval/eval-checklist.md"

Primary tools to practice this lesson: grep, python3. Reference sites: OpenAI evals (https://platform.openai.com/docs/guides/evals); OWASP LLM Top 10 (https://owasp.org/www-project-top-10-for-large-language-model-applications/); Hugging Face security (https://huggingface.co/docs/hub/security). Run every command in the box — install first, then the usage lines — only on YOUR lab / program scope.

2. Named guard rows

Input filter, output policy, tool allowlist, rate limit, human gate — five controls.

Eval case documents pass/fail and fix ticket stub on lab app.

3. Refused

No jailbreak weaponization kits; no stranger app 'eval.'

Failed eval → fix guard — not publish bypass.

4. What you ship: eval test matrix

Five eval cases + expected guard + result + NEVER jailbreak kit line.

5. What you record before the next lesson

Eval test matrix path.

6. Wrong vs right: stranger SaaS vs YOUR toy LLM

Worked failure — same MSF word, opposite target. Right never needs a café Wi-Fi or classmate laptop.

Wrong: Publish jailbreak bypass kit from lab eval as 'open source security tool.'
Right: Eval matrix from authorized `$AI_LAB` toy app. Next: Secrets in Prompts.

Hands-on Exercise