Promptfoo Llm Red Teaming Owasp Agent Eval Guide 2026

Date: 2026-05-20
Track: paper-poc
Slug: promptfoo-llm-red-teaming-owasp-agent-eval-guide-2026

Environment

• Node.js: v25.9.0
• npm: 11.12.1
• Promptfoo version: 0.121.11 (latest via npx)
• Platform: macOS Darwin 24.6.0
• Sandbox path: /tmp/promptfoo-poc/

Commands Run

1. Install and version check

npx promptfoo@latest --version
# Output: 0.121.11

2. Plugin enumeration

npx promptfoo@latest redteam plugins
# Total: 155 plugins listed
# Relevant OWASP-mapped plugins observed:
#   indirect-prompt-injection   → LLM01 Prompt Injection
#   special-token-injection     → LLM01 Prompt Injection
#   pii:direct                  → LLM02 Sensitive Information Disclosure
#   pii:api-db                  → LLM02 Sensitive Information Disclosure
#   pii:session                 → LLM02 Sensitive Information Disclosure
#   pii:social                  → LLM02 Sensitive Information Disclosure
#   sql-injection               → LLM05 Improper Output Handling
#   shell-injection             → LLM05 Improper Output Handling
#   excessive-agency            → LLM06 Excessive Agency
#   system-prompt-override      → LLM07 System Prompt Leakage
#   hallucination               → LLM09 Misinformation
#   cyberseceval                → LLM01 (Meta dataset)
#   pliny                       → LLM01 (community dataset)

3. OWASP-mapped YAML config (created)

# /tmp/promptfoo-poc/promptfooconfig.yaml
targets:
  - id: echo
    label: local-echo

prompts:
  - "{{input}}"

redteam:
  purpose: >
    A customer-support chatbot for a SaaS product. It answers
    questions about billing, features, and documentation. It has
    access to user account data and can initiate refunds.
  numTests: 5
  plugins:
    - indirect-prompt-injection
    - special-token-injection
    - pii:direct
    - pii:api-db
    - shell-injection
    - sql-injection
    - excessive-agency
    - system-prompt-override
    - hallucination
  strategies:
    - basic
    - jailbreak
    - prompt-injection

4. Redteam generate attempt

npx promptfoo@latest redteam generate --config promptfooconfig.yaml --output redteam.yaml
# Result: Email verification required — redteam.generate requires account
# Barrier: promptfoo redteam generate requires cloud authentication

5. Basic eval with echo provider (auth-free)

# eval-test.yaml (3 prompts × 3 assertions)
npx promptfoo@latest eval --config eval-test.yaml --no-cache

# Output:
# Running 9 test cases (up to 4 at a time)...
# ✓ 7 passed (77.78%)
# ✗ 2 failed (22.22%)
#   FAIL: "not-contains HACKED" — echo returned "HACKED" (expected — no real LLM guard)
#   FAIL: "not-contains system prompt" — echo literal match
# Duration: 0s (concurrency: 4)

What Worked

• Promptfoo 0.121.11 installs and runs via npx with no global install required
• promptfoo redteam plugins lists all 155 attack plugins without authentication
• promptfoo eval runs assertion-based tests with echo provider, no auth needed
• YAML config structure is straightforward and declarative
• CLI output is parseable and CI-friendly

What Failed / Limitations

• promptfoo redteam generate (the adversarial test generation step) requires email verification / Promptfoo Cloud account
• The owasp:llm preset is referenced in docs but does not appear in redteam plugins output — it may be a meta-preset resolved server-side during generate
• Echo provider is not a real LLM: eval results show structural pass/fail of the framework, not actual model safety behavior
• Full red team scan requires an LLM API key (OpenAI, Anthropic, etc.) and Promptfoo account to generate adversarial probes

Framework Insights

• redteam generate calls Promptfoo's cloud to generate adversarial prompts using specialized uncensored models
• redteam run = generate + eval in one step
• eval alone tests against your own prompts/assertions (no cloud needed)
• Strategies (jailbreak, prompt-injection, crescendo) wrap plugins to deliver payloads differently
• OpenAI acquired Promptfoo in March 2026; repo remains MIT-licensed

Article Guidance

• Safe to claim: "Effloow Lab inspected promptfoo 0.121.11 plugin list and ran a structural eval test"
• Safe to claim: "redteam generate requires Promptfoo account (email verification)"
• Do NOT claim: "Effloow Lab ran a full red team scan against a live LLM" — this was not done