Loading source
Pulling the file list, source metadata, and syntax-aware rendering for this listing.
Source from repo
Guidance for developing custom slash commands for Claude Code plugins from the official Anthropic repository.
Files
Skill
Size
Entrypoint
Format
Open file
Syntax-highlighted preview of this file as included in the skill package.
references/testing-strategies.md
1# Command Testing Strategies23Comprehensive strategies for testing slash commands before deployment and distribution.45## Overview67Testing commands ensures they work correctly, handle edge cases, and provide good user experience. A systematic testing approach catches issues early and builds confidence in command reliability.89## Testing Levels1011### Level 1: Syntax and Structure Validation1213**What to test:**14- YAML frontmatter syntax15- Markdown format16- File location and naming1718**How to test:**1920```bash21# Validate YAML frontmatter22head -n 20 .claude/commands/my-command.md | grep -A 10 "^---"2324# Check for closing frontmatter marker25head -n 20 .claude/commands/my-command.md | grep -c "^---" # Should be 22627# Verify file has .md extension28ls .claude/commands/*.md2930# Check file is in correct location31test -f .claude/commands/my-command.md && echo "Found" || echo "Missing"32```3334**Automated validation script:**3536```bash37#!/bin/bash38# validate-command.sh3940COMMAND_FILE="$1"4142if [ ! -f "$COMMAND_FILE" ]; then43echo "ERROR: File not found: $COMMAND_FILE"44exit 145fi4647# Check .md extension48if [[ ! "$COMMAND_FILE" =~ \.md$ ]]; then49echo "ERROR: File must have .md extension"50exit 151fi5253# Validate YAML frontmatter if present54if head -n 1 "$COMMAND_FILE" | grep -q "^---"; then55# Count frontmatter markers56MARKERS=$(head -n 50 "$COMMAND_FILE" | grep -c "^---")57if [ "$MARKERS" -ne 2 ]; then58echo "ERROR: Invalid YAML frontmatter (need exactly 2 '---' markers)"59exit 160fi61echo "✓ YAML frontmatter syntax valid"62fi6364# Check for empty file65if [ ! -s "$COMMAND_FILE" ]; then66echo "ERROR: File is empty"67exit 168fi6970echo "✓ Command file structure valid"71```7273### Level 2: Frontmatter Field Validation7475**What to test:**76- Field types correct77- Values in valid ranges78- Required fields present (if any)7980**Validation script:**8182```bash83#!/bin/bash84# validate-frontmatter.sh8586COMMAND_FILE="$1"8788# Extract YAML frontmatter89FRONTMATTER=$(sed -n '/^---$/,/^---$/p' "$COMMAND_FILE" | sed '1d;$d')9091if [ -z "$FRONTMATTER" ]; then92echo "No frontmatter to validate"93exit 094fi9596# Check 'model' field if present97if echo "$FRONTMATTER" | grep -q "^model:"; then98MODEL=$(echo "$FRONTMATTER" | grep "^model:" | cut -d: -f2 | tr -d ' ')99if ! echo "sonnet opus haiku" | grep -qw "$MODEL"; then100echo "ERROR: Invalid model '$MODEL' (must be sonnet, opus, or haiku)"101exit 1102fi103echo "✓ Model field valid: $MODEL"104fi105106# Check 'allowed-tools' field format107if echo "$FRONTMATTER" | grep -q "^allowed-tools:"; then108echo "✓ allowed-tools field present"109# Could add more sophisticated validation here110fi111112# Check 'description' length113if echo "$FRONTMATTER" | grep -q "^description:"; then114DESC=$(echo "$FRONTMATTER" | grep "^description:" | cut -d: -f2-)115LENGTH=${#DESC}116if [ "$LENGTH" -gt 80 ]; then117echo "WARNING: Description length $LENGTH (recommend < 60 chars)"118else119echo "✓ Description length acceptable: $LENGTH chars"120fi121fi122123echo "✓ Frontmatter fields valid"124```125126### Level 3: Manual Command Invocation127128**What to test:**129- Command appears in `/help`130- Command executes without errors131- Output is as expected132133**Test procedure:**134135```bash136# 1. Start Claude Code137claude --debug138139# 2. Check command appears in help140> /help141# Look for your command in the list142143# 3. Invoke command without arguments144> /my-command145# Check for reasonable error or behavior146147# 4. Invoke with valid arguments148> /my-command arg1 arg2149# Verify expected behavior150151# 5. Check debug logs152tail -f ~/.claude/debug-logs/latest153# Look for errors or warnings154```155156### Level 4: Argument Testing157158**What to test:**159- Positional arguments work ($1, $2, etc.)160- $ARGUMENTS captures all arguments161- Missing arguments handled gracefully162- Invalid arguments detected163164**Test matrix:**165166| Test Case | Command | Expected Result |167|-----------|---------|-----------------|168| No args | `/cmd` | Graceful handling or useful message |169| One arg | `/cmd arg1` | $1 substituted correctly |170| Two args | `/cmd arg1 arg2` | $1 and $2 substituted |171| Extra args | `/cmd a b c d` | All captured or extras ignored appropriately |172| Special chars | `/cmd "arg with spaces"` | Quotes handled correctly |173| Empty arg | `/cmd ""` | Empty string handled |174175**Test script:**176177```bash178#!/bin/bash179# test-command-arguments.sh180181COMMAND="$1"182183echo "Testing argument handling for /$COMMAND"184echo185186echo "Test 1: No arguments"187echo " Command: /$COMMAND"188echo " Expected: [describe expected behavior]"189echo " Manual test required"190echo191192echo "Test 2: Single argument"193echo " Command: /$COMMAND test-value"194echo " Expected: 'test-value' appears in output"195echo " Manual test required"196echo197198echo "Test 3: Multiple arguments"199echo " Command: /$COMMAND arg1 arg2 arg3"200echo " Expected: All arguments used appropriately"201echo " Manual test required"202echo203204echo "Test 4: Special characters"205echo " Command: /$COMMAND \"value with spaces\""206echo " Expected: Entire phrase captured"207echo " Manual test required"208```209210### Level 5: File Reference Testing211212**What to test:**213- @ syntax loads file contents214- Non-existent files handled215- Large files handled appropriately216- Multiple file references work217218**Test procedure:**219220```bash221# Create test files222echo "Test content" > /tmp/test-file.txt223echo "Second file" > /tmp/test-file-2.txt224225# Test single file reference226> /my-command /tmp/test-file.txt227# Verify file content is read228229# Test non-existent file230> /my-command /tmp/nonexistent.txt231# Verify graceful error handling232233# Test multiple files234> /my-command /tmp/test-file.txt /tmp/test-file-2.txt235# Verify both files processed236237# Test large file238dd if=/dev/zero of=/tmp/large-file.bin bs=1M count=100239> /my-command /tmp/large-file.bin240# Verify reasonable behavior (may truncate or warn)241242# Cleanup243rm /tmp/test-file*.txt /tmp/large-file.bin244```245246### Level 6: Bash Execution Testing247248**What to test:**249- !` commands execute correctly250- Command output included in prompt251- Command failures handled252- Security: only allowed commands run253254**Test procedure:**255256```bash257# Create test command with bash execution258cat > .claude/commands/test-bash.md << 'EOF'259---260description: Test bash execution261allowed-tools: Bash(echo:*), Bash(date:*)262---263264Current date: !`date`265Test output: !`echo "Hello from bash"`266267Analysis of output above...268EOF269270# Test in Claude Code271> /test-bash272# Verify:273# 1. Date appears correctly274# 2. Echo output appears275# 3. No errors in debug logs276277# Test with disallowed command (should fail or be blocked)278cat > .claude/commands/test-forbidden.md << 'EOF'279---280description: Test forbidden command281allowed-tools: Bash(echo:*)282---283284Trying forbidden: !`ls -la /`285EOF286287> /test-forbidden288# Verify: Permission denied or appropriate error289```290291### Level 7: Integration Testing292293**What to test:**294- Commands work with other plugin components295- Commands interact correctly with each other296- State management works across invocations297- Workflow commands execute in sequence298299**Test scenarios:**300301**Scenario 1: Command + Hook Integration**302303```bash304# Setup: Command that triggers a hook305# Test: Invoke command, verify hook executes306307# Command: .claude/commands/risky-operation.md308# Hook: PreToolUse that validates the operation309310> /risky-operation311# Verify: Hook executes and validates before command completes312```313314**Scenario 2: Command Sequence**315316```bash317# Setup: Multi-command workflow318> /workflow-init319# Verify: State file created320321> /workflow-step2322# Verify: State file read, step 2 executes323324> /workflow-complete325# Verify: State file cleaned up326```327328**Scenario 3: Command + MCP Integration**329330```bash331# Setup: Command uses MCP tools332# Test: Verify MCP server accessible333334> /mcp-command335# Verify:336# 1. MCP server starts (if stdio)337# 2. Tool calls succeed338# 3. Results included in output339```340341## Automated Testing Approaches342343### Command Test Suite344345Create a test suite script:346347```bash348#!/bin/bash349# test-commands.sh - Command test suite350351TEST_DIR=".claude/commands"352FAILED_TESTS=0353354echo "Command Test Suite"355echo "=================="356echo357358for cmd_file in "$TEST_DIR"/*.md; do359cmd_name=$(basename "$cmd_file" .md)360echo "Testing: $cmd_name"361362# Validate structure363if ./validate-command.sh "$cmd_file"; then364echo " ✓ Structure valid"365else366echo " ✗ Structure invalid"367((FAILED_TESTS++))368fi369370# Validate frontmatter371if ./validate-frontmatter.sh "$cmd_file"; then372echo " ✓ Frontmatter valid"373else374echo " ✗ Frontmatter invalid"375((FAILED_TESTS++))376fi377378echo379done380381echo "=================="382echo "Tests complete"383echo "Failed: $FAILED_TESTS"384385exit $FAILED_TESTS386```387388### Pre-Commit Hook389390Validate commands before committing:391392```bash393#!/bin/bash394# .git/hooks/pre-commit395396echo "Validating commands..."397398COMMANDS_CHANGED=$(git diff --cached --name-only | grep "\.claude/commands/.*\.md")399400if [ -z "$COMMANDS_CHANGED" ]; then401echo "No commands changed"402exit 0403fi404405for cmd in $COMMANDS_CHANGED; do406echo "Checking: $cmd"407408if ! ./scripts/validate-command.sh "$cmd"; then409echo "ERROR: Command validation failed: $cmd"410exit 1411fi412done413414echo "✓ All commands valid"415```416417### Continuous Testing418419Test commands in CI/CD:420421```yaml422# .github/workflows/test-commands.yml423name: Test Commands424425on: [push, pull_request]426427jobs:428test:429runs-on: ubuntu-latest430steps:431- uses: actions/checkout@v2432433- name: Validate command structure434run: |435for cmd in .claude/commands/*.md; do436echo "Testing: $cmd"437./scripts/validate-command.sh "$cmd"438done439440- name: Validate frontmatter441run: |442for cmd in .claude/commands/*.md; do443./scripts/validate-frontmatter.sh "$cmd"444done445446- name: Check for TODOs447run: |448if grep -r "TODO" .claude/commands/; then449echo "ERROR: TODOs found in commands"450exit 1451fi452```453454## Edge Case Testing455456### Test Edge Cases457458**Empty arguments:**459```bash460> /cmd ""461> /cmd '' ''462```463464**Special characters:**465```bash466> /cmd "arg with spaces"467> /cmd arg-with-dashes468> /cmd arg_with_underscores469> /cmd arg/with/slashes470> /cmd 'arg with "quotes"'471```472473**Long arguments:**474```bash475> /cmd $(python -c "print('a' * 10000)")476```477478**Unusual file paths:**479```bash480> /cmd ./file481> /cmd ../file482> /cmd ~/file483> /cmd "/path with spaces/file"484```485486**Bash command edge cases:**487```markdown488# Commands that might fail489!`exit 1`490!`false`491!`command-that-does-not-exist`492493# Commands with special output494!`echo ""`495!`cat /dev/null`496!`yes | head -n 1000000`497```498499## Performance Testing500501### Response Time Testing502503```bash504#!/bin/bash505# test-command-performance.sh506507COMMAND="$1"508509echo "Testing performance of /$COMMAND"510echo511512for i in {1..5}; do513echo "Run $i:"514START=$(date +%s%N)515516# Invoke command (manual step - record time)517echo " Invoke: /$COMMAND"518echo " Start time: $START"519echo " (Record end time manually)"520echo521done522523echo "Analyze results:"524echo " - Average response time"525echo " - Variance"526echo " - Acceptable threshold: < 3 seconds for fast commands"527```528529### Resource Usage Testing530531```bash532# Monitor Claude Code during command execution533# In terminal 1:534claude --debug535536# In terminal 2:537watch -n 1 'ps aux | grep claude'538539# Execute command and observe:540# - Memory usage541# - CPU usage542# - Process count543```544545## User Experience Testing546547### Usability Checklist548549- [ ] Command name is intuitive550- [ ] Description is clear in `/help`551- [ ] Arguments are well-documented552- [ ] Error messages are helpful553- [ ] Output is formatted readably554- [ ] Long-running commands show progress555- [ ] Results are actionable556- [ ] Edge cases have good UX557558### User Acceptance Testing559560Recruit testers:561562```markdown563# Testing Guide for Beta Testers564565## Command: /my-new-command566567### Test Scenarios5685691. **Basic usage:**570- Run: `/my-new-command`571- Expected: [describe]572- Rate clarity: 1-55735742. **With arguments:**575- Run: `/my-new-command arg1 arg2`576- Expected: [describe]577- Rate usefulness: 1-55785793. **Error case:**580- Run: `/my-new-command invalid-input`581- Expected: Helpful error message582- Rate error message: 1-5583584### Feedback Questions5855861. Was the command easy to understand?5872. Did the output meet your expectations?5883. What would you change?5894. Would you use this command regularly?590```591592## Testing Checklist593594Before releasing a command:595596### Structure597- [ ] File in correct location598- [ ] Correct .md extension599- [ ] Valid YAML frontmatter (if present)600- [ ] Markdown syntax correct601602### Functionality603- [ ] Command appears in `/help`604- [ ] Description is clear605- [ ] Command executes without errors606- [ ] Arguments work as expected607- [ ] File references work608- [ ] Bash execution works (if used)609610### Edge Cases611- [ ] Missing arguments handled612- [ ] Invalid arguments detected613- [ ] Non-existent files handled614- [ ] Special characters work615- [ ] Long inputs handled616617### Integration618- [ ] Works with other commands619- [ ] Works with hooks (if applicable)620- [ ] Works with MCP (if applicable)621- [ ] State management works622623### Quality624- [ ] Performance acceptable625- [ ] No security issues626- [ ] Error messages helpful627- [ ] Output formatted well628- [ ] Documentation complete629630### Distribution631- [ ] Tested by others632- [ ] Feedback incorporated633- [ ] README updated634- [ ] Examples provided635636## Debugging Failed Tests637638### Common Issues and Solutions639640**Issue: Command not appearing in /help**641642```bash643# Check file location644ls -la .claude/commands/my-command.md645646# Check permissions647chmod 644 .claude/commands/my-command.md648649# Check syntax650head -n 20 .claude/commands/my-command.md651652# Restart Claude Code653claude --debug654```655656**Issue: Arguments not substituting**657658```bash659# Verify syntax660grep '\$1' .claude/commands/my-command.md661grep '\$ARGUMENTS' .claude/commands/my-command.md662663# Test with simple command first664echo "Test: \$1 and \$2" > .claude/commands/test-args.md665```666667**Issue: Bash commands not executing**668669```bash670# Check allowed-tools671grep "allowed-tools" .claude/commands/my-command.md672673# Verify command syntax674grep '!\`' .claude/commands/my-command.md675676# Test command manually677date678echo "test"679```680681**Issue: File references not working**682683```bash684# Check @ syntax685grep '@' .claude/commands/my-command.md686687# Verify file exists688ls -la /path/to/referenced/file689690# Check permissions691chmod 644 /path/to/referenced/file692```693694## Best Practices6956961. **Test early, test often**: Validate as you develop6972. **Automate validation**: Use scripts for repeatable checks6983. **Test edge cases**: Don't just test the happy path6994. **Get feedback**: Have others test before wide release7005. **Document tests**: Keep test scenarios for regression testing7016. **Monitor in production**: Watch for issues after release7027. **Iterate**: Improve based on real usage data703