📦 deps(thirdparty): update snapshots

2026-05-29 08:33:53 +00:00
parent fdb52f1e96
commit 06e0d13d57
1615 changed files with 232858 additions and 0 deletions
@@ -0,0 +1,587 @@
+# Test Quality Metrics
+
+Comprehensive guide to measuring and improving test quality through coverage and mutation testing.
+
+## Coverage Metrics
+
+### Line Coverage
+
+Percentage of code lines executed during test runs.
+
+**Target**: ≥80% overall, ≥90% for critical paths
+
+**TypeScript/Bun**:
+
+```bash
+bun test --coverage
+
+# Output
+Coverage Summary:
+  Statements   : 85.2% ( 1420/1667 )
+  Branches     : 78.5% ( 314/400 )
+  Functions    : 82.1% ( 156/190 )
+  Lines        : 85.2% ( 1420/1667 )
+```
+
+**Rust**:
+
+```bash
+# Using cargo-tarpaulin
+cargo tarpaulin --out Html --output-dir coverage/
+
+# Using cargo-llvm-cov
+cargo llvm-cov --html
+```
+
+### Branch Coverage
+
+Percentage of decision branches (if/else, switch, ternary) executed.
+
+**Target**: ≥75%
+
+Example showing uncovered branch:
+
+```typescript
+function divide(a: number, b: number): number {
+  if (b === 0) {  // Branch covered
+    throw new Error('Division by zero')
+  }
+  return a / b    // Branch covered
+}
+
+// Test only covers success path
+test('divides numbers', () => {
+  expect(divide(10, 2)).toBe(5)
+})
+
+// Coverage: 50% branches (only success branch covered)
+```
+
+Fix with both branches:
+
+```typescript
+test('divides numbers', () => {
+  expect(divide(10, 2)).toBe(5)
+})
+
+test('throws on division by zero', () => {
+  expect(() => divide(10, 0)).toThrow('Division by zero')
+})
+
+// Coverage: 100% branches
+```
+
+### Function Coverage
+
+Percentage of functions called during tests.
+
+**Target**: ≥80%
+
+Uncovered functions often indicate:
+- Dead code that should be removed
+- Missing test cases
+- Helper functions only used in uncovered paths
+
+### Interpreting Coverage
+
+High coverage ≠ high quality. Coverage shows what's tested, not how well.
+
+**Example of misleading coverage**:
+
+```typescript
+function processPayment(amount: number): Result {
+  if (amount <= 0) {
+    return { type: 'error', code: 'INVALID_AMOUNT' }
+  }
+
+  const result = chargeCard(amount)
+  return { type: 'success', transactionId: result.id }
+}
+
+// Bad test with 100% coverage
+test('processes payment', () => {
+  processPayment(100)
+  processPayment(-10)
+})
+
+// No assertions! 100% coverage but 0% verification
+```
+
+Coverage shows code was executed, not that it was verified correct.
+
+## Mutation Testing
+
+Mutation testing verifies test quality by introducing small bugs and checking if tests catch them.
+
+### How It Works
+
+1. **Mutant Generation**: Tool mutates source code (e.g., `===` → `!==`, `+` → `-`)
+2. **Test Execution**: Run tests against each mutant
+3. **Classification**:
+   - **Killed**: Test fails (good — test caught the bug)
+   - **Survived**: Test passes (bad — test missed the bug)
+   - **Timeout**: Mutant caused infinite loop
+   - **No Coverage**: Line not executed by tests
+
+### Mutation Score
+
+```
+Mutation Score = (Killed Mutants / Total Mutants) × 100%
+```
+
+**Target**: ≥75%
+
+### TypeScript Mutation Testing
+
+Using Stryker:
+
+**Install**:
+
+```bash
+bun add -d @stryker-mutator/core @stryker-mutator/typescript-checker
+```
+
+**Configuration** (`stryker.conf.json`):
+
+```json
+{
+  "mutator": "typescript",
+  "packageManager": "bun",
+  "reporters": ["html", "clear-text", "progress"],
+  "testRunner": "bun",
+  "coverageAnalysis": "perTest",
+  "mutate": [
+    "src/**/*.ts",
+    "!src/**/*.test.ts",
+    "!src/**/*.spec.ts"
+  ],
+  "thresholds": {
+    "high": 80,
+    "low": 60,
+    "break": 50
+  }
+}
+```
+
+**Run**:
+
+```bash
+bun x stryker run
+
+# Output
+Mutation testing complete:
+  Killed: 78
+  Survived: 12
+  Timeout: 2
+  No Coverage: 8
+  Mutation Score: 78.0%
+```
+
+**Common Mutations**:
+
+| Original | Mutant | Catches |
+|----------|--------|---------|
+| `===` | `!==` | Equality assertions |
+| `>` | `>=` | Boundary tests |
+| `+` | `-` | Arithmetic verification |
+| `&&` | `||` | Logic tests |
+| `true` | `false` | Boolean verification |
+| `0` | `1` | Zero handling |
+| `return x` | `return undefined` | Return value tests |
+
+**Example Analysis**:
+
+```typescript
+function calculateDiscount(price: number, isPremium: boolean): number {
+  if (isPremium) {
+    return price * 0.8  // 20% discount
+  }
+  return price
+}
+
+// Weak test
+test('calculates discount', () => {
+  calculateDiscount(100, true)
+  calculateDiscount(100, false)
+})
+
+// Mutation: 0.8 → 0.9
+// Status: Survived (no assertion)
+```
+
+Fix with assertions:
+
+```typescript
+test('applies 20% discount for premium users', () => {
+  expect(calculateDiscount(100, true)).toBe(80)
+})
+
+test('no discount for regular users', () => {
+  expect(calculateDiscount(100, false)).toBe(100)
+})
+
+// Mutation: 0.8 → 0.9
+// Status: Killed (test fails with 90 !== 80)
+```
+
+### Rust Mutation Testing
+
+Using `cargo-mutants`:
+
+**Install**:
+
+```bash
+cargo install cargo-mutants
+```
+
+**Run**:
+
+```bash
+cargo mutants
+
+# Output
+Mutation testing results:
+  caught: 45
+  missed: 5
+  timeout: 1
+  unviable: 2
+  score: 90.0%
+```
+
+**Common Mutations**:
+
+| Original | Mutant | Catches |
+|----------|--------|---------|
+| `==` | `!=` | Equality tests |
+| `>` | `>=` | Boundary tests |
+| `&&` | `||` | Logic tests |
+| `Some(x)` | `None` | Option handling |
+| `Ok(x)` | `Err(...)` | Result handling |
+| `+` | `-` | Arithmetic verification |
+
+**Example**:
+
+```rust
+fn calculate_discount(price: i32, is_premium: bool) -> i32 {
+    if is_premium {
+        price * 80 / 100  // 20% discount
+    } else {
+        price
+    }
+}
+
+// Weak test
+#[test]
+fn test_discount() {
+    calculate_discount(100, true);
+    calculate_discount(100, false);
+}
+
+// Mutation: 80 → 90
+// Status: missed (no assertion)
+```
+
+Fix:
+
+```rust
+#[test]
+fn applies_discount_for_premium() {
+    assert_eq!(calculate_discount(100, true), 80);
+}
+
+#[test]
+fn no_discount_for_regular() {
+    assert_eq!(calculate_discount(100, false), 100);
+}
+
+// Mutation: 80 → 90
+// Status: caught (assertion fails)
+```
+
+## Quality Standards Matrix
+
+| Metric | Minimum | Good | Excellent |
+|--------|---------|------|-----------|
+| Line Coverage | 70% | 80% | 90% |
+| Branch Coverage | 65% | 75% | 85% |
+| Function Coverage | 75% | 85% | 95% |
+| Mutation Score | 60% | 75% | 85% |
+| Test Execution Time | <10s | <5s | <2s |
+
+## Improving Test Quality
+
+### Weak Assertion Detection
+
+**Problem**: Tests execute code but don't verify results
+
+```typescript
+// ❌ Weak - no verification
+test('processes order', () => {
+  processOrder({ items: [item1, item2] })
+})
+```
+
+**Solution**:
+
+```typescript
+// ✓ Strong - verifies result
+test('processes order', () => {
+  const result = processOrder({ items: [item1, item2] })
+  expect(result.type).toBe('success')
+  expect(result.total).toBe(150)
+})
+```
+
+### Missing Edge Cases
+
+Use mutation testing to find gaps:
+
+```typescript
+function validateAge(age: number): boolean {
+  return age >= 18  // Mutant: >= → >
+}
+
+// Current test
+test('validates age', () => {
+  expect(validateAge(20)).toBe(true)
+  expect(validateAge(16)).toBe(false)
+})
+
+// Mutation survived: >= → >
+// Missing: boundary test for exactly 18
+```
+
+Add boundary test:
+
+```typescript
+test('accepts exactly 18', () => {
+  expect(validateAge(18)).toBe(true)
+})
+
+// Now mutation is caught
+```
+
+### Test Redundancy
+
+Multiple tests verifying same thing:
+
+```typescript
+// Redundant tests
+test('validates positive number', () => {
+  expect(isPositive(5)).toBe(true)
+})
+
+test('validates another positive number', () => {
+  expect(isPositive(10)).toBe(true)
+})
+
+test('validates yet another positive number', () => {
+  expect(isPositive(100)).toBe(true)
+})
+```
+
+Consolidate:
+
+```typescript
+test.each([5, 10, 100])('validates positive number %i', (num) => {
+  expect(isPositive(num)).toBe(true)
+})
+```
+
+## Continuous Quality Monitoring
+
+### CI/CD Integration
+
+**TypeScript**:
+
+```yaml
+# .github/workflows/test.yml
+- name: Run tests with coverage
+  run: bun test --coverage
+
+- name: Check coverage thresholds
+  run: |
+    coverage=$(bun test --coverage --json | jq '.coverage.total.statements.pct')
+    if (( $(echo "$coverage < 80" | bc -l) )); then
+      echo "Coverage $coverage% below 80% threshold"
+      exit 1
+    fi
+
+- name: Run mutation testing
+  run: bun x stryker run
+  # Fail if mutation score < 75%
+```
+
+**Rust**:
+
+```yaml
+# .github/workflows/test.yml
+- name: Run tests with coverage
+  run: cargo tarpaulin --fail-under 80
+
+- name: Run mutation testing
+  run: cargo mutants
+  continue-on-error: true  # Warning only initially
+```
+
+### Tracking Over Time
+
+Monitor trends:
+
+```bash
+# Generate coverage badge
+coverage=$(bun test --coverage --json | jq '.coverage.total.statements.pct')
+echo "Coverage: $coverage%" > coverage.txt
+
+# Track mutation score
+mutation=$(bun x stryker run --json | jq '.mutationScore')
+echo "Mutation Score: $mutation%" > mutation.txt
+```
+
+## Advanced Techniques
+
+### Differential Coverage
+
+Only measure coverage on changed code:
+
+```bash
+# Get changed files
+git diff --name-only main... > changed.txt
+
+# Run coverage on changed files
+bun test --coverage --changed-files changed.txt
+```
+
+### Coverage Ratcheting
+
+Prevent coverage from decreasing:
+
+```bash
+# Save current coverage
+current=$(bun test --coverage --json | jq '.coverage.total.statements.pct')
+echo "$current" > .baseline-coverage
+
+# On future runs, compare
+baseline=$(cat .baseline-coverage)
+if (( $(echo "$current < $baseline" | bc -l) )); then
+  echo "Coverage decreased from $baseline% to $current%"
+  exit 1
+fi
+```
+
+### Mutation Testing Optimization
+
+Run only on changed code:
+
+```bash
+# Stryker incremental mode
+bun x stryker run --incremental
+
+# cargo-mutants on specific files
+cargo mutants --file src/auth/mod.rs
+```
+
+## Common Pitfalls
+
+### Pitfall 1: Chasing 100% Coverage
+
+**Problem**: Diminishing returns past 90%, testing trivial code
+
+```typescript
+// Trivial getter - not worth testing
+class User {
+  get email(): string {
+    return this._email
+  }
+}
+```
+
+**Solution**: Focus on behavior, not line count. Exclude trivial code from coverage requirements.
+
+### Pitfall 2: Gaming Metrics
+
+**Problem**: Tests that execute code without verification
+
+```typescript
+// ❌ High coverage, zero value
+test('calls all functions', () => {
+  func1()
+  func2()
+  func3()
+})
+```
+
+**Solution**: Use mutation testing to catch weak assertions.
+
+### Pitfall 3: Slow Mutation Testing
+
+**Problem**: Full mutation testing takes hours
+
+**Solution**: Run incrementally or in CI only:
+
+```bash
+# Local: Quick feedback on changed files
+bun x stryker run --mutate "src/auth/**/*.ts"
+
+# CI: Full suite
+bun x stryker run
+```
+
+## Quality Metrics Dashboard
+
+Example report format:
+
+```
+Test Quality Report
+===================
+
+Coverage:
+  Statements: 85.2% ░░░░░░░░▓▓
+  Branches:   78.5% ░░░░░░░▓▓▓
+  Functions:  82.1% ░░░░░░░░▓▓
+
+Mutation Testing:
+  Score:      78.0% ░░░░░░░▓▓▓
+  Killed:     78
+  Survived:   12
+  No Cov:     8
+
+Performance:
+  Unit Tests: 2.3s  ✓
+  Total:      8.7s  ✓
+
+Status: ✓ All thresholds met
+```
+
+## Actionable Improvement Plan
+
+1. **Week 1**: Establish baseline
+   - Run coverage analysis
+   - Run mutation testing
+   - Document current state
+
+2. **Week 2-3**: Fix critical gaps
+   - Add tests for uncovered critical paths
+   - Fix survived mutants in high-risk code
+   - Target 80% coverage, 75% mutation score
+
+3. **Week 4**: Automate
+   - Add CI coverage checks
+   - Set up coverage ratcheting
+   - Schedule weekly mutation testing
+
+4. **Ongoing**: Maintain
+   - Review coverage on each PR
+   - Run mutation testing monthly
+   - Gradually raise thresholds
+
+## Resources
+
+TypeScript:
+- [Stryker Documentation](https://stryker-mutator.io)
+- [Bun Test Coverage](https://bun.sh/docs/cli/test#coverage)
+
+Rust:
+- [cargo-tarpaulin](https://github.com/xd009642/tarpaulin)
+- [cargo-llvm-cov](https://github.com/taiki-e/cargo-llvm-cov)
+- [cargo-mutants](https://github.com/sourcefrog/cargo-mutants)