📦 deps(thirdparty): update snapshots
This commit is contained in:
@@ -0,0 +1,132 @@
|
||||
---
|
||||
name: "test-guard"
|
||||
description: "Review generated or changed test code against universal testing rules before it ships or is presented for approval."
|
||||
risk: "critical"
|
||||
source: "community"
|
||||
source_repo: "amElnagdy/guard-skills"
|
||||
source_type: "community"
|
||||
date_added: 2026-07-13
|
||||
author: "community"
|
||||
tags: []
|
||||
tools: []
|
||||
---
|
||||
|
||||
|
||||
# Test Guard
|
||||
|
||||
You are reviewing generated or changed test code before it ships. Enforce the rules below after the first test-writing pass and before the tests are presented, committed, or merged. Be a sharp reviewer, not a pedantic one: flag what wastes maintenance effort or hides real bugs, ignore cosmetic preferences.
|
||||
|
||||
These rules exist because coding agents over-generate tests. The common failure modes: mock-heavy unit tests that assert implementation details, near-duplicate test bodies that differ by one value, and tests that re-verify the framework instead of the project's logic. Each looks productive in a diff and costs maintenance forever.
|
||||
|
||||
## When to Use
|
||||
|
||||
Use this skill when reviewing generated or changed test code before it ships. Activate it reactively after an agent writes, edits, generates, or refactors tests — unit tests, integration tests, e2e tests, or snapshot tests in any framework.
|
||||
|
||||
## When this skill activates
|
||||
|
||||
- A coding agent has just written new test functions or test files, in any language
|
||||
- You are editing existing tests
|
||||
- You are reviewing a diff that contains test changes
|
||||
- The user asks you to write, add, or review tests
|
||||
|
||||
## Adapt to the project first
|
||||
|
||||
These rules are universal, but their application is not. Before reviewing:
|
||||
|
||||
1. Check the project's own agent instructions (CLAUDE.md, AGENTS.md) and testing docs. Project-specific testing rules win over this skill when they conflict.
|
||||
2. Identify the test stack, then read the matching reference for concrete patterns:
|
||||
- Python / pytest → [references/pytest.md](references/pytest.md)
|
||||
- PHP / PHPUnit / Pest / WordPress → [references/phpunit.md](references/phpunit.md)
|
||||
- JavaScript / TypeScript / Jest / Vitest → [references/jest.md](references/jest.md)
|
||||
3. If the project calls LLM APIs, uses agent frameworks, or wires up observability/telemetry, also read [references/llm-app-testing.md](references/llm-app-testing.md) — it adds three rules specific to LLM applications.
|
||||
4. Map the project's system boundaries: network calls, databases, filesystem, clock and randomness, third-party SDKs, LLM APIs. Existing fixtures and test helpers usually reveal where the project already draws these lines.
|
||||
|
||||
## What to do
|
||||
|
||||
1. Read the test code: the diff, the new file, or the section being modified.
|
||||
2. Check each test against the rules below.
|
||||
3. Report violations concisely: rule number, location, why it violates, suggested fix.
|
||||
4. If the user explicitly invokes this skill before test writing, apply the rules as you write — don't write violations and then flag them.
|
||||
|
||||
When writing new tests, ask for each test: "What specific bug does this catch that no other test in this suite catches?" If you can't answer clearly, don't write it.
|
||||
|
||||
## The Nine Rules
|
||||
|
||||
### Rule 1: Test behavior, not implementation
|
||||
Test what code does from the caller's perspective. Assert return values and observable side effects. Never assert that an internal helper was called with specific arguments — that test breaks on every refactor while catching nothing.
|
||||
|
||||
**Violation pattern:** asserting a mock of an internal function was called, where that function is not a system boundary.
|
||||
**Fix:** assert the return value or the state change the caller observes.
|
||||
|
||||
### Rule 2: Every mock must be justified
|
||||
Mock only at system boundaries: network and HTTP calls, LLM APIs, databases, filesystem I/O on external files, clock and randomness, third-party SDKs. Never mock internal classes or helper functions to isolate a "unit" — the seams you create hide the integration bugs worth catching.
|
||||
|
||||
When you mock a boundary, assert what the caller *does with the response*, not that the mock received specific arguments.
|
||||
|
||||
### Rule 3: One scenario per test, data-driven for variants
|
||||
If two or more tests share identical setup and differ only in input/output values, merge them into one data-driven test (`@pytest.mark.parametrize`, PHPUnit `#[DataProvider]`, Jest `test.each`).
|
||||
|
||||
**When separate tests ARE correct:** different setup, different assertions, different mock configurations, or genuinely different scenarios that happen to exercise the same function.
|
||||
|
||||
### Rule 4: Every test must justify its existence
|
||||
Ask: "What bug does this catch that no other test catches?" Delete tests that only catch typos, verify default values of data classes, or test trivial pass-through logic.
|
||||
|
||||
**Common unjustified tests:** constructors setting attributes, a function rejecting input the type system already forbids, string formatting of log messages, a constant equaling its literal value.
|
||||
|
||||
### Rule 5: Name tests for the scenario
|
||||
Pattern: `test_<scenario>_<expected_outcome>`. The name should read like a requirement, not echo the function signature.
|
||||
|
||||
| Bad | Good |
|
||||
|-----|------|
|
||||
| `test_parse_response_missing_field` | `test_malformed_response_falls_back_to_default` |
|
||||
| `test_get_language_no_class` | `test_element_without_class_returns_empty_language` |
|
||||
| `test_add_tags_single_string` | `test_single_tag_normalizes_to_list` |
|
||||
|
||||
### Rule 6: Production regression tests are sacred
|
||||
Tests that reproduce a real production bug are always justified. Reference the incident (date, issue ID, or short description) in the name or a comment, and never delete them. They are exempt from Rule 4 — their justification is the incident.
|
||||
|
||||
### Rule 7: No tests for framework guarantees
|
||||
Don't test that the validation library validates, the ORM commits, the router returns 404, or the test framework's fixtures work. Test *your* logic that sits on top of the framework.
|
||||
|
||||
**Violation pattern:** a test that would still pass if you deleted all the project's custom code and kept only framework defaults.
|
||||
|
||||
### Rule 8: State and value objects are real, never mocked
|
||||
Never mock a data model, DTO, entity, or state object. Construct a real instance. Mocking state hides field-name typos and validation errors — exactly the bugs worth catching. If constructing the real object is painful, that is design feedback, not a reason to mock; add a small builder or factory helper.
|
||||
|
||||
### Rule 9: Infrastructure under test gets real infrastructure
|
||||
When database queries, schema behavior, or persistence logic *is the subject* of the test, run against a real test database with real migrations applied via fixtures. Mocking the session there tests nothing. Mocking the database is fine when persistence is only a side effect of the behavior under test.
|
||||
|
||||
## Reporting format
|
||||
|
||||
When flagging violations, use this format:
|
||||
|
||||
```
|
||||
**Rule N violation** in `tests/path/file.ext::<test_name>`
|
||||
- What: <one sentence describing the violation>
|
||||
- Fix: <one sentence describing what to do instead>
|
||||
```
|
||||
|
||||
Group violations by file. If a file has no violations, don't mention it.
|
||||
|
||||
## Severity guide
|
||||
|
||||
Not all violations are equal. Use judgment:
|
||||
|
||||
- **Must fix:** Rules 1, 2, 8 — these hide real bugs or make tests brittle
|
||||
- **Should fix:** Rules 3, 4, 5, 7 — these cause bloat and maintenance drag
|
||||
- **Sacred:** Rule 6 — never delete, always allow
|
||||
- **Worth noting:** Rule 9 — test architecture; flag it, but don't block small changes on it
|
||||
|
||||
## References
|
||||
|
||||
- [references/pytest.md](references/pytest.md) — Python/pytest patterns: parametrize, fixtures, mock boundaries, real Pydantic instances
|
||||
- [references/phpunit.md](references/phpunit.md) — PHP/PHPUnit/Pest patterns, including WordPress and WooCommerce test boundaries
|
||||
- [references/jest.md](references/jest.md) — Jest/Vitest patterns: test.each, module mocks, msw, snapshot discipline
|
||||
- [references/llm-app-testing.md](references/llm-app-testing.md) — three extra rules for LLM applications: prompt contracts, observability wiring, agent-flow transitions
|
||||
|
||||
## What this skill does NOT do
|
||||
|
||||
- It does not run tests. Use the project's test runner for that.
|
||||
- It does not enforce code style — that's the linter's job.
|
||||
- It does not decide *what* to test — only *how* to test it.
|
||||
- It does not flag pre-existing violations in files you're not touching, unless asked to audit.
|
||||
@@ -0,0 +1,50 @@
|
||||
# Test Guard — JavaScript / TypeScript / Jest / Vitest Patterns
|
||||
|
||||
Concrete applications of the nine rules for JS/TS projects. Read this when reviewing or writing Jest or Vitest tests.
|
||||
|
||||
## Rule 2: Mock boundaries in JS/TS
|
||||
|
||||
Justified mock targets:
|
||||
|
||||
- Network: prefer `msw` (Mock Service Worker) over `jest.mock`-ing your own fetch wrapper — it mocks at the true boundary
|
||||
- LLM / third-party SDK clients (`openai`, `@anthropic-ai/sdk`, Stripe, etc.)
|
||||
- Timers and randomness: `vi.useFakeTimers()` / `jest.useFakeTimers()`, seeded RNG
|
||||
- Filesystem and process env in Node code
|
||||
|
||||
Unjustified mocks:
|
||||
|
||||
- `jest.mock('../utils/helpers')` — mocking your own internal module to isolate a "unit" (Rule 2)
|
||||
- Mocking a class's private method via prototype patching (Rule 1)
|
||||
- Hand-built object literals pretending to be domain entities when a real constructor or factory exists (Rule 8)
|
||||
|
||||
## Rule 3: test.each
|
||||
|
||||
```ts
|
||||
// Violation: three near-identical it() blocks
|
||||
// Fix:
|
||||
test.each([
|
||||
['Hello World', 'hello-world'],
|
||||
[' padded ', 'padded'],
|
||||
['Café Menu', 'cafe-menu'],
|
||||
])('slugify(%s) → %s', (raw, expected) => {
|
||||
expect(slugify(raw)).toBe(expected);
|
||||
});
|
||||
```
|
||||
|
||||
## Snapshot discipline
|
||||
|
||||
Snapshot tests are implementation tests in disguise unless the snapshot *is* the contract (e.g., a public JSON output, a CLI's help text). Avoid snapshots of:
|
||||
|
||||
- Full component trees that change on every styling tweak (Rule 1 — brittle, asserts implementation)
|
||||
- Large objects nobody reviews — an unread snapshot approves itself (Rule 4)
|
||||
|
||||
Prefer targeted assertions: `expect(screen.getByRole('button')).toHaveTextContent('Save')`.
|
||||
|
||||
## UI component tests
|
||||
|
||||
- Test what the user sees and does (Testing Library queries by role/label), not component internals or state hooks (Rule 1).
|
||||
- Don't test that React renders, routes resolve, or props propagate — framework guarantees (Rule 7).
|
||||
|
||||
## Rule 9: Real persistence
|
||||
|
||||
For data-layer logic (Prisma/Drizzle/Knex queries), run against a real test database — `testcontainers`, a Dockerized Postgres, or SQLite where compatible. Mocking the query builder to test the query builder tests nothing.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Test Guard — LLM Application Rules
|
||||
|
||||
Three additional rules for projects that call LLM APIs, use agent/workflow frameworks (LangGraph, CrewAI, custom state machines), or wire up observability/telemetry (Langfuse, LangSmith, OpenTelemetry). Apply these on top of the nine core rules.
|
||||
|
||||
## Rule 10: Prompt tests — test the contract, not the content
|
||||
|
||||
Prompt text changes constantly; tests pinned to wording rot within a week. Don't assert specific phrasing.
|
||||
|
||||
Do test:
|
||||
|
||||
- The prompt template exists and loads without error (smoke test)
|
||||
- Template variables are substituted correctly — no leftover `{placeholder}` markers
|
||||
- The prompt contains required structural markers *if the caller parses them* (e.g., a JSON schema block, a delimiter the parser splits on)
|
||||
|
||||
## Rule 11: Observability is infrastructure
|
||||
|
||||
Don't unit-test telemetry wiring. The violation pattern is asserting a tracing/analytics mock's call arguments:
|
||||
|
||||
```python
|
||||
# Violation — tests wiring, not behavior
|
||||
mock_tracer.assert_called_once_with(session_id=..., tags=[...])
|
||||
```
|
||||
|
||||
Mocking observability calls to *prevent side effects* during tests is fine and often necessary. Just don't assert on the mock's call args. If telemetry breaks, dashboards show it; a unit test asserting wiring only breaks refactors.
|
||||
|
||||
## Rule 12: Agent and flow tests test transitions
|
||||
|
||||
For agent frameworks and state machines: test that given a state plus an event, the flow reaches the correct next state with the correct fields set. Mock the LLM calls to return controlled responses.
|
||||
|
||||
Test: state in → state out.
|
||||
|
||||
Don't test: the exact prompt string passed to the LLM, the number of LLM calls made, or internal retry logic — those are implementation details (Rule 1) that change with every model upgrade.
|
||||
|
||||
A useful pattern is a table of transition cases (data-driven, Rule 3): starting state, mocked LLM response, expected resulting state.
|
||||
|
||||
## Severity
|
||||
|
||||
- **Must fix:** Rule 12 violations that assert prompt strings or call counts — they break on every model/prompt change
|
||||
- **Should fix:** Rule 10 wording assertions
|
||||
- **Worth noting:** Rule 11 — flag it, but don't block a small change on it
|
||||
@@ -0,0 +1,56 @@
|
||||
# Test Guard — PHP / PHPUnit / Pest Patterns
|
||||
|
||||
Concrete applications of the nine rules for PHP projects, including WordPress and WooCommerce. Read this when reviewing or writing PHP tests.
|
||||
|
||||
## Rule 2: Mock boundaries in PHP
|
||||
|
||||
Justified mock targets:
|
||||
|
||||
- HTTP: Guzzle handlers/middleware, `pre_http_request` filter in WordPress
|
||||
- External SDKs: payment gateways, mail providers, LLM API clients
|
||||
- Clock: inject a clock (`psr/clock`) instead of calling `time()` directly
|
||||
- Filesystem on external paths (prefer `vfsStream` or temp dirs over mocking)
|
||||
|
||||
Unjustified mocks:
|
||||
|
||||
- Mockery/Prophecy doubles for the project's own value objects, DTOs, or entities — construct real instances (Rule 8)
|
||||
- Mocking internal services just to isolate a class — if wiring is painful, fix the constructor, don't fake the collaborator
|
||||
- Partial mocks of the class under test — you are no longer testing the class
|
||||
|
||||
## Rule 3: Data providers
|
||||
|
||||
```php
|
||||
/**
|
||||
* @see Rule 3 — variants of one scenario belong in a data provider.
|
||||
*/
|
||||
#[DataProvider('provideSlugCases')]
|
||||
public function test_slugify_normalizes_input( string $raw, string $expected ): void {
|
||||
$this->assertSame( $expected, slugify( $raw ) );
|
||||
}
|
||||
|
||||
public static function provideSlugCases(): array {
|
||||
return array(
|
||||
'lowercases' => array( 'Hello World', 'hello-world' ),
|
||||
'strips padding' => array( ' padded ', 'padded' ),
|
||||
'transliterates' => array( 'Café Menu', 'cafe-menu' ),
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
Pest equivalent: `it('normalizes slug', ...)->with([...])`.
|
||||
|
||||
## WordPress-specific boundaries
|
||||
|
||||
- **Integration tests** (`WP_UnitTestCase` / `wp-env` / `wp-cli scaffold`): use the real WordPress test framework with factories — `self::factory()->post->create()`, `self::factory()->user->create()`. Don't mock `WP_Post` or `WP_User`; the factories exist precisely so you don't have to (Rule 8).
|
||||
- **Unit tests without WordPress loaded** (Brain Monkey / WP_Mock): mocking WordPress functions like `get_option()` or `apply_filters()` is a boundary mock and justified. But assert what *your code does* with the values, not that `get_option` was called with specific args (Rule 1).
|
||||
- Mock outbound HTTP with the `pre_http_request` filter rather than patching `wp_remote_get` internals.
|
||||
- Don't test that WordPress sanitizes, escapes, or that hooks fire — that's core's guarantee (Rule 7). Test your callback's behavior given an input.
|
||||
|
||||
## WooCommerce notes
|
||||
|
||||
- Build real `WC_Product` / `WC_Order` objects via `WC_Helper_Product` and `WC_Helper_Order` in integration tests — never `MagicMock`-style doubles of them (Rule 8).
|
||||
- Cart and checkout logic is stateful: prefer integration tests over heavily mocked unit tests; mocked carts hide hook-ordering bugs.
|
||||
|
||||
## Rule 9: Real database
|
||||
|
||||
`WP_UnitTestCase` already wraps each test in a transaction against a real schema — use it for query, meta, and persistence logic instead of mocking `$wpdb`. Mocking `$wpdb->prepare` or `$wpdb->get_results` to test a query builder tests nothing.
|
||||
@@ -0,0 +1,65 @@
|
||||
# Test Guard — Python / pytest Patterns
|
||||
|
||||
Concrete applications of the nine rules for pytest projects. Read this when reviewing or writing Python tests.
|
||||
|
||||
## Rule 2: Mock boundaries in Python
|
||||
|
||||
Justified mock targets:
|
||||
|
||||
- HTTP clients: `httpx`, `requests`, `aiohttp` (or use `respx` / `responses` instead of raw mocks)
|
||||
- LLM SDK calls: `openai`, `anthropic`, `litellm.completion` and friends
|
||||
- Database sessions, when the database is not the subject (see Rule 9)
|
||||
- Filesystem I/O on external paths (`tmp_path` fixture is often better than mocking)
|
||||
- Clock and randomness: `time.time`, `datetime.now`, `random` (prefer `freezegun` or injected clocks)
|
||||
|
||||
Unjustified mocks (common agent-generated violations):
|
||||
|
||||
- `MagicMock()` standing in for a Pydantic model or dataclass — construct the real thing
|
||||
- Mocking internal utility functions to isolate a "unit"
|
||||
- Mocking `json.loads` / `json.dumps` or other stdlib pure functions
|
||||
|
||||
## Rule 3: Parametrize
|
||||
|
||||
```python
|
||||
# Violation: three copy-pasted tests differing by one value
|
||||
def test_slug_lowercase(): ...
|
||||
def test_slug_strips_spaces(): ...
|
||||
def test_slug_handles_unicode(): ...
|
||||
|
||||
# Fix
|
||||
@pytest.mark.parametrize(
|
||||
("raw", "expected"),
|
||||
[
|
||||
("Hello World", "hello-world"),
|
||||
(" padded ", "padded"),
|
||||
("Café Menu", "cafe-menu"),
|
||||
],
|
||||
)
|
||||
def test_slugify_normalizes_input(raw, expected):
|
||||
assert slugify(raw) == expected
|
||||
```
|
||||
|
||||
## Rule 8: Real Pydantic/dataclass instances
|
||||
|
||||
```python
|
||||
# Wrong — hides field typos and validation errors
|
||||
state = MagicMock()
|
||||
state.user_id = "123"
|
||||
state.status = "ACTIVE"
|
||||
|
||||
# Right — Pydantic validates the construction itself
|
||||
state = UserState(user_id="123", status="ACTIVE")
|
||||
```
|
||||
|
||||
If a model needs many fields, add a factory fixture or use `factory_boy` — don't fall back to `MagicMock`.
|
||||
|
||||
## Rule 9: Real database via fixtures
|
||||
|
||||
Use a fixture that applies real migrations (e.g., a session-scoped test database with `alembic upgrade head`), and function-scoped transactions rolled back per test. `pytest-postgresql`, `testcontainers`, or an SQLite-compatible fallback all work; the point is real schema, not a mocked session, whenever query or persistence logic is the subject.
|
||||
|
||||
## pytest-specific smells
|
||||
|
||||
- `assert mock.call_count == N` on anything internal — Rule 1 violation
|
||||
- `@patch` stacks three or more deep — the test is coupled to implementation; restructure
|
||||
- Asserting log output via `caplog` for messages no caller parses — Rule 4 violation
|
||||
- Fixtures that build mocks of project classes — Rule 8 violation, make the fixture build real objects
|
||||
Reference in New Issue
Block a user