📦 deps(thirdparty): update snapshots

This commit is contained in:
ci[bot]
2026-07-18 00:02:59 +00:00
parent 82f7c6e56a
commit 47ce7f78dc
1446 changed files with 141041 additions and 6442 deletions
@@ -0,0 +1,40 @@
# Test Guard — LLM Application Rules
Three additional rules for projects that call LLM APIs, use agent/workflow frameworks (LangGraph, CrewAI, custom state machines), or wire up observability/telemetry (Langfuse, LangSmith, OpenTelemetry). Apply these on top of the nine core rules.
## Rule 10: Prompt tests — test the contract, not the content
Prompt text changes constantly; tests pinned to wording rot within a week. Don't assert specific phrasing.
Do test:
- The prompt template exists and loads without error (smoke test)
- Template variables are substituted correctly — no leftover `{placeholder}` markers
- The prompt contains required structural markers *if the caller parses them* (e.g., a JSON schema block, a delimiter the parser splits on)
## Rule 11: Observability is infrastructure
Don't unit-test telemetry wiring. The violation pattern is asserting a tracing/analytics mock's call arguments:
```python
# Violation — tests wiring, not behavior
mock_tracer.assert_called_once_with(session_id=..., tags=[...])
```
Mocking observability calls to *prevent side effects* during tests is fine and often necessary. Just don't assert on the mock's call args. If telemetry breaks, dashboards show it; a unit test asserting wiring only breaks refactors.
## Rule 12: Agent and flow tests test transitions
For agent frameworks and state machines: test that given a state plus an event, the flow reaches the correct next state with the correct fields set. Mock the LLM calls to return controlled responses.
Test: state in → state out.
Don't test: the exact prompt string passed to the LLM, the number of LLM calls made, or internal retry logic — those are implementation details (Rule 1) that change with every model upgrade.
A useful pattern is a table of transition cases (data-driven, Rule 3): starting state, mocked LLM response, expected resulting state.
## Severity
- **Must fix:** Rule 12 violations that assert prompt strings or call counts — they break on every model/prompt change
- **Should fix:** Rule 10 wording assertions
- **Worth noting:** Rule 11 — flag it, but don't block a small change on it