67 lines
2.6 KiB
Markdown
67 lines
2.6 KiB
Markdown
# Confidence Calibration
|
|
|
|
Confidence measures how strongly the available evidence supports a codebase
|
|
finding. It does not measure requirement clarity or implementation readiness.
|
|
|
|
## Calibration Dimensions
|
|
|
|
Consider these dimensions together; do not average them mechanically:
|
|
|
|
- **Directness**: runtime observation and source code outweigh inference.
|
|
- **Coverage**: evidence spans the relevant execution paths and modules.
|
|
- **Corroboration**: code, tests, documentation, and history agree.
|
|
- **Counter-evidence**: plausible alternatives and contradictory cases were checked.
|
|
- **Remaining gaps**: unknown behavior is bounded and stated explicitly.
|
|
|
|
## Typical Levels
|
|
|
|
| Level | Evidence state |
|
|
| --- | --- |
|
|
| 0 | No repository evidence collected yet. |
|
|
| 1 | Broad structure or search results only. |
|
|
| 2 | Relevant code located, but behavior or scope is not verified. |
|
|
| 3 | Main path traced with some corroboration; meaningful gaps remain. |
|
|
| 4 | Multiple sources support the finding and counter-examples were checked. |
|
|
| 5 | Relevant scope is covered, alternatives are ruled out, and conclusions are directly supported. |
|
|
|
|
A clear user request can narrow the investigation but cannot raise confidence in
|
|
a finding by itself. Documentation alone normally remains at level 1-2. A
|
|
single code path without tests, history, or runtime confirmation normally
|
|
remains at level 2-3.
|
|
|
|
## Examples
|
|
|
|
### Focused Question, Strong Evidence
|
|
|
|
Question: "Does deleting this adapter break the HTTP client?"
|
|
|
|
Evidence: imports show no production consumer, tests cover the replacement
|
|
adapter, and history confirms the old adapter was retained only for migration.
|
|
After checking dynamic loading and configuration references, report at level
|
|
4-5. Cite each source and distinguish observed facts from the final inference.
|
|
|
|
### Cross-Module Finding, Incomplete Evidence
|
|
|
|
Question: "Why do all write operations depend on the legacy package?"
|
|
|
|
Evidence: searches show imports in several modules, but runtime registration
|
|
and generated code have not been inspected. Stay at level 2-3. Continue tracing
|
|
composition roots and tests before claiming an architectural dependency.
|
|
|
|
### Early Delivery
|
|
|
|
The user asks for findings before runtime behavior or history can be checked.
|
|
Deliver the verified portion at its current level and include `△ Caveats` with:
|
|
|
|
- the evidence already inspected;
|
|
- assumptions required by the conclusion;
|
|
- unverified modules or execution paths;
|
|
- the next checks that would materially change confidence.
|
|
|
|
Do not present the early report as level 5 even when the likely explanation is
|
|
simple.
|
|
|
|
## Delivery Check
|
|
|
|
Before assigning level 4 or 5, confirm that:
|