86 lines
3.8 KiB
Diff
86 lines
3.8 KiB
Diff
diff --git a/skills/thirdparty/codebase-recon/SKILL.md b/skills/thirdparty/codebase-recon/SKILL.md
|
|
--- a/skills/thirdparty/codebase-recon/SKILL.md
|
|
+++ b/skills/thirdparty/codebase-recon/SKILL.md
|
|
@@ -252 +252 @@ Core methodology:
|
|
-- [confidence.md](../pathfinding/references/confidence.md) — confidence calibration (shared with pathfinding)
|
|
+- [confidence-calibration.md](references/confidence-calibration.md) — evidence-based confidence calibration
|
|
@@ -263 +263,0 @@ Related skills:
|
|
-- `outfitter:pathfinding` — clarifying requirements before analysis
|
|
diff --git a/skills/thirdparty/codebase-recon/references/confidence-calibration.md b/skills/thirdparty/codebase-recon/references/confidence-calibration.md
|
|
new file mode 100644
|
|
--- /dev/null
|
|
+++ b/skills/thirdparty/codebase-recon/references/confidence-calibration.md
|
|
@@ -0,0 +1,66 @@
|
|
+# Confidence Calibration
|
|
+
|
|
+Confidence measures how strongly the available evidence supports a codebase
|
|
+finding. It does not measure requirement clarity or implementation readiness.
|
|
+
|
|
+## Calibration Dimensions
|
|
+
|
|
+Consider these dimensions together; do not average them mechanically:
|
|
+
|
|
+- **Directness**: runtime observation and source code outweigh inference.
|
|
+- **Coverage**: evidence spans the relevant execution paths and modules.
|
|
+- **Corroboration**: code, tests, documentation, and history agree.
|
|
+- **Counter-evidence**: plausible alternatives and contradictory cases were checked.
|
|
+- **Remaining gaps**: unknown behavior is bounded and stated explicitly.
|
|
+
|
|
+## Typical Levels
|
|
+
|
|
+| Level | Evidence state |
|
|
+| --- | --- |
|
|
+| 0 | No repository evidence collected yet. |
|
|
+| 1 | Broad structure or search results only. |
|
|
+| 2 | Relevant code located, but behavior or scope is not verified. |
|
|
+| 3 | Main path traced with some corroboration; meaningful gaps remain. |
|
|
+| 4 | Multiple sources support the finding and counter-examples were checked. |
|
|
+| 5 | Relevant scope is covered, alternatives are ruled out, and conclusions are directly supported. |
|
|
+
|
|
+A clear user request can narrow the investigation but cannot raise confidence in
|
|
+a finding by itself. Documentation alone normally remains at level 1-2. A
|
|
+single code path without tests, history, or runtime confirmation normally
|
|
+remains at level 2-3.
|
|
+
|
|
+## Examples
|
|
+
|
|
+### Focused Question, Strong Evidence
|
|
+
|
|
+Question: "Does deleting this adapter break the HTTP client?"
|
|
+
|
|
+Evidence: imports show no production consumer, tests cover the replacement
|
|
+adapter, and history confirms the old adapter was retained only for migration.
|
|
+After checking dynamic loading and configuration references, report at level
|
|
+4-5. Cite each source and distinguish observed facts from the final inference.
|
|
+
|
|
+### Cross-Module Finding, Incomplete Evidence
|
|
+
|
|
+Question: "Why do all write operations depend on the legacy package?"
|
|
+
|
|
+Evidence: searches show imports in several modules, but runtime registration
|
|
+and generated code have not been inspected. Stay at level 2-3. Continue tracing
|
|
+composition roots and tests before claiming an architectural dependency.
|
|
+
|
|
+### Early Delivery
|
|
+
|
|
+The user asks for findings before runtime behavior or history can be checked.
|
|
+Deliver the verified portion at its current level and include `△ Caveats` with:
|
|
+
|
|
+- the evidence already inspected;
|
|
+- assumptions required by the conclusion;
|
|
+- unverified modules or execution paths;
|
|
+- the next checks that would materially change confidence.
|
|
+
|
|
+Do not present the early report as level 5 even when the likely explanation is
|
|
+simple.
|
|
+
|
|
+## Delivery Check
|
|
+
|
|
+Before assigning level 4 or 5, confirm that:
|
|
+
|
|
+- more than one relevant evidence source supports the conclusion;
|
|
+- direct observations and inferences are labeled separately;
|
|
+- at least one plausible counter-example was considered;
|
|
+- the investigated scope matches the wording of the conclusion;
|
|
+- remaining uncertainty is absent or explicitly bounded.
|