📦 deps(thirdparty): update snapshots

This commit is contained in:
ci[bot]
2026-07-13 07:44:54 +00:00
parent cb3f8713ee
commit b3cc57caff
92 changed files with 12712 additions and 97 deletions
@@ -0,0 +1,54 @@
# lore scripts
Cross-platform Python 3.6+ helpers that reduce repetitive mechanical work. No third-party dependencies. Called by `init` / `sync` / `audit` / `compress` / `lore mirror`; can also be run standalone for ad-hoc inspection.
The script list and quick-reference command examples live in the project root `README.md` "Scripts" section. This file covers the things that don't fit there: design intent, integration points, and limits.
## Design notes
**Cross-platform first.** Python standard library only. No `bash`, no `jq`, no platform-specific tools. The same invocation works on Windows, Linux, macOS.
**JSON-friendly output.** Every script supports `--json` for machine consumption. Agent callers parse the output; humans pipe to `less` or `jq` (if available).
**Composition.** `find_duplicates.py` and `find_stale.py` shell out to `list_entries.py --json` rather than re-implementing the parser. One source of truth for entry format — if the format ever changes, only `list_entries.py` needs updating.
**Read-only by default.** None of these scripts write to `.lore/`. They observe; the agent decides what to do with findings.
**Run from project root.** `list_entries.py` walks up the directory tree looking for `.lore/`. The other scripts depend on it via subprocess, so the same constraint applies transitively.
## When each script is called
| Script | Call site | Purpose |
|---|---|---|
| `history.py` | lore history | List git commits related to a memory entry / file / scope |
| `id_hash.py` | Any time a new entry is written (init / sync) | Compute the 4-char content hash for the entry ID |
| `list_entries.py` | Pre-step of query / audit / compress | Enumerate all entries as JSON for downstream processing |
| `find_duplicates.py` | sync step 5 (de-duplication) | Identify candidate duplicate entries before writing |
| `find_stale.py` | audit step 2; compress step 2; lore mirror (optional) | Identify entries past the verified-date threshold or already marked `#stale` |
## Output channels
**stdout is the data channel; stderr is the warning channel.** All scripts follow this split so `--json` consumers never have to filter noise out of their parsers. Currently `list_entries.py` is the only script that emits a warning:
- `[WARN] .lore/.config.json has no schema_version field.` — fires once per invocation when the config file exists but lacks the version field. Add `"schema_version": 1` to silence it.
- `[WARN] .lore/.config.json#schema_version=N is newer than this lore skill expects (max: 1).` — fires when the config version exceeds what this skill understands. Pull the latest lore from upstream.
Both warnings are informational; `list_entries.py` always produces the same stdout regardless of config state. See `references/compatibility.md` for the full schema versioning policy.
## Testing
Without a real `.lore/`, you can sanity-check that imports and argument parsing work:
```bash
python scripts/id_hash.py "test entry"
python scripts/list_entries.py # should print "(no entries)" or exit with a clear error
```
`list_entries.py`, `find_duplicates.py`, and `find_stale.py` require a populated `.lore/` to produce meaningful output. Set one up via `lore init` first.
## Limitations
- **Token-overlap dedup, not semantic.** Jaccard similarity catches rewrites with similar words but misses semantic equivalence (e.g. "use TypeScript" vs "TypeScript-only codebase"). Deeper checks still need an LLM pass.
- **Naive date math.** `find_stale.py` uses wall-clock dates from `#verified` / `#added` tags. If the system's clock is wrong, results will be off.
- **No automatic archive promotion.** The script reports pending-archive entries but does not move them. Use `lore sync` to actually relocate to `.lore/archive/`.
- **Hash collisions on identical text are theoretically possible** (4 hex chars = 16 bits = 1 in 65536). In practice a lore project will not hit this. If it does, slightly edit the entry text to bump the hash.
@@ -0,0 +1,54 @@
# lore 脚本
跨平台 Python 3.6+ 辅助脚本,减少重复的机械工作。无第三方依赖。被 `init` / `sync` / `audit` / `compress` / `lore mirror` 调用,也可独立运行做临时检查。
脚本清单和命令速查在仓库根 `README.md` 的"Scripts"章节里。本文件覆盖根 README 不适合放的内容:设计意图、集成点、局限。
## 设计要点
**优先跨平台。** 仅使用 Python 标准库,不依赖 `bash``jq` 或任何平台特定工具。Windows / Linux / macOS 行为完全一致。
**JSON 友好输出。** 每个脚本都支持 `--json` 便于机器消费。Agent 调用方解析输出;人类可以直接 `less``jq`(如果装了)。
**组合而非重复。** `find_duplicates.py``find_stale.py` 通过 `list_entries.py --json` 复用解析器,不重复实现 entry 格式解析。Entry 格式只在一处定义——将来格式变更只需改 `list_entries.py`
**默认只读。** 这些脚本不写 `.lore/`,只观察。Agent 决定如何处理发现的问题。
**从项目根目录运行。** `list_entries.py` 向上遍历定位 `.lore/`。其他脚本通过 subprocess 调用它,所以这个约束会传递生效。
## 何时调用
| 脚本 | 调用点 | 用途 |
|---|---|---|
| `history.py` | lore history | 列出与 memory entry / file / scope 相关的 git commits |
| `id_hash.py` | 写新 entry 时(init / sync| 计算 entry ID 的 4 字符内容 hash |
| `list_entries.py` | query / audit / compress 的预步骤 | 把所有 entry 枚举为 JSON 供后续处理 |
| `find_duplicates.py` | sync 步骤 5(去重)| 写之前找出可能的重复 entry |
| `find_stale.py` | audit 步骤 2compress 步骤 2lore mirror(可选)| 找出过期 entry 或已标记 `#stale` 的 entry |
## 输出通道
**stdout 是数据通道;stderr 是警告通道。** 所有脚本遵循这个分离,这样 `--json` 消费者就不必从解析结果里过滤噪音。当前只有 `list_entries.py` 会发警告:
- `[WARN] .lore/.config.json has no schema_version field.` —— 配置文件存在但缺 `schema_version` 字段时,每个调用触发一次。加 `"schema_version": 1` 即可消除。
- `[WARN] .lore/.config.json#schema_version=N is newer than this lore skill expects (max: 1).` —— 配置版本超过本 skill 能理解的范围时触发。从上游 pull 最新 lore。
两条警告都是告知性质;`list_entries.py` 不管配置状态如何,stdout 输出始终一致。完整 schema 版本策略见 `references/compatibility.md`
## 测试
没有真实 `.lore/` 时,可以快速验证 import 和参数解析是否正常:
```bash
python scripts/id_hash.py "test entry"
python scripts/list_entries.py # 应输出 "(no entries)" 或清晰报错
```
`list_entries.py``find_duplicates.py``find_stale.py` 需要有内容的 `.lore/` 才能产出有意义的输出。先用 `lore init` 建一个。
## 局限
- **去重只到词袋重叠程度。** Jaccard 相似度能抓到词汇相似的改写,但抓不到语义等价(如 "use TypeScript" vs "TypeScript-only codebase")。更深的检查仍需 LLM 介入。
- **日期计算比较朴素。** `find_stale.py` 直接用 `#verified` / `#added` 标签的日期。如果系统时钟不对,结果会偏差。
- **不自动 archive。** 脚本会报告待 archive 的 entry,但不会移动它们。实际搬迁到 `.lore/archive/` 仍需通过 `lore sync` 完成。
- **理论上可能有 hash 冲突**(4 个十六进制字符 = 16 位 = 1/65536 概率)。实际项目基本不会遇到。如果遇到了,对 entry 文本做微调以改变 hash。
@@ -0,0 +1,211 @@
#!/usr/bin/env python3
"""Find potential duplicate entries in .lore/.
Usage:
python find_duplicates.py # default threshold 0.7
python find_duplicates.py --threshold=0.85
python find_duplicates.py --json
python find_duplicates.py --candidate "<text>"
python find_duplicates.py --candidate-file path/to/candidate.txt
echo '<text>' | python find_duplicates.py --candidate-stdin
Detection strategies:
1. Identical hash suffix (4 chars after the date) — these are exact
text matches and indicate either a real duplicate or a hash
collision. Always reported.
2. Token-based Jaccard similarity above `--threshold` on the entry
text. Catches rewrites that mean the same thing but produce a
different hash (e.g. "use Zustand" vs "we chose Zustand").
Output is sorted by similarity (descending). Run from the project root.
This script is the mechanical part of `sync` step 5 (de-duplication).
The agent still decides what to do with each pair.
When a candidate is supplied (via --candidate, --candidate-file, or
--candidate-stdin), the candidate is also included in the comparison
set so sync step 5 can detect "this proposed entry duplicates an
existing one" before appending. Without a candidate, only
already-appended entries are compared.
"""
import json
import re
import subprocess
import sys
from pathlib import Path
def get_entries():
"""Invoke list_entries.py --json to get parsed entries."""
script = Path(__file__).parent / "list_entries.py"
r = subprocess.run(
[sys.executable, str(script), "--json"],
capture_output=True,
text=True,
)
if r.returncode != 0:
print(r.stderr, file=sys.stderr)
sys.exit(1)
return json.loads(r.stdout)
def read_candidate(args):
"""Return the candidate text or None.
Sources, in priority order:
1. --candidate "<text>"
2. --candidate-file <path>
3. --candidate-stdin (reads entire stdin)
"""
inline = None
file_path = None
use_stdin = False
i = 0
while i < len(args):
a = args[i]
if a.startswith("--candidate="):
inline = a.split("=", 1)[1]
elif a.startswith("--candidate-file="):
file_path = a.split("=", 1)[1]
elif a in ("--candidate", "--candidate-file"):
if i + 1 >= len(args) or args[i + 1].startswith("--"):
die(2, f"{a} requires a value")
i += 1
if a == "--candidate":
inline = args[i]
else:
file_path = args[i]
elif a == "--candidate-stdin":
use_stdin = True
i += 1
if inline is not None:
return inline
if file_path is not None:
try:
return Path(file_path).read_text(encoding="utf-8")
except OSError as exc:
die(2, f"failed to read candidate file {file_path}: {exc}")
if use_stdin:
if sys.stdin.isatty():
die(2, "--candidate-stdin given but stdin is a TTY")
return sys.stdin.read()
return None
def die(code, message):
print(f"error: {message}", file=sys.stderr)
sys.exit(code)
def synthetic_candidate_entry(text):
"""Build a candidate entry dict shaped like list_entries.py output.
The synthetic entry has layer "CANDIDATE" so it compares only against
existing entries on the same layer when the agent supplies --layer.
"""
return {
"id": "CANDIDATE-unsaved",
"layer": "CANDIDATE",
"scope": "_candidate",
"file": "<candidate>",
"text": text.strip(),
"tags": {},
}
def tokenize(text: str):
return set(re.findall(r"\w+", text.lower()))
def jaccard(a: set, b: set):
if not a or not b:
return 0.0
return len(a & b) / len(a | b)
def hash_suffix(eid: str):
return eid.split("-")[-1]
def main():
args = sys.argv[1:]
threshold = 0.7
json_output = "--json" in args
layer_filter = None
for arg in args:
if arg.startswith("--threshold="):
threshold = float(arg.split("=", 1)[1])
elif arg.startswith("--layer="):
layer_filter = arg.split("=", 1)[1]
candidate_text = read_candidate(args)
entries = get_entries()
if layer_filter is not None:
entries = [e for e in entries if e.get("layer") == layer_filter]
candidates = []
if candidate_text:
candidates.append(synthetic_candidate_entry(candidate_text))
pairs = []
# existing-vs-existing pairs (unchanged behavior)
for i, a in enumerate(entries):
for b in entries[i + 1:]:
if a["layer"] != b["layer"]:
continue
if hash_suffix(a["id"]) == hash_suffix(b["id"]):
pairs.append((a, b, 1.0, "identical hash"))
continue
sim = jaccard(tokenize(a["text"]), tokenize(b["text"]))
if sim >= threshold:
pairs.append((a, b, sim, f"similar text (≥{threshold})"))
# candidate-vs-existing pairs
if candidates:
# --layer narrows entries above; without it, compare the proposed
# entry with every layer because the candidate has not been assigned
# a canonical layer yet.
compare_set = entries
for a in compare_set:
sim = jaccard(
tokenize(candidates[0]["text"]),
tokenize(a["text"]),
)
if sim >= threshold:
pairs.append((candidates[0], a, sim,
f"candidate similar to existing (≥{threshold})"))
pairs.sort(key=lambda x: -x[2])
if json_output:
out = [
{
"similarity": round(sim, 3),
"reason": reason,
"a": a,
"b": b,
}
for a, b, sim, reason in pairs
]
print(json.dumps(out, indent=2, ensure_ascii=False))
return
if not pairs:
if candidate_text:
print("No potential duplicates found for the candidate.")
else:
print("No potential duplicates found.")
return
for a, b, sim, reason in pairs:
print(f"[{sim:.2f}] {reason}")
print(f" A: [{a['file']}] {a['id']} {a['text']}")
print(f" B: [{b['file']}] {b['id']} {b['text']}")
print()
if __name__ == "__main__":
main()
@@ -0,0 +1,116 @@
#!/usr/bin/env python3
"""Find stale entries in .lore/.
Usage:
python find_stale.py # default: 90-day threshold
python find_stale.py --days=180
python find_stale.py --json
Reports two categories:
Stale : entry has not been `#verified` within the threshold
(or has no #verified at all, and was added > threshold
days ago).
Pending arch : entry already carries a `#stale:` tag and is waiting
to be moved into .lore/archive/.
Output is plain text by default, JSON with --json.
Used by:
- `audit` workflow (read-only)
- `compress` workflow (advisory)
- `lore mirror` (sanity check before regenerating)
"""
import json
import subprocess
import sys
from datetime import date, datetime, timedelta
from pathlib import Path
def get_entries():
script = Path(__file__).parent / "list_entries.py"
r = subprocess.run(
[sys.executable, str(script), "--json"],
capture_output=True,
text=True,
)
if r.returncode != 0:
print(r.stderr.strip(), file=sys.stderr)
sys.exit(1)
try:
return json.loads(r.stdout)
except json.JSONDecodeError as exc:
print(f"error: list_entries.py returned invalid JSON: {exc}",
file=sys.stderr)
sys.exit(1)
def parse_date(s: str):
try:
return datetime.strptime(s, "%Y-%m-%d").date()
except (ValueError, TypeError):
return None
def main():
days = 90
json_output = "--json" in sys.argv[1:]
for arg in sys.argv[1:]:
if arg.startswith("--days="):
days = int(arg.split("=", 1)[1])
today = date.today()
cutoff = today - timedelta(days=days)
entries = get_entries()
stale = []
pending_arch = []
for e in entries:
# Already marked stale → pending archive
if "stale" in e["tags"]:
pending_arch.append(e)
continue
# Determine the entry's freshness date
last_v = parse_date(e["last_verified"])
added = parse_date(e["tags"].get("added"))
ref_date = last_v or added
if ref_date is None:
continue # no date info, can't decide
if ref_date < cutoff:
stale.append(e)
if json_output:
out = {
"threshold_days": days,
"as_of": today.isoformat(),
"stale": stale,
"pending_archive": pending_arch,
}
print(json.dumps(out, indent=2, ensure_ascii=False))
return
print(f"=== Stale (unverified > {days} days, as of {today}) ===")
if not stale:
print(" (none)")
for e in stale:
ref = e["last_verified"] or e["tags"].get("added", "unknown")
print(f" [{e['file']}] {e['id']} {e['text']}")
print(f" ref date: {ref}")
print()
print("=== Pending archive (tagged #stale) ===")
if not pending_arch:
print(" (none)")
for e in pending_arch:
print(f" [{e['file']}] {e['id']} {e['text']}")
print(f" marked stale: {e['tags']['stale']}")
if __name__ == "__main__":
main()
@@ -0,0 +1,527 @@
#!/usr/bin/env python3
"""`lore history` — list git commits related to an entry, file, or scope.
Usage:
lore history <entry-id>
lore history <file-path>
lore history --scope=<name>
lore history --since=<YYYY-MM-DD>
lore history --json
See references/history-command.md for the full specification.
"""
import re
import subprocess
import sys
from pathlib import Path
import json as _json # standard library; aliased to avoid clashing with future vars
# Entry ID pattern: LAYER-YYYY-MM-DD-xxxx (4 hex chars)
ENTRY_ID_RE = re.compile(r"^[A-Z]+-\d{4}-\d{2}-\d{2}-[a-f0-9]{4}$")
def parse_arg(arg: str):
"""Dispatch the first positional argument to entry / file / scope form.
Returns a dict {"form": "entry"|"file"|"scope", "value": str}, or None
if the argument matches none of the recognized patterns.
"""
if not arg:
return None
if arg.startswith("--scope="):
return {"form": "scope", "value": arg.split("=", 1)[1]}
if ENTRY_ID_RE.match(arg):
return {"form": "entry", "value": arg}
if "/" in arg or arg.startswith("."):
return {"form": "file", "value": arg}
return None
def find_entry(entries, entry_id):
"""Look up an entry by ID in the list from list_entries.py --json.
Returns the entry dict, or None if not found.
"""
for e in entries:
if e.get("id") == entry_id:
return e
return None
def extract_added_date(tags):
"""Return the value of the 'added' tag, or None if absent.
The entry dict's `tags` field is {name: value, ...} as produced
by list_entries.py.
"""
if not tags:
return None
return tags.get("added")
# Match a backtick-quoted path inside an entry's text. The path must
# contain at least one slash OR start with a dot OR end with a common
# code extension, to avoid false positives like `Zustand`.
BACKTICK_PATH_RE = re.compile(
r"`([^\s`]+\.[a-zA-Z0-9]{1,8}(?:\.[a-zA-Z0-9]{1,8})*"
r"|[^\s`]+/[^\s`]+"
r"|\.[a-zA-Z][^\s`]*)`"
)
def resolve_code_file(entry):
"""Decide which file path to git-log for this entry.
Priority:
1. First backtick-quoted path in entry.text (looks like a file).
2. Scope directory at project root (e.g. "frontend" for scope "frontend").
3. "." for the _global scope (project root).
The path returned is relative to the project root. git log handles
"." to mean the whole repo.
"""
if entry.get("text"):
m = BACKTICK_PATH_RE.search(entry["text"])
if m:
return m.group(1)
scope = entry.get("scope", "_global")
if scope == "_global":
return "."
return scope
# Single-line per commit. The trailing %s for body is multi-line content
# that we capture separately (not in the delimited format string) by
# running a second pass with a different format. For v1 we use a simple
# format and parse body via a follow-up `git show` only if needed.
#
# To keep parsing simple, we use a delimiter unlikely to appear in real
# commit metadata: ASCII Unit Separator (\x1f).
COMMIT_DELIM = "\x1f"
# git log format: hash\x1fauthor\x1fdate(iso)\x1fsubject
# We use %x1f (the same delimiter) inline so the format string is portable.
# The body is fetched separately via the second invocation below.
FORMAT_STRING = "%H%x1f%an%x1f%ai%x1f%s"
def run_git_log(project_root, since, code_file, n=None):
"""Run `git log` and return a list of commit dicts.
Args:
project_root: Path to the git repo root.
since: ISO date string, or None for full history.
code_file: Path relative to project_root to filter by.
n: Optional int cap on number of commits.
Returns:
List of dicts as produced by parse_commit_line + body-fetch.
Raises:
RuntimeError: if git exits non-zero or is missing.
"""
cmd = [
"git",
"-C", str(project_root),
"log",
f"--pretty=format:{FORMAT_STRING}",
]
if since:
cmd.append(f"--since={since}")
if n is not None:
cmd.append(f"-n{n}")
cmd.extend(["--", code_file])
try:
proc = subprocess.run(
cmd,
capture_output=True,
text=True,
encoding="utf-8",
errors="replace",
check=False,
)
except FileNotFoundError as exc:
raise RuntimeError(f"git executable not found on PATH: {exc}")
if proc.returncode != 0:
raise RuntimeError(f"git log failed: {proc.stderr.strip()}")
commits = []
for line in proc.stdout.splitlines():
if not line:
continue
parsed = parse_commit_line(line)
if parsed is None:
continue
parsed["body"] = "" # filled in by fetch_body if requested later
commits.append(parsed)
return commits
def parse_commit_line(line):
"""Parse one delimited git log line. Returns dict or None on malformed input."""
parts = line.split(COMMIT_DELIM)
if len(parts) != 4:
return None
full_hash, author, date, subject = parts
if len(full_hash) < 7:
return None
return {
"hash": full_hash,
"short": full_hash[:7],
"author": author,
"date": date[:10], # take YYYY-MM-DD from full ISO timestamp
"subject": subject,
"body": "", # populated by fetch_commit_body
}
# Match PR/issue references. Order matters: longer keywords first so
# "Closes" doesn't get eaten by "#NNN" alone. We require word boundary
# (or start of string) before the keyword to avoid matching substrings
# like "address#N" mid-word.
REFS_RE = re.compile(
r"(?:\(|\b(?:Closes|Refs|Fixes|Resolves)\s+)"
r"(#\d+)",
re.IGNORECASE,
)
def extract_refs(message):
"""Return a list of PR/issue references found in a commit message.
Each item is either "#NNN" (from parens form) or "Keyword #NNN"
(from Closes/Refs/Fixes/Resolves form). Duplicates are removed
in order of appearance.
"""
matches = []
seen = set()
for m in REFS_RE.finditer(message):
prefix = m.group(0).split("#")[0]
ref = "#" + m.group(1)[1:] # normalize to "#NNN"
if ref in seen:
continue
seen.add(ref)
if prefix.startswith("("):
matches.append(ref)
else:
matches.append(f"{prefix.strip()} {ref}")
return matches
def truncate_body(body, max_lines=3):
"""Trim a multi-line string to at most `max_lines`, stripping blank tails.
Used to keep commit bodies short in the Markdown output. The subject
is already shown separately; the body is supplementary context.
"""
lines = body.splitlines()
trimmed = lines[:max_lines]
while trimmed and not trimmed[-1].strip():
trimmed.pop()
return "\n".join(trimmed)
def fetch_commit_body(project_root, commit_hash):
"""Fetch the full commit message (subject + body) via `git show`.
Returns a string with the subject as the first line and the body
(if any) following a blank line. Trailing blank lines are removed.
"""
cmd = [
"git", "-C", str(project_root),
"show", "-s", "--format=%B", commit_hash,
]
try:
proc = subprocess.run(
cmd, capture_output=True, text=True,
encoding="utf-8", errors="replace", check=False,
)
except FileNotFoundError:
return ""
if proc.returncode != 0:
return ""
return proc.stdout.rstrip()
def render_json(meta, commits):
"""Render the JSON output for a `lore history` invocation.
Output matches the schema documented in the spec.
"""
payload = {
"entry_id": meta["entry_id"],
"lore_file": meta["lore_file"],
"code_file": meta["code_file"],
"since": meta["since"],
"since_source": meta["since_source"],
"commits": commits,
}
return _json.dumps(payload, indent=2, ensure_ascii=False)
def render_markdown(meta, commits):
"""Render the Markdown output for a `lore history` invocation.
Args:
meta: dict with keys entry_id, lore_file, code_file, since,
since_source.
commits: list of commit dicts (see parse_commit_line + extract_refs).
Returns:
Markdown string ready for stdout.
"""
lines = []
lines.append(f"# history: [{meta['entry_id']}]")
lines.append("")
lines.append(f"> Entry: {meta['lore_file']}")
since_suffix = " (entry #added date)" if meta.get("since_source") == "entry_added" else ""
lines.append(f"> Since: {meta['since']}{since_suffix}")
lines.append(f"> File: {meta['code_file']}")
lines.append(f"> Commits: {len(commits)} (showing all)")
lines.append("")
if not commits:
return "\n".join(lines) + "\n"
for c in commits:
lines.append(f"## {c['short']} ({c['date']}, {c['author']})")
lines.append(c["subject"])
if c.get("body"):
body = truncate_body(c["body"], max_lines=3)
lines.append(f' Body: "{body}"')
if c.get("refs"):
lines.append(f" Refs: {', '.join(c['refs'])}")
lines.append("")
lines.append("## Suggested next step")
lines.append("Run `lore sync` to check whether any of these commits")
lines.append("introduce a [REFINED] candidate for this entry.")
lines.append("")
return "\n".join(lines)
# Exit codes per spec section "Error handling".
ERR_USAGE = 2 # no arg / unrecognized arg (also used by argparse path)
ERR_NO_LORE = 2 # .lore/ not found
ERR_NO_ENTRY = 3 # entry ID not in index
ERR_NOT_GIT = 4 # not a git repository
ERR_NO_GIT = 5 # git CLI missing
ERR_BAD_SCOPE = 6 # scope name not in scopes/
ERR_GIT_FAIL = 7 # git log returned non-zero for other reasons
def die(code, message):
"""Print message to stderr and exit with the given code."""
print(f"error: {message}", file=sys.stderr)
sys.exit(code)
def _load_entries_via_subprocess():
"""Run scripts/list_entries.py --json and return the parsed list.
Mirrors the pattern in find_duplicates.py / find_stale.py.
Returns [] if no entries.
"""
here = Path(__file__).resolve().parent
cmd = [sys.executable, str(here / "list_entries.py"), "--json"]
try:
proc = subprocess.run(cmd, capture_output=True, text=True,
encoding="utf-8", errors="replace", check=False)
except FileNotFoundError as exc:
die(ERR_NO_GIT, f"python executable not found: {exc}")
if proc.returncode != 0:
die(ERR_NO_LORE, f"list_entries.py failed: {proc.stderr.strip()}")
try:
return _json.loads(proc.stdout)
except _json.JSONDecodeError as exc:
die(ERR_NO_LORE, f"list_entries.py returned invalid JSON: {exc}")
def _find_lore_root_or_die():
"""Walk up from CWD to find .lore/. Die with ERR_NO_LORE if not found."""
p = Path(".").resolve()
while p != p.parent:
if (p / ".lore").is_dir():
return p
p = p.parent
die(ERR_NO_LORE, ".lore/ not found. Run 'lore init' first.")
def _build_meta_entry(entry, code_file, since, since_source):
return {
"entry_id": entry["id"],
"lore_file": entry["file"],
"code_file": code_file,
"since": since,
"since_source": since_source,
}
def _resolve_scope_to_md_files(project_root, scope_name):
"""For scope form: list the (layer_file, md_path) tuples under the scope."""
scopes_dir = project_root / ".lore" / "scopes" / scope_name
if not scopes_dir.is_dir():
available = sorted(
p.name for p in (project_root / ".lore" / "scopes").iterdir()
if p.is_dir()
) if (project_root / ".lore" / "scopes").is_dir() else []
available_display = ", ".join(available) if available else "(none)"
die(ERR_BAD_SCOPE, f"Scope '{scope_name}' not found. Available: {available_display}")
files = []
for md in sorted(scopes_dir.glob("*.md")):
files.append((md.stem, md))
return files
def _is_git_repo(project_root):
try:
proc = subprocess.run(
["git", "-C", str(project_root), "rev-parse", "--git-dir"],
capture_output=True, text=True, check=False,
)
except FileNotFoundError:
die(ERR_NO_GIT, "git executable not found on PATH.")
return proc.returncode == 0
def _enrich_commits_with_body_and_refs(project_root, commits):
"""For each commit, fetch body and extract refs. Mutates in place."""
for c in commits:
msg = fetch_commit_body(project_root, c["hash"])
if msg:
# Body is everything after the first line.
parts = msg.split("\n", 1)
subject = parts[0]
body = parts[1].strip() if len(parts) > 1 else ""
c["subject"] = subject
c["body"] = truncate_body(body, max_lines=3)
c["refs"] = extract_refs(msg)
def main():
args = sys.argv[1:]
json_mode = "--json" in args
since_override = None
for a in args:
if a.startswith("--since="):
since_override = a.split("=", 1)[1]
positional = [a for a in args if a != "--json" and not a.startswith("--since=")]
if not positional:
print("usage: lore history <entry-id|file-path|--scope=NAME>",
file=sys.stderr)
die(ERR_USAGE, "missing argument")
parsed = parse_arg(positional[0])
if parsed is None:
die(ERR_USAGE, f"unrecognized argument: {positional[0]}")
project_root = _find_lore_root_or_die()
if not _is_git_repo(project_root):
die(ERR_NOT_GIT,
"Not a git repository. 'lore history' requires git; "
"use 'lore query' for in-memory answers.")
if parsed["form"] == "entry":
entries = _load_entries_via_subprocess()
entry = find_entry(entries, parsed["value"])
if entry is None:
ids = ", ".join(e["id"] for e in entries[:20])
more = "" if len(entries) <= 20 else f" (and {len(entries)-20} more)"
die(ERR_NO_ENTRY,
f"Entry {parsed['value']} not found. Available: {ids}{more}")
since = since_override or extract_added_date(entry.get("tags", {}))
if since is None:
print("warning: entry has no #added tag; using full history",
file=sys.stderr)
since = "1970-01-01"
code_file = resolve_code_file(entry)
try:
commits = run_git_log(project_root, since, code_file)
except RuntimeError as exc:
die(ERR_GIT_FAIL, str(exc))
_enrich_commits_with_body_and_refs(project_root, commits)
meta = _build_meta_entry(entry, code_file, since, "entry_added")
out = render_json(meta, commits) if json_mode else render_markdown(meta, commits)
print(out)
return
if parsed["form"] == "file":
since = since_override or "1970-01-01"
code_file = parsed["value"]
try:
commits = run_git_log(project_root, since, code_file)
except RuntimeError as exc:
die(ERR_GIT_FAIL, str(exc))
_enrich_commits_with_body_and_refs(project_root, commits)
meta = {
"entry_id": f"<file:{code_file}>",
"lore_file": "(direct file query)",
"code_file": code_file,
"since": since,
"since_source": "user_arg" if since_override else "default",
}
out = render_json(meta, commits) if json_mode else render_markdown(meta, commits)
print(out)
return
if parsed["form"] == "scope":
layer_files = _resolve_scope_to_md_files(project_root, parsed["value"])
scope_payloads = [] # only used when json_mode is True
for layer_name, md_path in layer_files:
# For scope form we treat each .md file as a "code file" stand-in:
# we git log the md file's project-relative path to find commits
# that touched that lore file. (Useful for tracking lore edits.)
rel = str(md_path.relative_to(project_root))
try:
commits = run_git_log(project_root, "1970-01-01", rel)
except RuntimeError as exc:
die(ERR_GIT_FAIL, str(exc))
_enrich_commits_with_body_and_refs(project_root, commits)
if json_mode:
meta = {
"entry_id": f"<scope:{parsed['value']}/{layer_name}>",
"lore_file": rel,
"code_file": rel,
"since": "1970-01-01",
"since_source": "scope_form",
}
scope_payloads.append({
"layer": layer_name,
"payload": _json.loads(render_json(meta, commits)),
})
else:
print(f"## Scope: {parsed['value']} / {layer_name}")
print("")
if not commits:
print("(no commits)")
print("")
continue
for c in commits:
print(f"### {c['short']} ({c['date']}, {c['author']})")
print(c["subject"])
if c.get("body"):
print(f' Body: "{c["body"]}"')
if c.get("refs"):
print(f" Refs: {', '.join(c['refs'])}")
print("")
if json_mode:
print(_json.dumps(
{
"form": "scope",
"scope": parsed["value"],
"layers": [item["layer"] for item in scope_payloads],
"results": scope_payloads,
},
indent=2,
ensure_ascii=False,
))
return
if __name__ == "__main__":
main()
@@ -0,0 +1,32 @@
#!/usr/bin/env python3
"""Compute the 4-char content hash for a lore entry ID.
Usage:
python id_hash.py "Use Next.js App Router; reason: streaming + RSC"
Output:
The 4-char lowercase hex hash that goes into an entry's ID, e.g. `a3f2`.
The hash is `sha256(text).hexdigest()[:4]`. This is the same algorithm
described in `references/entry-format.md` (ID generation section), so
running this script always produces the ID component a lore agent
would assign.
Cross-platform: works on Windows / Linux / macOS with Python 3.6+.
"""
import sys
import hashlib
def main():
if len(sys.argv) < 2 or sys.argv[1] in ("-h", "--help"):
print(__doc__, file=sys.stderr)
sys.exit(0)
text = sys.argv[1]
h = hashlib.sha256(text.encode("utf-8")).hexdigest()[:4]
print(h)
if __name__ == "__main__":
main()
@@ -0,0 +1,183 @@
#!/usr/bin/env python3
"""List all lore entries in `.lore/` as JSON or human-readable text.
Usage:
python list_entries.py # human-readable
python list_entries.py --json # JSON output
python list_entries.py --scope=frontend
python list_entries.py --layer=ARCH
Walks `.lore/_global/*` and `.lore/scopes/*/*` and parses every
Markdown bullet that matches the entry format. Output is one record per
entry with these fields:
id full ID, e.g. "ARCH-2026-07-09-a3f2"
layer prefix, e.g. "ARCH" / "DEC" / "CONV"
layer_file source file stem, e.g. "ARCHITECTURE"
scope scope name, or "_global"
file path relative to .lore/, e.g. "scopes/frontend/ARCHITECTURE.md"
text entry body, with tags stripped
tags dict of tag name -> value, e.g. {"added": "2026-07-09", "verified": "2026-07-15"}
last_verified value of #verified tag, or None
Used by:
- query / audit / compress workflows (pre-step enumeration)
- find_duplicates.py
- find_stale.py
"""
import json
import re
import sys
from pathlib import Path
# Schema version this skill understands. Bumped only on breaking
# config changes; see references/compatibility.md.
KNOWN_SCHEMA_VERSION = 1
def check_schema_version(lore_root: Path) -> None:
"""Warn if .lore/.config.json is missing or has an unknown schema_version.
Output goes to stderr so it does not pollute --json consumers.
Idempotent and best-effort: any failure (missing file, malformed
JSON, permission error) is silent — config is optional and the
user can address it separately.
"""
cfg_path = lore_root / ".config.json"
if not cfg_path.exists():
return
try:
cfg = json.loads(cfg_path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, OSError):
return
version = cfg.get("schema_version")
if version is None:
print(
"[WARN] .lore/.config.json has no schema_version field. "
"Add \"schema_version\": 1 so future lore upgrades can detect "
"this config and prompt for migrations when they exist.",
file=sys.stderr,
)
elif isinstance(version, int) and version > KNOWN_SCHEMA_VERSION:
print(
f"[WARN] .lore/.config.json#schema_version={version} is newer "
f"than this lore skill expects (max: {KNOWN_SCHEMA_VERSION}). "
"Pull the latest lore from upstream.",
file=sys.stderr,
)
def find_lore_root(start: Path) -> Path:
"""Walk up from start to find the project root containing .lore/."""
p = start.resolve()
while p != p.parent:
if (p / ".lore").is_dir():
return p / ".lore"
p = p.parent
return None
def parse_entry(line: str):
"""Parse one Markdown bullet line. Returns dict or None if not an entry."""
m = re.match(
r"^\s*-\s*\[([A-Z]+)-(\d{4}-\d{2}-\d{2})-([a-f0-9]{4})\]\s+(.*?)\s*$",
line,
)
if not m:
return None
layer, date, h, rest = m.group(1), m.group(2), m.group(3), m.group(4)
eid = f"{layer}-{date}-{h}"
# Extract #tag:value pairs
tag_re = re.compile(r"#(added|verified|stale|archived):(\S+)")
tags = {name: val for name, val in tag_re.findall(rest)}
text = tag_re.sub("", rest).strip()
return {
"id": eid,
"layer": layer,
"layer_file": None, # filled in by caller
"scope": None, # filled in by caller
"file": None, # filled in by caller
"text": text,
"tags": tags,
"last_verified": tags.get("verified"),
}
def collect_entries(root: Path):
entries = []
layers_dirs = [("_global", root / "_global"), ("scopes", root / "scopes")]
for section_name, section_path in layers_dirs:
if not section_path.exists():
continue
for md_file in sorted(section_path.rglob("*.md")):
if section_name == "_global":
scope = "_global"
else:
scope = md_file.parent.name
layer_file = md_file.stem
try:
with open(md_file, encoding="utf-8") as f:
for line in f:
e = parse_entry(line)
if e is None:
continue
e["scope"] = scope
e["layer_file"] = layer_file
e["file"] = str(md_file.relative_to(root))
entries.append(e)
except OSError as exc:
print(f"warning: cannot read {md_file}: {exc}", file=sys.stderr)
return entries
def main():
args = sys.argv[1:]
scope_filter = None
layer_filter = None
json_output = "--json" in args
for arg in args:
if arg.startswith("--scope="):
scope_filter = arg.split("=", 1)[1]
elif arg.startswith("--layer="):
layer_filter = arg.split("=", 1)[1]
root = find_lore_root(Path("."))
if root is None:
print("error: .lore/ not found (run from project root or below)",
file=sys.stderr)
sys.exit(1)
check_schema_version(root)
entries = collect_entries(root)
if scope_filter:
entries = [e for e in entries if e["scope"] == scope_filter]
if layer_filter:
entries = [e for e in entries if e["layer"] == layer_filter]
if json_output:
print(json.dumps(entries, indent=2, ensure_ascii=False))
return
if not entries:
print("(no entries)")
return
for e in entries:
verified = (
f" [verified:{e['last_verified']}]" if e["last_verified"] else ""
)
stale = " [STALE]" if "stale" in e["tags"] else ""
print(f"[{e['file']}] {e['id']} {e['text']}{verified}{stale}")
if __name__ == "__main__":
main()