📦 deps(thirdparty): update snapshots
This commit is contained in:
ci[bot]
+44
-138
@@ -1,157 +1,63 @@
|
||||
# AAS Agent-First Control Plane Preview Profile
|
||||
# AAS Agent-Owned Selection Profile
|
||||
|
||||
Status: approved intermediate release profile
|
||||
Date: 2026-07-17
|
||||
Certified design: `docs/maintainers/aas-agent-first-control-plane-v1-design.md`
|
||||
Status: active product profile
|
||||
Updated: 2026-07-19
|
||||
|
||||
## Purpose
|
||||
|
||||
This profile validates whether Codex, Claude Code, and comparable agents can
|
||||
use AAS to compose useful local skill stacks before AAS claims the stronger
|
||||
certified-v1 guarantees. It is additive: it does not change the frozen v1
|
||||
design, benchmark, hostile corpus, verifier, or completion criteria. A passing
|
||||
preview does not complete the active v1 goal.
|
||||
This profile defines the supported boundary for AAS Core after semantic skill selection moved to the coding agent.
|
||||
|
||||
The permitted preview claim is:
|
||||
> Codex or Claude inspects the project, searches and reads the complete local catalog, and chooses exact skill IDs. AAS Core validates, pins, compares, and plans that agent-owned selection; it does not recommend skills.
|
||||
|
||||
> AAS Agent-First Preview helps Codex and Claude compose a local, explainable,
|
||||
> reproducible skill stack. Full-catalog recommendation quality and
|
||||
> transactional apply/recovery safety are not yet certified.
|
||||
The earlier deterministic recommendation design and goal documents are retained as historical architecture records. They are not current product policy or release gates.
|
||||
|
||||
Preview output must not use `implementationVerified`, `releaseReady`,
|
||||
`released`, `certified`, or an equivalent unqualified claim.
|
||||
## Supported surfaces
|
||||
|
||||
## Included preview surfaces
|
||||
- A complete, integrity-verified local catalog in which every canonical skill is searchable, readable, selectable, and usable.
|
||||
- Local stdio MCP tools `search_skills`, `get_skill`, `compose_stack`, `inspect_stack`, and `diff_stack`, plus `aas://skills/{id}`.
|
||||
- Minimal, schema-validated `aas-stack.json` with pinned catalog identity, targets, goals, and exact agent-selected skill IDs.
|
||||
- CLI manifest validation, immutable plan preview, and read-only diagnosis.
|
||||
- Workbench import and review of the agent-owned stack and immutable plan.
|
||||
|
||||
- Minimal, schema-validated `aas-stack.json` with pinned catalog identity,
|
||||
targets, approved intent and policy, and exact skill IDs.
|
||||
- CLI `stack init`, `stack recommend`, `stack validate`, `stack plan`, and
|
||||
`stack doctor` from the packed npm candidate.
|
||||
- The local stdio MCP with exactly `search_skills`, `get_skill`,
|
||||
`recommend_stack`, `inspect_stack`, `diff_stack`, and
|
||||
`aas://skills/{id}`.
|
||||
- Deterministic recommendation with structured factors, two visible lanes
|
||||
(`recommended` and `discoveryCandidates`), explicit unknowns, stable
|
||||
tie-breaking, and fail-closed policy decisions.
|
||||
- Functional Node 22/24 coverage on Linux, macOS, and Windows from one exact
|
||||
content-addressed tarball.
|
||||
- Workbench schema/import/render tests and a local production build. A live
|
||||
Pages deployment remains outside the preview until separately approved.
|
||||
## Selection contract
|
||||
|
||||
1. The coding agent owns semantic selection. It may inspect the project with its normal local capabilities, search broadly, read full skill content when useful, compare alternatives, and choose exact IDs.
|
||||
2. AAS Core does not rank, recommend, promote, demote, exclude, or abstain on skills.
|
||||
3. Catalog metadata is informational only. Missing, incomplete, cautionary, or manually reviewed metadata must never make a canonical skill unsearchable, unselectable, or unusable.
|
||||
4. `compose_stack` validates catalog identity, target shape, goals, exact IDs, and structural limits, then returns the pinned stack shape. It does not substitute a different selection.
|
||||
5. `aas-stack.json` has no Core selection policy. User constraints can guide the agent's reasoning, but they are not an MCP eligibility filter or manifest gate.
|
||||
|
||||
## Functional gate
|
||||
|
||||
The packed-product smoke path must prove:
|
||||
|
||||
1. **Catalog completeness** — packaged catalog count and IDs equal the canonical registry; exact-ID search, `get_skill`, and content reads work for every canonical skill.
|
||||
2. **MCP contract** — the five supported read-only tools and resource template work over real stdio framing without repository scanning or state writes.
|
||||
3. **Agent-owned composition** — `compose_stack` preserves the exact ordered ID selection supplied by the agent and returns a structurally valid manifest without a policy field.
|
||||
4. **No metadata gating** — skills with unknown, critical, manual, blocked, incomplete, or absent informational metadata remain searchable, selectable, composable, and plannable.
|
||||
5. **Stack lifecycle** — compose, inspect, validate, plan, and doctor succeed in isolated roots without materializing target skills or managed state.
|
||||
6. **Workbench** — bounded text-only import/review tests and production build pass without ambient filesystem access.
|
||||
|
||||
## Experimental writes
|
||||
|
||||
`stack apply` and `stack recover` are present for controlled development but
|
||||
are not preview-supported safety claims. They are disabled by default:
|
||||
`stack apply` and `stack recover` remain experimental opt-ins. The supported public path stops after manifest validation and immutable plan review. Planning may write only the explicitly requested plan artifact.
|
||||
|
||||
- apply requires the additional `--experimental-apply` flag and the existing
|
||||
exact plan-digest approval;
|
||||
- recovery requires `--experimental-recovery` and retains its existing
|
||||
recovery-plan approval;
|
||||
- successful experimental writes return `releaseProfile: "preview"` and
|
||||
`certificationStatus: "experimental"`;
|
||||
- absence of the opt-in fails before runtime resolution or target writes with
|
||||
a structured policy error.
|
||||
## Trust and privacy boundaries
|
||||
|
||||
Internal transaction tests are development evidence only. Certification still
|
||||
requires the frozen production-binary crash, boundary, race, rollback, and
|
||||
recovery verifier.
|
||||
|
||||
## Preview functional gate
|
||||
|
||||
Every matrix job must install the exact candidate tarball with lifecycle
|
||||
scripts disabled and run without checkout-only runtime dependencies. No job may
|
||||
be skipped or allowed to fail.
|
||||
|
||||
On Windows, the preview verifier may materialize its own isolated runtime-cache
|
||||
fixture and must then have the production core verify the complete identity and
|
||||
every cached byte before `plan`, `doctor`, or MCP use. This proves the read-only
|
||||
functional lifecycle without claiming that Windows cache-promotion durability
|
||||
is certified. Native directory-flush and interrupted-promotion evidence remains
|
||||
part of the certified-v1 transaction gate.
|
||||
|
||||
Windows preview creation of the regenerable manifest and immutable plan uses
|
||||
the explicit `--preview-windows-output` opt-in. The CLI fsyncs the file and
|
||||
returns `outputDurability: "fileSyncedDirectoryUnverified"` together with
|
||||
`certificationStatus: "notCertified"`; without that flag it remains fail-closed.
|
||||
This opt-in never applies to skill installation, host configuration, apply, or
|
||||
recovery.
|
||||
|
||||
Required functional suites are:
|
||||
|
||||
1. **Package and entrypoints** — allowlisted package contents; `aas`,
|
||||
`aas-mcp`, and the legacy alias exist; legacy invocation creates no stack
|
||||
state implicitly.
|
||||
2. **Stack lifecycle** — `init -> recommend -> validate -> plan -> doctor`
|
||||
succeeds in isolated roots and does not materialize target skills or AAS
|
||||
managed state.
|
||||
3. **Determinism and explanation** — repeated identical inputs produce the
|
||||
same canonical recommendation payload and expose factor, coverage, evidence,
|
||||
exclusion, and unknown fields.
|
||||
4. **Policy** — proved incompatibility or forbidden risk is excluded;
|
||||
incomplete evidence remains visible; malformed or over-limit input fails
|
||||
closed.
|
||||
5. **MCP contract** — the five tools and one resource template work over real
|
||||
stdio framing; project and cache snapshots remain unchanged by tool calls.
|
||||
6. **Write guard** — apply and recovery without their experimental flags fail
|
||||
with structured policy codes and leave project, cache, and managed state
|
||||
unchanged.
|
||||
7. **Workbench** — bounded text-only import/review tests and production build
|
||||
pass without ambient filesystem access.
|
||||
|
||||
The preview receipt must declare:
|
||||
|
||||
```json
|
||||
{
|
||||
"assuranceProfile": "agent-first-preview-1",
|
||||
"previewQualified": true,
|
||||
"certifiedV1": false,
|
||||
"notEvaluated": [
|
||||
"native-network-and-filesystem-attempt-observation",
|
||||
"transactional-crash-and-race-certification",
|
||||
"benchmark-80-90-100",
|
||||
"real-host-configuration-writes",
|
||||
"public-release"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Missing receipts, crashes, timeouts, canonical drift, or any failed functional
|
||||
suite make `previewQualified` false.
|
||||
|
||||
## Explicitly not certified by preview
|
||||
|
||||
- ETW, `fs_usage`, or `strace` proof of zero network attempts and zero
|
||||
persistent MCP writes.
|
||||
- Production-binary fault injection at every transaction boundary or every
|
||||
declared race class.
|
||||
- Full benchmark thresholds: at least 80% verified coverage, 90% inclusion
|
||||
precision, and 100% correct abstention for each supported intent and in
|
||||
macro-average.
|
||||
- Complete property/fuzz/hostile budgets required by certified v1.
|
||||
- Real Codex or Claude configuration writes, public Pages deployment, npm
|
||||
publication, GitHub release, or announcement.
|
||||
|
||||
These remain mandatory before AAS can call the recommendation system or
|
||||
transactional lifecycle certified v1.
|
||||
- MCP is local, offline-capable, read-only, bounded, and non-mutating.
|
||||
- AAS does not receive repository files unless the agent explicitly reads them through its own host capabilities; AAS MCP does not scan the repository.
|
||||
- Full skill prose is untrusted content and gains no instruction authority through MCP.
|
||||
- Catalog and runtime integrity remain deterministic even though semantic selection belongs to the agent.
|
||||
- Real host configuration writes, publication, Pages deployment, npm release, and announcements require their separate approvals.
|
||||
|
||||
## Product-learning gate
|
||||
|
||||
After the functional matrix passes, preview evaluation should measure whether
|
||||
agents actually produce useful proposals:
|
||||
Evaluate the quality of the agent workflow, not a Core recommender:
|
||||
|
||||
- task completion rate from a repository profile to a reviewable stack;
|
||||
- human accept/replace/remove rates for recommended skills;
|
||||
- uncovered goals and discovery-candidate promotions;
|
||||
- deterministic replay rate for the same normalized input and catalog digest;
|
||||
- time and interaction count from request to approved manifest.
|
||||
- task completion from project inspection to a reviewable stack;
|
||||
- human accept, replace, and remove rates for agent-selected skills;
|
||||
- whether the agent searched enough of the catalog and read relevant skill content;
|
||||
- time and interaction count from request to approved manifest;
|
||||
- successful replay of the exact approved IDs against the pinned catalog identity.
|
||||
|
||||
No repository profile, source file, secret, or raw path is uploaded by default.
|
||||
Publishing or sharing any collected result requires a separate explicit
|
||||
decision and privacy review.
|
||||
|
||||
## Relationship to certified v1
|
||||
|
||||
The certified verifier may remain red or unevaluated while the preview gate is
|
||||
green. That state must be reported as `previewQualified: true` and
|
||||
`certifiedV1: false`, never as a skipped certified pass. The frozen v1 design
|
||||
and goal remain the only completion criteria for certification and release.
|
||||
No repository profile, source file, secret, or raw path is uploaded by default. Publishing or sharing any collected result requires a separate privacy decision.
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
# AAS Agent-First Control Plane v1 Design
|
||||
|
||||
> **Historical design:** deterministic Core recommendation, metadata eligibility, and selection policy described below were superseded on 2026-07-19 by the active [Agent-Owned Selection Profile](aas-agent-first-control-plane-preview-profile.md). Retained for architecture history; not current product policy.
|
||||
|
||||
Status: frozen and approved for implementation
|
||||
Date: 2026-07-17
|
||||
|
||||
> **Historical target design:** This frozen document records the stronger certified-v1 target and preserves the terminology approved at that time. It is not a statement of current public guarantees. The supported public preview stops after plan review; see [`aas-agent-first-control-plane-preview-profile.md`](aas-agent-first-control-plane-preview-profile.md). In current product documentation, AAS Core is the product and `aas-stack.json` plus the plan are its durable artifacts.
|
||||
|
||||
## Product statement
|
||||
|
||||
> L'agente compone. Tu controlli. AAS mantiene lo stack aggiornato.
|
||||
|
||||
@@ -1,8 +1,12 @@
|
||||
# AAS Agent-First Control Plane v1 Goal
|
||||
|
||||
> **Historical goal:** recommendation-quality, metadata, and policy gates described below were superseded on 2026-07-19 by the active [Agent-Owned Selection Profile](aas-agent-first-control-plane-preview-profile.md). Retained for decision history; not a current release gate.
|
||||
|
||||
Status: approved source packet for the active Codex goal
|
||||
Design source: `docs/maintainers/aas-agent-first-control-plane-v1-design.md`
|
||||
|
||||
> **Historical goal packet:** This file preserves the original certified-v1 finish line, including apply/recovery and independent verification work. It is not the current public preview contract or a claim that those gates shipped. The supported public preview stops after plan review; see [`aas-agent-first-control-plane-preview-profile.md`](aas-agent-first-control-plane-preview-profile.md).
|
||||
|
||||
## Fit
|
||||
|
||||
Use a durable goal. The work crosses catalog schema, deterministic recommendation, CLI lifecycle, local MCP, transactional filesystem behavior, host adapters, Workbench review, cross-platform packaging, benchmark construction, security abuse testing, and protected release verification. It needs repeated implementation/verification loops and has an independent observable finish line.
|
||||
|
||||
+7
@@ -1,5 +1,12 @@
|
||||
# AAS Agent-First Control Plane v1 Worklog
|
||||
|
||||
- 2026-07-19: Semantic skill selection moved to Codex and Claude. Core now exposes the complete catalog and validates/pins exact agent-selected IDs through `compose_stack`; selection policy and metadata eligibility gates were retired. Every canonical skill must remain searchable, readable, selectable, and usable. Earlier recommendation entries below are historical.
|
||||
|
||||
## 2026-07-18 — Baseline workflow retired
|
||||
|
||||
- The standalone `aas-v1-baseline` pull-request workflow and required status check were retired by maintainer decision. The obsolete verifier corpus, harness, tuning runner, and paused apply/optimize workflows were subsequently removed. The protected `pr-policy`, `pr-evidence`, `source-validation`, and `artifact-preview` gates remain required.
|
||||
- Entries below this point are a historical construction log. References to frozen verifier assets, matrices, paths, or pending certification gates describe the state at that date and are not current repository policy.
|
||||
|
||||
## 2026-07-17 — Goal activation and clean baseline
|
||||
|
||||
- Active objective is defined by the approved design and goal documents.
|
||||
|
||||
@@ -39,11 +39,8 @@ This document summarizes the repository coherence audit performed after the `app
|
||||
- missing examples and missing limitations sections,
|
||||
- overly long `SKILL.md` files that should probably be split into `references/`,
|
||||
- plus the existing structural/safety checks (frontmatter, risk, `When to Use`, offensive disclaimer, dangling links).
|
||||
- The report also includes a non-blocking `suggested_risk` for skills that are still marked `unknown` or appear to be misclassified, so maintainers can resolve risk classification during PR review without changing the contributor gate.
|
||||
- Added `tools/scripts/sync_risk_labels.py` (also exposed as `npm run sync:risk-labels`) for conservative legacy cleanup: it only rewrites `risk: unknown` when the suggestion is high-confidence enough to be safely automated.
|
||||
- The sync now covers explicit high-confidence `safe`, `critical`, `offensive`, and `none` patterns. When a skill is promoted to `offensive`, the sync also inserts the canonical `AUTHORIZED USE ONLY` notice so the label and content guardrail stay aligned.
|
||||
- The intended maintainer loop is: `audit:skills` to inspect `suggested_risk`, `sync:risk-labels` for the safe automated subset, then manual review for the ambiguous tail that should not be batch-classified.
|
||||
- Use `npm run audit:skills` for the maintainer view and `npm run audit:skills -- --json-out ... --markdown-out ...` when you want artifacts for triage or cleanup tracking.
|
||||
- Risk inference from isolated words was removed because it produced widespread false positives. The audit validates declared metadata; ambiguous labels require semantic review.
|
||||
- Use `npm run audit:skills` for the maintainer view and `npm run audit:skills -- --json-out ... --markdown-out ...` when you want objective structural artifacts for triage or cleanup tracking.
|
||||
|
||||
### 3. Cross references
|
||||
|
||||
@@ -79,8 +76,6 @@ This document summarizes the repository coherence audit performed after the `app
|
||||
npm run validate # skill validation (soft)
|
||||
npm run validate:strict # hardening / diagnostic pass
|
||||
npm run audit:skills # full skill audit with finding codes and status
|
||||
npm run sync:risk-labels # conservative sync for high-confidence legacy risk labels
|
||||
npm run sync:risk-labels -- --dry-run # preview legacy risk rewrites before touching files
|
||||
npm run validate:references # workflow, bundle, and docs/users/bundles.md references
|
||||
npm run security:docs # documentation command-risk scan (required for security-sensitive guidance)
|
||||
npm run build # chain + catalog
|
||||
@@ -90,5 +85,5 @@ npm test # suite test
|
||||
## Open issues / follow-up
|
||||
|
||||
- Gradual cleanup of legacy skills so `npm run validate:strict` can become a hard CI gate in the future.
|
||||
- Continue reducing the remaining `risk: unknown` tail with conservative sync passes plus manual maintainer review for ambiguous cases.
|
||||
- Continue reducing the remaining `risk: unknown` tail through semantic maintainer review.
|
||||
- Keep translated docs aligned in a separate pass after the canonical English docs are stable.
|
||||
|
||||
@@ -1,28 +0,0 @@
|
||||
# Local skill reviewer threat model
|
||||
|
||||
## Assets and trust boundaries
|
||||
|
||||
Protected assets are repository contents, credentials, local configuration, network authority, review integrity, and truthful completion state. Trusted code is limited to deterministic discovery, validation, local level assignment, scoring, triage, packet construction, schemas, and atomic state/output machinery. Skill files, bundles, Codex judgments, cached data, YAML, Markdown, URLs, candidates, and patches are untrusted.
|
||||
|
||||
The local `manual-review-required` status is namespaced to `source: local-skill-reviewer`. It is a triage escalation, not a Tessl result, CI fallback attestation, exact-head approval, or merge authorization.
|
||||
|
||||
## Primary threats and controls
|
||||
|
||||
| Threat | Control | Failure behavior |
|
||||
| --- | --- | --- |
|
||||
| Prompt injection or fabricated scores | Production triage executes no model; semantic packets place a trusted hostile-input instruction outside source data; imported judgments bind exact evidence and all anchor comparisons | Reject malformed, unbound, stale, or out-of-range judgment |
|
||||
| Tool, shell, MCP, browser, URL, Tessl, or Codex CLI execution | Runtime analyzer has no tool or service adapter and never follows content-derived links or commands | Abort review; offline sentinel tests fail on invocation |
|
||||
| Credential exfiltration | Likely credential values are redacted from heuristic evidence; semantic packets include only tracked, bounded Git blobs under the allowed bundle roots | Reject sensitive identifiers or malformed imports; repository secret scanning remains a separate gate |
|
||||
| Path traversal or symlink escape | Git mode/object allowlist, frozen-index reads, POSIX-relative normalization, bounded descriptor reads, physical containment, and symlink-safe output parents | Structured input/output failure |
|
||||
| YAML expansion or unsafe tags | Frontmatter byte, node, and depth caps; strict parser; aliases, anchors, and tags forbidden | Validation failure and manual escalation |
|
||||
| Oversized or binary bundle | Per-file, bundle, and count caps; primary Markdown requires UTF-8; binary bytes are hash-only; semantic packets omit rather than invisibly truncate supplemental text | Cap stop or explicit metadata-only/omitted source |
|
||||
| Cache poisoning or stale evidence | Whole-bundle hashes plus rubric, schema, validator, analyzer, runtime, threshold, and reviewer identity | Cache miss and safe recomputation |
|
||||
| Crash misreported as completion | Atomic writes and explicit pending/running/completed/failed state; semantic artifact sets require final hash-bound markers | Resume or reject incomplete output |
|
||||
| False Tessl equivalence | Output calls itself `local-triage-only`, uses `local_quality_score`, and includes a non-equivalence disclaimer; documentation preserves separate validation, blind, and Tessl-repeat figures | Claim is rejected in review; no Tessl-pass prediction is emitted |
|
||||
| Cost or retry storm | Production runtime has no Tessl dependency; future Tessl use is a separate sample audit only | No paid-service call from triage |
|
||||
| Unreviewed repository mutation | Reviewer output is outside the repo and the production CLI has no apply path; Codex correction requires normal authorization, diff inspection, references, validation, security, and tests | Correction cannot be called complete until gates rerun |
|
||||
| Merge bypass | `--merge-gate` always emits P0 and records that exact-head attestation is still required | Maintainer workflow remains blocking |
|
||||
|
||||
## Residual risk
|
||||
|
||||
Deterministic rules cannot fully understand semantics and can produce false positives or negatives. Codex can still be influenced by hostile text; exact evidence and schema validation prove provenance, not model invulnerability. The accepted 74.5% measurement is Codex-assisted validation performance, not deterministic or blind accuracy. Blind equivalence and stability were not demonstrated. Independent security, repository, and exact-head merge review remain necessary.
|
||||
@@ -1,94 +0,0 @@
|
||||
# Local skill reviewer
|
||||
|
||||
The local skill reviewer is the production triage path for AAS skills. It runs offline, freezes tracked skill inputs from the Git index, performs deterministic validation and quality checks, assigns local 1-3 rubric levels, computes a `local_quality_score`, and emits priorities plus evidence for follow-up.
|
||||
|
||||
It is not Tessl, is not equivalent to Tessl, and cannot guarantee that a skill will pass a Tessl review. Tessl is not a runtime dependency. When credits are available again it may be used only as a separate sample audit.
|
||||
|
||||
## Production contract
|
||||
|
||||
For each skill, the reviewer emits:
|
||||
|
||||
- deterministic validation and AAS policy findings;
|
||||
- eight locally assigned 1-3 quality levels, confidence, and matched evidence;
|
||||
- a weighted `local_quality_score` from 0 to 100;
|
||||
- `triage.reviewStatus`, `triage.priority`, and stable `reasonCodes`;
|
||||
- exact input and tool-version bindings for cache invalidation and replay.
|
||||
|
||||
`triage.reviewStatus` is either `pass` or `manual-review-required`. Here, `pass` means only that no configured escalation rule fired; it is not a quality certification, Tessl result, or merge approval. The local `manual-review-required` value belongs to `source: local-skill-reviewer`; it must not be confused with the identically worded CI/Tessl fallback. It never satisfies the maintainer workflow's exact-head review attestation.
|
||||
|
||||
The production escalation rules mark a skill for manual review when it has a validation error, a broken-reference warning, deterministic policy findings, a score below 50, a score within three points of 50 or 75, low-confidence extreme levels, critical/offensive risk, or an explicit merge gate. Priorities are:
|
||||
|
||||
- `P0`: merge-blocking candidate; Codex review and the normal exact-head maintainer attestation remain required;
|
||||
- `P1`: validation, high-risk, or below-50 findings;
|
||||
- `P2`: other escalation reasons or clean middle-band triage;
|
||||
- `P3`: clean high-band triage.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
npm run review:skills:triage -- --result-dir /private/tmp/aas-review-full --concurrency 4
|
||||
npm run review:skills:triage -- --resume --result-dir /private/tmp/aas-review-full --concurrency 4
|
||||
npm run review:skills:local -- review short --result-dir /private/tmp/aas-review
|
||||
npm run review:skills:local -- review short --merge-gate --result-dir /private/tmp/aas-review
|
||||
npm run review:skills:local:test
|
||||
```
|
||||
|
||||
For one skill, create its packet, obtain the Codex judgment, then import and verify:
|
||||
|
||||
```bash
|
||||
npm run review:skills:semantic:packet -- short --result-dir /private/tmp/aas-semantic-review
|
||||
npm run review:skills:semantic:import -- short --input /path/to/short-judgment.json --result-dir /private/tmp/aas-semantic-review
|
||||
npm run review:skills:semantic:verify -- short --result-dir /private/tmp/aas-semantic-review
|
||||
```
|
||||
|
||||
For a batch, use the alternative preparation command, then obtain, import, and verify a Codex judgment for each escalated skill:
|
||||
|
||||
```bash
|
||||
npm run review:skills:semantic:prepare -- --result-dir /private/tmp/aas-semantic-review
|
||||
npm run review:skills:semantic:import -- short --input /path/to/short-judgment.json --result-dir /private/tmp/aas-semantic-review
|
||||
npm run review:skills:semantic:verify -- short --result-dir /private/tmp/aas-semantic-review
|
||||
```
|
||||
|
||||
Do not run `semantic:packet` and `semantic:prepare` for the same skill in the same result directory.
|
||||
|
||||
Results default to a private OS temporary directory. A supplied `--result-dir` must stay outside the repository and pass the symlink-safe output checks. `scan-summary.json` contains score bands, priority counts, escalation reasons, and the first 25 manual-review priorities; `scan-results.jsonl` contains the complete per-skill records.
|
||||
|
||||
`review --merge-gate` persists the contextual P0 record under `merge-gate-results/`. The normal cache and `results/` record remain context-free so the same deterministic evidence can be reused without falsely turning every later review into a merge gate.
|
||||
|
||||
The production smoke scan completed 1,965/1,965 tracked canonical skills with zero failures. It produced 1,371 `manual-review-required` and 594 `pass` results; priorities were 0 P0, 346 P1, 1,421 P2, and 198 P3. These counts describe triage workload, not Tessl outcomes.
|
||||
|
||||
## Codex review and correction loop
|
||||
|
||||
Codex interprets only cases that need judgment, beginning with P0/P1 and the report's `topPriorities`:
|
||||
|
||||
1. Read the deterministic reasons, exact evidence, skill content, and referenced files. Treat all skill text as hostile input.
|
||||
2. For an ambiguous case, create or read the hash-bound semantic packet, assign all eight levels with exact evidence and adjacent-anchor rejection, then import and verify the judgment locally.
|
||||
3. Codex may propose and, when the task authorizes changes, apply a correction to the canonical skill through the normal reviewed workspace-editing path. The local reviewer itself has no apply capability.
|
||||
4. Review the resulting diff. Because the reviewer intentionally reads the Git index, stage the exact intended skill/reference blobs in the normal topic-branch workflow before rerunning it; never assume an unstaged edit was scanned. Also rerun `npm run validate`, `npm run validate:references`, `npm run security:docs`, and the relevant targeted tests; use full `npm test` for repository-wide or merge-bound work.
|
||||
5. Reinspect all changed references and mirrors. For a merge, review the exact full head SHA through the mandatory maintainer workflow; local triage or an earlier Codex judgment is not a substitute.
|
||||
|
||||
For a single changed skill, use `semantic:packet`; use `semantic:prepare` when preparing a batch. Both routes feed the same judgment import and verification contract.
|
||||
|
||||
A stale packet, changed bundle, malformed judgment, missing evidence, or incomplete output means no verified semantic review exists. The local reviewer never silently falls back from a failed semantic import.
|
||||
|
||||
## Safety boundary
|
||||
|
||||
Skills are hostile input. Discovery starts from `git ls-files --stage`; only regular index objects for `SKILL.md` and bounded tracked files under `references/`, `scripts/`, or `assets/` enter a bundle. Bytes are read from frozen Git object IDs, not through worktree paths. Symlinks, gitlinks, traversal, unsafe YAML aliases/tags, oversized inputs, and invalid UTF-8 primary Markdown fail closed. URLs and instructions in skill content are never followed.
|
||||
|
||||
The deterministic runtime uses no model, Codex CLI, Tessl CLI, API, browser, MCP, or network. Supplemental binary files affect bundle hashes but are not parsed or included as instructions. Output files are atomic and bound to their inputs, schemas, thresholds, runtime, and reviewer version.
|
||||
|
||||
## Measurement and limits
|
||||
|
||||
The accepted operational reference is **149/200 exact labels (74.5%)** on the 25-skill validation set for the Codex-assisted adjudication procedure. It is not a blind result and is not the accuracy of the deterministic scanner alone. The immutable receipt is `tools/config/local-skill-review-operational-receipt.json`.
|
||||
|
||||
The untouched 35-skill blind experiment measured 57.143% exact agreement for frozen deterministic v9 and 72.857% for the Codex-assisted procedure. Tessl's forced-repeat self-agreement on a separate 15-skill panel was 74.167%. The preregistered parity target failed, the blind cohort is now revealed, and no further tuning on these labels is permitted. These measurements support production use for triage and Codex-assisted review only; they do not demonstrate rigorous equivalence or blind stability.
|
||||
|
||||
The local score measures rubric conformance, not real-world task success. Scenario-based evaluations remain necessary for behavioral claims.
|
||||
|
||||
## Historical calibration artifacts
|
||||
|
||||
The parity manifests, collectors, fixtures, and metric scripts remain frozen as audit evidence and regression material. They are not imported by the production triage runtime and are not exposed as its supported operating path. The old calibration npm shortcut was removed to prevent accidental continued fitting on revealed labels.
|
||||
|
||||
## Criterion provenance
|
||||
|
||||
The local rubric, schemas, anchors, reason codes, fixtures, and wording are original clean-room material. Only publicly observable dimension names, 1-3 scale, and weights informed the design. No private Tessl prompt, server validator, model behavior, example, or implementation is treated as provenance.
|
||||
@@ -30,22 +30,18 @@ Use `--dry-run` to exercise local classification without approving a run or merg
|
||||
|
||||
`merge:batch` will:
|
||||
|
||||
- refresh the PR body when the Quality Bar checklist is missing
|
||||
- record the existing workflow-run generation, then close and reopen the PR if stale metadata needs a fresh `pull_request` event
|
||||
- fetch the exact base/head objects and classify the complete raw Git diff
|
||||
- recompute changed-skill evidence with evaluator code materialized from the trusted `main` commit
|
||||
- reject incomplete evidence coverage, deterministic quality/security/provenance regressions, and base/head drift
|
||||
- for external PRs, poll for asynchronously-created fork runs and approve only post-reopen runs waiting on `action_required` when every path, mode, object, size, and workflow identity is allowlisted
|
||||
- for external PRs, poll for asynchronously-created fork runs and approve only runs waiting on `action_required` when every path, mode, object, size, and workflow identity is allowlisted
|
||||
- for same-repository maintainer PRs, allow repository-wide source changes while still enforcing trusted changed-skill evidence, exact-head review, required checks, branch protection, and immutable PR identity
|
||||
- wait only for check suites belonging to the post-reopen workflow generation; older runs on the same head SHA cannot satisfy or fail the gate
|
||||
- wait for the latest required checks bound to the exact head SHA
|
||||
- call GitHub's immediate squash-merge endpoint and continue only when it reports `merged: true`
|
||||
- pull the protected `main`; its trusted workflow opens a canonical-sync bot PR for generated artifacts and contributor credits when needed
|
||||
|
||||
## What It Automates
|
||||
|
||||
- PR body normalization against the repository template
|
||||
- stale PR metadata refresh
|
||||
- generation-bound required-check polling for the current PR head
|
||||
- exact-head required-check polling
|
||||
- handoff of post-merge contributor and artifact drift to the canonical-sync PR lane
|
||||
|
||||
## What It Does Not Automate
|
||||
|
||||
@@ -21,7 +21,7 @@ Resolve conflicts **on the PR branch** so the PR becomes mergeable, then use "Sq
|
||||
- `README.md` is mixed ownership: contributor prose edits are allowed, but workflow-managed metadata is canonicalized on `main`.
|
||||
- If derived files appear in a PR refresh or merge conflict, prefer **`main`'s side** and remove them from the PR branch instead of hand-maintaining them there.
|
||||
- Do not block a PR only because shared generated files would be regenerated differently after other merges. `main` auto-syncs the final state after merge.
|
||||
- If a skill PR leaves `risk: unknown`, that is not automatically a blocker. Maintainers can review the suggested classification with `npm run audit:skills`, optionally run `npm run sync:risk-labels` locally after merge, and still keep the contributor PR source-only.
|
||||
- If a skill PR leaves `risk: unknown`, that is not automatically a blocker. Review the actual behavior semantically; do not infer risk from isolated words or rewrite it automatically.
|
||||
|
||||
### Steps (maintainer resolves conflicts on the contributor’s branch)
|
||||
|
||||
|
||||
@@ -41,11 +41,10 @@ Every new or relocated skill and every canonical skill-content change requires m
|
||||
|
||||
## Fork Review States
|
||||
|
||||
The Skill Review workflow separates three outcomes:
|
||||
The Skill Review workflow separates two outcomes:
|
||||
|
||||
- `review`: a semantic review actually ran using trusted base scripts;
|
||||
- `manual-review-required`: repository secrets were unavailable to a fork, so a maintainer must review and attest to the exact head SHA;
|
||||
- `missing-review-credentials`: an internal branch expected semantic-review credentials but none were configured, so the check fails closed.
|
||||
- `manual-review-required`: Tessl credentials or quota were unavailable, or Tessl did not produce a passing semantic result, so a maintainer must review and attest to the exact head SHA.
|
||||
|
||||
A successful `manual-review-required` check means only that the requirement was recorded. It is not a successful semantic review.
|
||||
|
||||
|
||||
@@ -47,7 +47,7 @@ Use this as a diagnostic signal. It is useful for spotting legacy quality debt,
|
||||
- Confirm `README.md` reflects the current version and generated counts.
|
||||
- Confirm Credits & Sources, contributors, and support links are still correct.
|
||||
- If PR or CI workflow behavior changed during the cycle, confirm maintainer and contributor docs mention the active checks (for example the `skill-review` workflow for `SKILL.md` pull requests).
|
||||
- If maintainers used `npm run sync:risk-labels` or a comparable cleanup flow during the cycle, make sure the maintainer docs still describe the current audit -> sync -> repo-state loop.
|
||||
- If maintainers changed declared risk labels during the cycle, confirm that each change has semantic review evidence rather than lexical inference.
|
||||
|
||||
5. Prepare the protected release PR:
|
||||
|
||||
@@ -65,22 +65,29 @@ This command:
|
||||
- creates and pushes `release/vX.Y.Z`
|
||||
- opens a release PR containing the scripted canonical release state
|
||||
|
||||
Prerelease versions use the same protected flow, for example `15.0.0-rc.1`. They must have their own exact changelog section.
|
||||
|
||||
6. Merge the release PR through required checks, update local `main`, then publish the GitHub release:
|
||||
|
||||
```bash
|
||||
npm run release:publish -- X.Y.Z
|
||||
```
|
||||
|
||||
This command proves local `main` equals protected `origin/main` and the exact squash commit of the merged `release/vX.Y.Z` PR, checks that no canonical-sync PR or release-state drift remains, creates or reuses the matching local/remote tag safely, and creates the GitHub release object from the matching `CHANGELOG.md` section. It never pushes `main` directly and can be retried after a partial tag/release failure.
|
||||
This command proves local `main` equals protected `origin/main` and the exact squash commit of the merged `release/vX.Y.Z` PR, checks that no canonical-sync PR or release-state drift remains, creates or reuses the matching local/remote tag safely, and creates the GitHub release object from the matching `CHANGELOG.md` section. SemVer prereleases are marked as GitHub prereleases. It never pushes `main` directly and can be retried after a partial tag/release failure.
|
||||
|
||||
7. Publish to npm if needed:
|
||||
|
||||
```bash
|
||||
npm publish
|
||||
npm publish --tag latest
|
||||
```
|
||||
|
||||
Normally this still happens via the existing GitHub release workflow after the GitHub release is published.
|
||||
That workflow now reruns `sync:release-state`, installs Python dependencies from `tools/requirements.txt`, refreshes tracked web assets, fails on canonical drift via `git diff --exit-code`, executes tests and docs security checks, runs the web-app coverage gate, enforces `npm audit --audit-level=high`, builds the web app, and dry-runs the npm package before `npm publish`.
|
||||
Normally this still happens via the existing GitHub release workflow after the GitHub release is published. The workflow publishes stable versions explicitly to npm's `latest` dist-tag and prerelease versions explicitly to `next`; it fails closed on an invalid version. Verify both tags after a prerelease so `latest` remains on the last stable release.
|
||||
|
||||
```bash
|
||||
npm view agentic-awesome-skills dist-tags --json
|
||||
```
|
||||
|
||||
The workflow reruns `sync:release-state`, installs Python dependencies from `tools/requirements.txt`, refreshes tracked web assets, fails on canonical drift via `git diff --exit-code`, executes tests and docs security checks, runs the web-app coverage gate, enforces `npm audit --audit-level=high`, builds the web app, and dry-runs the npm package before publishing.
|
||||
|
||||
## Canonical Sync Bot
|
||||
|
||||
|
||||
@@ -6,21 +6,21 @@ This document keeps the repository's GitHub-facing discovery copy aligned with t
|
||||
|
||||
Preferred positioning:
|
||||
|
||||
> Installable GitHub library of 1,965+ agentic skills for Claude Code, Cursor, Codex CLI, Gemini CLI, Antigravity, and other AI coding assistants.
|
||||
> AAS Core is the local, deterministic boundary that exposes the complete catalog, validates exact skill IDs selected by the coding agent from an explicit project profile, and produces an immutable preview plan, backed by 1,968+ cataloged skills.
|
||||
|
||||
Key framing:
|
||||
|
||||
- installable skill library;
|
||||
- GitHub-first discovery;
|
||||
- broad multi-tool compatibility;
|
||||
- bundles, workflows, and onboarding docs;
|
||||
- useful for both breadth and curated evaluation.
|
||||
- AAS Core as the primary product;
|
||||
- local MCP and CLI as the agent-facing and operator-facing interfaces;
|
||||
- `aas-stack.json` and the immutable plan as durable artifacts;
|
||||
- GitHub as the canonical source, with the hosted catalog and Workbench as discovery and review surfaces;
|
||||
- catalog, plugins, bundles, workflows, and direct installers as supporting content and distribution.
|
||||
|
||||
## GitHub About settings
|
||||
|
||||
Preferred description:
|
||||
|
||||
> Installable GitHub library of 1,965+ agentic skills for Claude Code, Cursor, Codex CLI, Gemini CLI, Antigravity, and more. Includes installer CLI, bundles, workflows, and official/community skill collections.
|
||||
> Local, deterministic AAS Core for explainable coding-agent skill stacks, backed by 1,967+ cataloged skills. Includes a read-only MCP, stack CLI, Workbench review, plugins, bundles, workflows, and direct distribution.
|
||||
|
||||
Preferred homepage:
|
||||
|
||||
@@ -28,8 +28,9 @@ Preferred homepage:
|
||||
|
||||
Preferred social preview:
|
||||
|
||||
- use a clean preview image that says `1,965+ Agentic Skills`;
|
||||
- mention Claude Code, Cursor, Codex CLI, and Gemini CLI;
|
||||
- lead with `AAS Core` and the profile → stack → plan flow;
|
||||
- present `1,969+ Agentic Skills` as supporting catalog evidence, not a second product;
|
||||
- mention Codex and Claude as the current Core agent path, with broader host compatibility as distribution support;
|
||||
- avoid dense text and tiny logos that disappear in social cards.
|
||||
|
||||
Preferred topics:
|
||||
@@ -75,11 +76,12 @@ Suggested release template:
|
||||
```md
|
||||
## [VERSION] - YYYY-MM-DD - "User-facing title"
|
||||
|
||||
> Installable skill library update for Claude Code, Cursor, Codex CLI, Gemini CLI, and related AI coding assistants.
|
||||
> AAS Core and supporting catalog update for local, explainable coding-agent skill stacks.
|
||||
|
||||
Start here:
|
||||
|
||||
- Install: `npx agentic-awesome-skills`
|
||||
- AAS Core guide: `docs/users/aas-core.md`
|
||||
- Direct skill distribution: `npx agentic-awesome-skills`
|
||||
- Choose your tool: `README.md#choose-your-tool`
|
||||
- Specialized plugins: `README.md#recommended-specialized-plugins`
|
||||
- Bundles: `docs/users/bundles.md`
|
||||
@@ -113,9 +115,9 @@ Start here:
|
||||
|
||||
Suggested pinned discussion topics:
|
||||
|
||||
- `Start here: best skills by tool`
|
||||
- `Best bundles for new users`
|
||||
- `How to choose between Antigravity and other skill libraries`
|
||||
- `Start here: compose a project stack with AAS Core`
|
||||
- `Choose between an agent-selected Core stack and direct distribution`
|
||||
- `How the catalog, plugins, bundles, and workflows support Core`
|
||||
|
||||
Discussion posts should:
|
||||
|
||||
|
||||
@@ -72,7 +72,7 @@ The update process refreshes:
|
||||
- Canonical skills index (`skills_index.json`)
|
||||
- Compatibility mirror (`data/skills_index.json`)
|
||||
- Web app skills data (`apps\web-app\public\skills.json`)
|
||||
- All 1,965+ skills from the skills directory
|
||||
- All 1,969+ skills from the skills directory
|
||||
|
||||
## When to Update
|
||||
|
||||
|
||||
Reference in New Issue
Block a user