diff --git a/tools/tsl-codegen/README.md b/tools/tsl-codegen/README.md index f821ac73..8394b07b 100644 --- a/tools/tsl-codegen/README.md +++ b/tools/tsl-codegen/README.md @@ -1,21 +1,24 @@ # TSL Codegen Toolkit -本工具把 YAML/JSON 录入文件转换为 TSL API skill 使用的 Markdown 函数文档， -并根据 Markdown 重建 `function_index.tsv` +本工具把一个或多个 tsf function、独立 class、完整 unit 转换为统一 +`declarations` json/yaml 录入文件，再生成 TSL API skill 使用的 markdown 文档， +并根据 markdown 重建统一 API 索引 `function_index.tsv`。 ## 目录结构 ```text tools/tsl-codegen/ ├─ README.md 使用说明 - ├─ STANDARD.md 函数文档与录入格式标准 + ├─ STANDARD.md API 文档与录入格式标准 ├─ examples/ - │ ├─ example.yaml YAML 录入例子 - │ └─ example.json JSON 录入例子 + │ ├─ example.yaml yaml 录入例子 + │ └─ example.json json 录入例子 ├─ scripts/ - │ ├─ generate.py YAML/JSON → Markdown - │ ├─ lint.py Markdown 格式校验 - │ └─ build_index.py 重建 function_index.tsv + │ ├─ convert_tsf.py tsf → json/yaml + │ ├─ generate.py yaml/json → markdown + │ ├─ lint.py markdown 格式校验 + │ ├─ api_markdown.py lint/index 共用标题模型 + │ └─ build_index.py 重建 13 列 function_index.tsv └─ tests/ 工具测试 ``` @@ -29,38 +32,165 @@ tools/tsl-codegen/ cd /path/to/playbook ``` -JSON 使用 Python 标准库，不需要额外安装解析包。YAML 需要安装 `pyyaml`： +json 使用 Python 标准库，不需要额外安装解析包。yaml 需要安装 `pyyaml`： ```bash python -m pip install pyyaml ``` +markdown 生成器会强制使用仓库锁定版本的 Prettier。首次使用前安装 Node.js，并在 +仓库根目录安装依赖： + +```bash +npm install +``` + ### 2. 阅读标准 先阅读 [`STANDARD.md`](STANDARD.md)。其中定义： -- Markdown 函数条目的固定结构 -- YAML/JSON 录入字段 +- function、class、unit 同级 H2 顶级声明和混合页面的固定层级 +- tsf function、独立 class 和完整 unit 文档块格式 +- yaml/json 统一 `declarations` 判别联合与成员字段 +- 13 列统一 API 索引和完全限定名称 - 参数表、返回类型和示例代码规则 -录入文件和生成的 Markdown 都必须符合该标准 +录入文件和生成的 markdown 都必须符合该标准 -### 3. 准备自己的 YAML 或 JSON +### 3. 准备自己的 yaml 或 json + +#### 从 tsf 转换 + +转换器接受任意混合输入：function、独立 class 和完整 unit TSF 可以在同一条命令中 +出现。每个文件贡献一个对外 declaration，生成的 `declarations` 严格保持命令行 +输入顺序。页面级 `module` 和 `path` 通过参数统一提供，`--format` 明确选择 +json 或 yaml： + +```bash +python tools/tsl-codegen/scripts/convert_tsf.py \ + src/OpenXmlAttribute.tsf \ + src/ParseOpenXml.tsf \ + src/OpenXmlRuntime.tsf \ + --format json \ + --module "OfficeXml / OpenXml" \ + --path officexml/openxml/elements \ + --output tmp/openxml-elements.json +``` + +生成 yaml 时，把 `--format json` 改为 `--format yaml`，并把输出文件后缀改为 +`.yaml`。混合输入规则不变。 + +查看中文帮助： + +```bash +python tools/tsl-codegen/scripts/convert_tsf.py --help +``` + +转换规则： + +- function TSF 只转换第一个独立顶层 `function` +- 独立 class 只公开第一个与文件名一致的 class；后续内部 class 和 private 成员忽略 +- class 开头默认 public；public/protected 进入草稿；类方法只接受 `class function` +- 类方法没有 `///` 描述时，使用最终声明或修饰符分号后的同一行 `//` 注释作为描述； + 正式 `///` 描述优先，类外实现行和函数体内注释不读取 +- property 类型可选；源码未声明类型时草稿中的 `type` 为空字符串，访问方式仍然必填 +- 完整 unit 只收录 interface 的 function/var/const/class；implementation 全部忽略 +- function、class、unit 可以任意混合；重复 function signature、重复 class/unit + 名称会报错，不同 signature 的 function 重载和跨 kind 同名允许 +- 公开 API 中的 `procedure` 当前不支持 +- 输入支持 utf-8（含 BOM）和 gb18030；输出统一为 utf-8 +- 参数类型和返回类型都是可选的 TSL 注解；转换器从函数签名和 `///` 文档块中尽量 + 解析，无法取得时保留空字段，生成待手工完善的录入稿 +- `@param: name {类型} 参数说明` 可以为无类型签名补充参数类型；花括号内的全部内容 + 都属于类型；`@returns:` 可以补充返回类型；签名和 `///` 同时提供类型时必须一致 +- 描述、参数类型或说明等编辑性缺口不会阻止转换；类型冲突、文档结构或声明冲突会按 + “文件:行号”报错；全部输入成功后才写输出文件 +- json/yaml 输出补齐各 kind 的固定维护字段：function/method 固定包含 + `desc`、`params`、`returns`，property 固定包含 `desc`、`type`、`params`、`access`， + field/variable 固定包含 `desc`、`type`，constant 固定包含 `desc`、`type`、`value`； + 参数固定包含 `name`、`type`、`desc`，class/unit 固定包含 `desc`、`members`；缺失值 + 使用空字符串或空数组，条件字段只在源码确实存在时输出 +- 生成器把 method `returns`、property/constant `type` 的空字符串按可选字段未填写处理； + 顶级/unit function 返回类型和 field/variable 类型等必填内容仍须补全 +- 输出文件不得与任一输入 tsf 相同 + +#### 手工准备 从以下例子选择一种格式： - [`examples/example.yaml`](examples/example.yaml) - [`examples/example.json`](examples/example.json) -例子中的函数是格式示例，不是真实 TSL API。复制例子到自己的工作目录，再修改 -`module`、`path` 和 `functions`。例如： +两份完整例子深度等价，同时包含顶级 function、完整 class、class function、 +typed/untyped property、字段、常量、完整 unit 和 interface class 成员 -```text -tmp/my-functions.yaml -tmp/my-functions.json +例子中的 API 是格式示例，不是真实 TSL API。复制例子到自己的工作目录，再修改 +`module`、`path` 和非空有序 `declarations`。录入根只允许这三个字段；旧 +`functions`、`class`、`unit` 根不兼容，生成器会直接拒绝。混合页面的最小 +完整外形： + +```json +{ + "module": "OfficeXml / OpenXml", + "path": "officexml/openxml/elements", + "declarations": [ + { + "kind": "class", + "name": "OpenXmlAttribute", + "desc": "表示 OpenXml 属性", + "members": [] + }, + { + "kind": "function", + "name": "ParseOpenXml", + "signature": "ParseOpenXml(xml)", + "desc": "解析 OpenXml 文本", + "params": [ + {"name": "xml", "type": "string", "desc": "OpenXml 文本"} + ], + "returns": "OpenXmlElement" + } + ] +} ``` -一个录入文件对应一个 Markdown 叶子页。录入文件不放入 skill；是否长期保留由 +生成的 markdown 使用纯 API 标题和独立声明行。class function 的参数类型进入参数表， +返回类型写在返回行： + +```markdown +# OfficeXml / OpenXml + +## `OpenXmlAttribute` + +声明：class + +表示 OpenXml 属性 + +### `CreateVirtual(position, row_index, _story)` + +声明：class function + +创建虚段落 + +可见性：`public` + +| 参数 | 类型 | 说明 | +| ----------- | --------- | -------- | +| `position` | integer | 段落位置 | +| `row_index` | integer | 行索引 | +| `_story` | StoryNode | 故事节点 | + +返回：ParagraphSegment +``` + +录入文件可放在自己的临时目录，例如： + +```text +tmp/my-api.yaml +tmp/my-api.json +``` + +一个录入文件对应一个 markdown 叶子页。录入文件不放入 skill；是否长期保留由 维护者自行决定 ### 4. 选择 Skill 中的目标位置 @@ -82,18 +212,18 @@ skills/tsl-api-reference/ └─ lookup.py ``` -Markdown 目标路径固定为： +markdown 目标路径固定为： ```text skills/tsl-api-reference/references/codegen///.md ``` - ``：用户文档默认使用 `project`，也可以自定义单级目录名。`builtin` 和 - `dotnet` 由 playbook 项目维护，不应用于存放用户自己的函数文档 + `dotnet` 由 playbook 项目维护，不应用于存放用户自己的 API 文档 - ``：功能分类目录，例如 `base`、`runtime`、`document` -- `.md`：同类函数的叶子文档，例如 `array.md`、`string.md` +- `.md`：相关 API 的叶子文档，例如 `array.md`、`elements.md` -录入文件的 `module` 是 Markdown 一级标题，不是目录名。例如： +录入文件的 `module` 是 markdown 一级标题，不是目录名。例如： ```text module: 我的项目 / 数组 @@ -107,63 +237,60 @@ path: base/array rg --files skills/tsl-api-reference/references/codegen/project ``` -生成后可以直接打开目标 Markdown 手动阅读。例如： +生成后可以直接打开目标 markdown 手动阅读。例如： ```text skills/tsl-api-reference/references/codegen/project/base/array.md ``` -### 5. 生成 Markdown +### 5. 生成 markdown #### 新建叶子页 目标文件不存在时，可以直接生成到 skill。例如： ```bash -python tools/tsl-codegen/scripts/generate.py tmp/my-functions.yaml +python tools/tsl-codegen/scripts/generate.py tmp/my-api.yaml ``` -JSON 使用相同命令： +json 使用相同命令： ```bash -python tools/tsl-codegen/scripts/generate.py tmp/my-functions.json +python tools/tsl-codegen/scripts/generate.py tmp/my-api.json ``` 生成器读取录入文件中的 `path`，默认写入 `skills/tsl-api-reference/references/codegen/project/.md`。不指定 `--scope` 时，scope 就是 `project` +写入前会自动使用仓库的 `.prettierrc.json` 格式化 markdown，使新页面与现有 +builtin 页面保持一致。未安装 Prettier 或格式化失败时，生成器会停止且不写目标文件 + 需要使用自定义 scope 时，通过 `--scope` 指定单级目录名： ```bash -python tools/tsl-codegen/scripts/generate.py tmp/my-functions.json --scope my-project +python tools/tsl-codegen/scripts/generate.py tmp/my-api.json --scope my-project ``` #### 修改现有叶子页 -生成器会整体覆盖 `path` 对应的页面。只有录入文件包含该页面的全部函数时才运行 -生成器。只修改现有页面中的少量函数时，应按照 `STANDARD.md` 直接编辑 Markdown +生成器会整体覆盖 `path` 对应的页面。只有录入文件包含该页面的全部 API 时才运行 +生成器。只修改现有页面中的少量条目时，应按照 `STANDARD.md` 直接编辑 markdown -### 6. 手动检查并校验 Markdown +### 6. 手动检查并校验 markdown 以下命令以新建页面 -`skills/tsl-api-reference/references/codegen/project/base/my_functions.md` 为例 +`skills/tsl-api-reference/references/codegen/project/base/my_api.md` 为例 -先打开文件，检查页面标题、函数签名、参数、返回类型和示例 +先打开文件，检查页面标题、API 名称、成员层级、参数、返回类型和示例 -#### 6.1 格式化表格（可选） +#### 6.1 格式说明 -此步骤不是必需的，仅用于对齐 Markdown 表格列宽。使用前需要安装 Node.js，并在 -仓库根目录安装 `prettier`： +通过 `generate.py` 生成的页面已经完成 Prettier 格式化，不需要再次处理。直接手工 +编辑 markdown 后，可以单独格式化目标文件： ```bash -npm install --save-dev prettier -``` - -然后格式化目标文件： - -```bash -npx prettier --write skills/tsl-api-reference/references/codegen/project/base/my_functions.md +npx --no-install prettier --write skills/tsl-api-reference/references/codegen/project/base/my_api.md ``` #### 6.2 校验 @@ -171,7 +298,7 @@ npx prettier --write skills/tsl-api-reference/references/codegen/project/base/my 校验目标文件： ```bash -python tools/tsl-codegen/scripts/lint.py --file skills/tsl-api-reference/references/codegen/project/base/my_functions.md +python tools/tsl-codegen/scripts/lint.py --file skills/tsl-api-reference/references/codegen/project/base/my_api.md ``` 校验整个项目目录： @@ -185,46 +312,55 @@ python tools/tsl-codegen/scripts/lint.py --dir skills/tsl-api-reference/referenc 校验器将以下问题视为错误： - 缺少描述 -- 缺少返回类型 +- 顶层/unit function 缺少返回类型 - 有参数但没有参数表 - 无参数但存在参数表 -- 参数表不是固定三列 +- 参数表不是固定三列，或参数名称/顺序与签名不一致 +- 参数表数据行缺少参数名、类型或说明 +- class member 缺少 visibility +- property 缺少访问方式，field/variable 缺少类型，constant 缺少值 +- class/unit 成员标题层级、声明形式或子标题错误，或出现 private API 以下问题默认作为警告： - 可选参数说明未以 `可选。` 开头 - tags 行为空 -### 7. 更新并验证函数索引 +### 7. 更新并验证统一 API 索引 -Markdown 确认无误后，重建 TSV： +markdown 确认无误后，重建 TSV： -索引会分别保存函数的 tags 和描述。关键词检索会同时匹配函数名、签名、模块、 -tags 和描述 +索引固定为 13 列：保留原 8 列并追加 kind、binding、visibility、owner、 +qualified_name。关键词检索会同时匹配原有字段和这些扩展字段。 ```bash python tools/tsl-codegen/scripts/build_index.py --skill-dir skills/tsl-api-reference ``` -检查 TSV 是否与 Markdown 一致： +检查 TSV 是否与 markdown 一致： ```bash python tools/tsl-codegen/scripts/build_index.py --skill-dir skills/tsl-api-reference --check ``` -默认从 `/references/codegen` 读取 Markdown，并写入 +默认从 `/references/codegen` 读取 markdown，并写入 `/data/function_index.tsv` -最后使用函数名验证 skill 可以检索到新条目。把 `myFunction` 替换为真实函数名： +最后使用简单名称或完全限定名称验证 skill 可以检索到新条目： ```bash python skills/tsl-api-reference/scripts/lookup.py --name myFunction +python skills/tsl-api-reference/scripts/lookup.py --name DemoUnit.Document.Save ``` +查询简单成员名会返回所有 owner 下的同名 API；`Unit.Class.Member` 等完全限定名称 +用于缩小范围。重载共享完全限定名称，查询会返回全部重载。关键词候选显示 +`class function`、`static field` 等实际声明术语。 + 最终提交： -- 新增或修改的 Markdown 叶子页 +- 新增或修改的 markdown 叶子页 - `skills/tsl-api-reference/data/function_index.tsv` -TSL API skill 只需要 Markdown 和 TSV。录入用的 YAML/JSON 可以由维护者在自己的 +TSL API skill 只需要 markdown 和 TSV。录入用的 yaml/json 可以由维护者在自己的 版本库中管理 diff --git a/tools/tsl-codegen/STANDARD.md b/tools/tsl-codegen/STANDARD.md index 459fc2b9..2b7630d5 100644 --- a/tools/tsl-codegen/STANDARD.md +++ b/tools/tsl-codegen/STANDARD.md @@ -1,41 +1,127 @@ -# TSL 函数文档标准 +# TSL API 文档标准 -本文件是 TSL codegen 函数文档的唯一标准，包含 Markdown 存储格式和 YAML/JSON -录入格式 +本文件是 TSL codegen API 文档的唯一标准，包含 markdown 存储格式、tsf 源码 +文档格式、yaml/json 录入格式和统一索引格式。 -适用范围：`references/codegen/**/*.md` 中的函数条目 +适用范围：`references/codegen/**/*.md` 中的 function、class 和 unit API。 + +## 目录 + +- [TSL API 文档标准](#tsl-api-文档标准) + - [目录](#目录) + - [基本原则](#基本原则) + - [markdown 存储格式](#markdown-存储格式) + - [页面与统一顶级声明](#页面与统一顶级声明) + - [function](#function) + - [class](#class) + - [unit](#unit) + - [综合示例](#综合示例) + - [tsf 源码文档格式](#tsf-源码文档格式) + - [function](#function-1) + - [class](#class-1) + - [unit](#unit-1) + - [录入数据结构](#录入数据结构) + - [function](#function-2) + - [class](#class-2) + - [unit](#unit-2) + - [录入格式（用户必看）](#录入格式用户必看) + - [yaml](#yaml) + - [json](#json) + - [统一 API 索引](#统一-api-索引) ## 基本原则 -- Markdown 是唯一存储源 -- YAML/JSON 用于生成 Markdown -- 函数签名由维护者提供；工具原样输出，不校正签名内容 +- markdown 是唯一存储源 +- yaml/json 用于生成 markdown +- tsf 源码文档可以转换为 json 或 yaml 录入稿，再由同一生成流程产出 markdown +- 一个录入文件对应一个 markdown 叶子页；录入根固定为 + `module`、`path`、`declarations` +- `declarations` 是非空有序列表，function、class、unit 可以按输入顺序混合 +- 生成器写入 markdown 前必须使用仓库锁定版本的 Prettier 统一格式 +- 转换器允许生成描述或类型不完整的草稿；生成器严格校验，不发布不完整录入稿 +- function 调用签名由维护者或转换器提供；工具不推断 API 语义 +- API 标题的反引号内只写名称或调用签名，不写 function、class、property 等声明术语 +- 每个 API 用独立的 `声明：...` 行记录声明种类 +- 描述紧跟声明行；tags 是描述的检索补充，位于完整描述之后 +- 描述末尾不写句号 - 示例输出必须全部写成 `//` 注释，不得把裸结果写成 TSL 语句 - 一个 `tsl` 代码块只放一个独立示例 -## Markdown 存储格式 +## markdown 存储格式 -每个函数条目按以下顺序书写： +### 页面与统一顶级声明 -1. `## \`函数签名\`` -2. ``，可选 -3. 函数描述，必填；首个非空内容必须是描述 -4. 参数表，有参数时必填 -5. 参数取值说明，可选 -6. `返回：类型` -7. `### 示例` 和 `tsl` 代码块，可选 +每个 markdown 叶子页使用一个 H1 模块标题，并包含一个或多个 H2 顶级声明。 +function、class、unit 都是同级的顶级声明，可以出现在同一页面中；每个 H2 的声明 +种类写在自身正文的 `声明：function|class|unit` 行中。 -### 标签 +目录承担领域或模块分组，叶子页承担 API 主题分组，H2 承担顶级声明分组。markdown +不要求一个 class 对应一个 md；多个独立 class TSF 的对外声明可以进入同一叶子页， +也可以在内容过长或主题不同时拆到同一目录下的多个叶子页。例如 +`OpenXmlAttribute` 和 `OpenXmlElement` 可以作为两个 H2 同处 +`officexml/openxml/elements.md`。每个 TSF 文件对应一个对外顶级声明，一个 markdown +叶子页可以收录多个对外顶级声明。 -标签紧跟函数签名，用空格分隔检索关键词： +顶级声明和成员都按文档顺序存储，不按名称、种类或可见性重新排序。每个顶级声明 +按以下共同顺序开始： + +1. H2 标题：function 写调用签名，class 和 unit 写简单名称 +2. 唯一且精确的 `声明：function`、`声明：class` 或 `声明：unit` +3. 描述，必填；声明行之后的首个非空正文必须是描述 +4. ``，可选；位于完整描述之后 + +标题层级和声明值必须符合下表，不允许在其他层级使用这些声明值： + +| 位置 | 标题层级 | 允许的声明值 | +| ---------------------------- | -------- | ------------------------------------------------------------------------------------------ | +| 顶级声明 | H2 | `function`、`class`、`unit` | +| 顶级 class 成员 | H3 | `function`、`class function`、`property`、`field`、`static field`、`const`、`static const` | +| unit interface direct member | H3 | `function`、`var`、`const`、`class` | +| unit interface class 成员 | H4 | 与顶级 class 成员相同 | + +H2、H3、H4 中反引号包围的内容是 API 名称或调用签名。`function`、 +`class function`、`property` 等声明术语只写在声明行中，不写进标题。 + +本标准中的“调用签名”只包含名称和参数名，例如 `abs(x)`，不是 TSL 声明语句 +`function abs(x);`。声明行只参与文档结构识别，不属于名称、调用签名或调用语法。 +例如 class 下的 H3 标题 `Save()` 紧跟 `声明：function`，表示实例 function；可调用 +内容仍是 `Save()`，不能把两行拼成 `function Save()`。 + +每个 API 必须且只能写一行 `声明：...`，并将它作为标题后的第一条非空正文。 +`声明` 后使用全角冒号，声明值使用本标准规定的小写形式。普通内容标题不包含 +反引号，也不紧跟声明行，因此不进入 API 索引；普通 H2 同时结束前一个顶级声明的 +正文范围。 + +同一所属范围和声明种类内，重复 class/unit 名称或重复 function 调用签名不允许； +function 重载和跨声明种类同名允许。 + +### function + +#### 页面结构 + +顶级 function 使用 H2，function 的示例标题使用 H3。每个 function 声明按以下 +顺序书写： + +1. `## \`调用签名\`` +2. `声明：function` +3. 函数描述，必填 +4. ``，可选 +5. 参数表，有参数时必填 +6. 参数取值说明，可选 +7. `返回：类型` +8. `### 示例`，可选；包含一个或多个“范例NN：说明”和独立的 `tsl` 代码块 + +#### 标签 + +标签位于完整函数描述之后，用空格分隔检索关键词： ```markdown ``` -没有关键词时删除整行，不保留空标签 +没有关键词时删除整行，不保留空标签。 -### 参数表 +#### 参数表 参数表固定为三列： @@ -47,13 +133,14 @@ 规则： -- 参数名必须与函数签名一致 +- 参数名必须与 function 调用签名一致 - 必填参数说明直接写用途 - 可选参数说明以 `可选。` 开头，并说明默认值 - 多类型使用 `\|`，例如 `nil\|array` - 变参使用 `...` 作为参数名 +- 无参 function 省略参数表 -### 参数取值 +#### 参数取值 枚举参数在参数表之后、返回类型之前列出： @@ -64,20 +151,23 @@ - `1` — 去重 ``` -### 返回类型 +#### 返回类型 -每个函数必须包含非空返回类型： +每个顶层 function 必须包含非空返回类型： ```markdown 返回：array ``` -### 示例代码 +#### 示例代码 +- 每个示例以“范例NN：说明”开头，编号按出现顺序自动生成并至少保留两位 - 代码围栏使用 `tsl` +- 一个代码围栏只放一个独立示例 - 字符串使用直引号 `'` 或 `"` - 注释使用 `//`，不使用 `(* *)` - 每条语句保留分号 +- 没有输出时可以省略输出注释 - 单行输出写成 `// 输出：<值>` - 多行输出第一行写 `// 输出：`，后续每一行输出都以 `//` 开头 @@ -90,22 +180,27 @@ return demoLines(); // 第二行 ``` -### 完整条目示例 +#### 完整页面示例 -以下函数仅用于说明文档格式，不代表真实 TSL API +以下 API 仅用于说明存储格式，不代表真实 TSL API： ````markdown +# 示例 / 数组 + ## `demoFn(src, mode, factor, ...)` - + +声明：function 按指定模式处理数组并返回结果 -| 参数 | 类型 | 说明 | -| -------- | ---------- | --------------------------------- | -| `src` | array | 待处理数组 | -| `mode` | integer | 处理模式，取值见下。 | -| `factor` | float | 可选。默认 1.0，结果乘以该系数。 | -| `...` | nil\|array | 可选。需要追加处理的其他数组。 | + + +| 参数 | 类型 | 说明 | +| -------- | ---------- | -------------------------------- | +| `src` | array | 待处理数组 | +| `mode` | integer | 处理模式，取值见下 | +| `factor` | float | 可选。默认 1.0，结果乘以该系数 | +| `...` | nil\|array | 可选。需要追加处理的其他数组 | **mode 取值** @@ -116,52 +211,787 @@ return demoLines(); ### 示例 +范例01：指定处理模式和系数 + ```tsl src := array(1, 1, 2); return demoFn(src, 1, 2.0); // 输出：array(2,4) ``` -```` -无参函数省略参数表： +范例02：使用默认系数 + +```tsl +src := array(1, 1, 2); +return demoFn(src, 0); +``` -````markdown ## `demoNow()` +声明：function + 返回示例值 返回：integer +```` + +### class + +#### 页面结构 + +顶级 class 使用 H2，class 成员使用 H3，function 的参数取值或示例使用 H4。H2 正文 +按以下顺序书写： + +1. `## \`ClassName\`` +2. `声明：class` +3. class 描述，必填 +4. tags，可选；位于完整描述之后 +5. `父类：BaseClass`，可选；多父类按声明顺序列出 +6. class 成员，按源码顺序书写 + +#### 成员标题与正文 + +class 成员标题统一只写反引号包围的名称或调用签名，例如 +`### \`Create(name)\``、`### \`Title\``。成员的声明种类由标题后的声明行表达： + +- 实例方法：`声明：function` +- 类方法：`声明：class function` +- property：`声明：property` +- 实例/静态字段：`声明：field` / `声明：static field` +- 实例/静态常量：`声明：const` / `声明：static const` + +每个成员都按“标题、声明行、描述、tags、成员数据”的共同顺序开始。function 的 +正文结构与顶级 function 相同，实例 function 和 class function 都按描述、tags、 +参数表、参数取值、返回类型、示例的顺序书写。源码声明了返回类型时必须写 +`返回：类型`；没有返回类型时省略，不补写 `void`。位于 class 中时， +标题及其子标题整体下移一级：function 标题从 H2 变为 H3，参数取值和示例标题从 H3 +变为 H4。成员另外必须写 `可见性：public|protected`，并可写 `修饰符`；类方法由 +`声明：class function` 表达。 + +property 必须写访问方式；有类型时写 `类型：...`，没有类型时省略。field 必须写 +类型，const 必须写值。 + +#### 完整页面示例 + +以下示例以 `OpenXmlAttribute` 的公开成员为基础，并加入一个 class function，用于 +说明 class 的 markdown 存储格式： + +```markdown +# OfficeXml / OpenXml + +## `OpenXmlAttribute` + +声明：class + +表示一个 OpenXml 属性 + + + +### `create(_prefix, _local_name)` + +声明：function + +创建不带初始值的 OpenXml 属性 + + + +可见性：`public` + +修饰符：`overload` + +| 参数 | 类型 | 说明 | +| ------------- | ------ | ------------ | +| `_prefix` | string | 命名空间前缀 | +| `_local_name` | string | 本地名称 | + +### `create(_prefix, _local_name, _value)` + +声明：function + +创建带初始值的 OpenXml 属性 + + + +可见性：`public` + +修饰符：`overload` + +| 参数 | 类型 | 说明 | +| ------------- | ------ | ------------ | +| `_prefix` | string | 命名空间前缀 | +| `_local_name` | string | 本地名称 | +| `_value` | any | 属性值 | + +### `CreateVirtual(position, row_index, _story)` + +声明：class function + +创建虚段落 + +可见性：`public` + +| 参数 | 类型 | 说明 | +| ----------- | --------- | -------- | +| `position` | integer | 段落位置 | +| `row_index` | integer | 行索引 | +| `_story` | StoryNode | 故事节点 | + +返回：ParagraphSegment + +### `Prefix` + +声明：property + +命名空间前缀 + +可见性：`public` + +类型：string + +访问：read / write + +### `LocalName` + +声明：property + +本地名称 + +可见性：`public` + +类型：string + +访问：read / write + +### `ElementName` + +声明：property + +包含前缀的完整属性名称 + +可见性：`public` + +类型：string + +访问：read / write + +### `Value` + +声明：property + +属性值 + +可见性：`public` + +类型：any + +访问：read / write + +### `NamespaceUri` + +声明：property + +命名空间 URI + +可见性：`public` + +类型：string + +访问：read / write +``` + +### unit + +#### 页面结构 + +顶级 unit 使用 H2，interface direct member 使用 H3，interface class 的成员或 +unit function 的参数取值/示例使用 H4，interface class function 的参数取值/示例 +使用 H5。H2 正文按以下顺序书写： + +1. `## \`UnitName\`` +2. `声明：unit` +3. unit 描述，必填 +4. tags，可选；位于完整描述之后 +5. interface direct member，按源码顺序书写 + +#### 成员标题与正文 + +H3 direct member 标题只写名称或调用签名。标题后的声明行只允许： + +- interface function：`声明：function` +- variable：`声明：var` +- constant：`声明：const` +- interface class：`声明：class` + +每个 direct member 都按“标题、声明行、描述、tags、成员数据”的共同顺序开始。 +unit direct member 都来自 interface，因此正文不重复写 public；统一索引把其 +visibility 规范化为 `public`。unit function 复用顶级 function 格式并要求返回 +类型，var 必须写类型，const 必须写值。interface class 的 H4 成员复用顶级 +class 的成员格式，并显式写 `可见性：public|protected`。 + +#### 完整页面示例 + +```markdown +# 示例 / 文档运行时 + +## `DocumentUnit` + +声明：unit + +提供文档运行时接口 + + + +### `DefaultSize` + +声明：const + +默认缓冲区大小 + +值：`100` + +### `CurrentDocument` + +声明：var + +当前文档 + +类型：Document + +### `OpenDocument(path)` + +声明：function + +打开文档 + +| 参数 | 类型 | 说明 | +| ------ | ------ | -------- | +| `path` | string | 文档路径 | + +返回：Document + +### `Document` + +声明：class + +文档对象 + +父类：`BaseDocument` + +#### `Save()` + +声明：function + +保存文档 + +可见性：`public` + +返回：boolean +``` + +### 综合示例 + +同一叶子页可以按录入顺序混合三种顶级声明。以下 API 仅用于说明存储格式，不代表 +真实 TSL API： + +````markdown +# OfficeXml / OpenXml + +## `OpenXmlAttribute` + +声明：class + +表示 OpenXml 属性 + + + +### `create(_prefix, _local_name)` + +声明：function + +创建属性 + + + +可见性：`public` + +| 参数 | 类型 | 说明 | +| ------------- | ------ | ------------ | +| `_prefix` | string | 命名空间前缀 | +| `_local_name` | string | 本地名称 | + +## `ParseOpenXml(xml)` + +声明：function + +解析 OpenXml 文本 + + + +| 参数 | 类型 | 说明 | +| ----- | ------ | ------------ | +| `xml` | string | OpenXml 文本 | + +返回：OpenXmlElement ### 示例 +范例01：解析文本 + ```tsl -return demoNow(); -// 输出：1 +return ParseOpenXml(''); ``` + +## `OpenXmlRuntime` + +声明：unit + +提供 OpenXml 运行时接口 + + + +### `OpenXmlElement` + +声明：class + +表示文档元素 + + + +#### `Save()` + +声明：function + +保存元素 + +可见性：`public` + +返回：boolean ```` +## tsf 源码文档格式 + +tsf 主要使用 `function`、`unit` 和 `type` 三种顶层组织方式。本标准按这三种方式 +分别定义源码文档格式。转换器可以一次接收任意混合的 TSF 输入；每个文件贡献一个 +对外顶级声明，并严格保持命令行输入顺序。 + +### function + +独立顶层 `function` 可以在源码中记录录入数据所需的函数级内容。签名、参数类型、 +默认参数和返回类型直接读取源码声明。 + +- 一个 tsf 文件只记录第一个主函数；文件中的后续辅助函数不进入录入数据 +- 顶层 `procedure` 不按 `function` 格式处理 +- 多个 tsf 可以与 class/unit TSF 混合生成同一个 json/yaml 录入文件；页面级 + `module` 和 `path` 在转换时统一提供，不写进单个函数的注释 + +#### 文档块位置 + +文档块必须是主函数 `begin` 之后的第一段非空内容，并且位于任何可执行语句、 +编译指令或其他注释之前 + +文档块由连续的 `///` 行组成。允许按照函数体缩进；解析时忽略 `///` 之前的空白， +并移除 `///` 及其后的一个可选空格。遇到第一行非 `///` 内容时，文档块结束；函数体 +后续位置的注释是普通注释 + +TSL 解释器将 `///` 作为普通的 `//` 行注释；第三个 `/` 是 codegen 用来识别文档行的 +标记 + +#### 文档块结构 + +文档块按以下顺序书写： + +1. 函数描述，一行或多行，必填 +2. `@tags: 标签1 标签2`，可选，最多一行，标签使用空白分隔 +3. 参数组，按声明顺序书写；每组先写 `@param: name 参数说明`，再按需紧跟一个 + `@values: name` +4. `@returns: 类型`，可选，最多一行 +5. 示例组，可选，可以重复；所有示例组必须位于文档块末尾 + +空的 `///` 行可以在多行函数描述或示例中保留空行。指令名固定为小写；未知指令、 +重复的 `@tags:`/`@returns:`、同一参数重复的 `@param:`/`@values:`，以及不符合上述 +顺序的指令均视为错误。`@example:` 可以重复，`@output:` 在同一示例组内最多出现 +一次。 + +| 写法 | 必填 | json 映射 | 规则 | +| ----------------------- | -------- | ---------------------------------- | -------------------------------------------------------------- | +| 描述正文 | 是 | `declarations[].desc` | 第一条指令之前的所有正文，保留换行 | +| `@tags: 标签1 标签2` | 否 | `declarations[].tags` | 使用一个或多个空白字符分隔；忽略首尾空白；标签本身不得包含空白 | +| `@param: name 参数说明` | 有参时 | `declarations[].params[].desc` | 第一个词是参数名，其余内容是说明；说明不得为空 | +| `@values: name` | 否 | `declarations[].params[].values` | 值表规则见下；参数必须已经由 `@param:` 声明 | +| `@returns: 类型` | 否 | `declarations[].returns` | 类型不得为空；与声明返回类型同时存在时必须一致 | +| `@example: 示例说明` | 否 | `declarations[].examples[].desc` | 开始一个示例组；说明不得为空 | +| 示例代码 | 示例组内 | `declarations[].examples[].code` | 内容行额外缩进两个空格；至少包含一个非空代码行 | +| `@output:` | 否 | `declarations[].examples[].output` | 原始输出额外缩进两个空格；存在时不得为空 | + +`@param:` 和 `@values:` 中的参数名与函数声明大小写无关地匹配，json 使用函数声明中的 +参数拼写。 + +#### 字段来源与映射 + +以下字段由主函数声明以及必要的文档指令确定： + +| tsf 声明内容 | json 字段 | 转换规则 | +| -------------- | ---------------------------------- | --------------------------------------------------- | +| 函数名 | `declarations[].name` | 保留声明名称 | +| 函数名和参数名 | `declarations[].signature` | 规范化为只含名称的调用形式，不复制类型或默认值 | +| 参数类型 | `declarations[].params[].type` | 保留声明中的类型 | +| 参数默认值 | `declarations[].params[].optional` | 存在默认值时写入 `true`，默认表达式不另建 json 字段 | +| 返回类型 | `declarations[].returns` | 读取声明和 `@returns:`，按下述规则合并 | + +要直接得到可生成 markdown 的完整录入数据，函数必须显式声明每个参数的类型，为每个 +参数提供非空的 `@param:`，并通过函数声明或 `@returns:` 提供返回类型。缺少这些内容 +时只能得到待手工完善的录入稿。 +可选参数的说明仍应写清默认值含义；`optional: true` 只表达该参数可以省略。 + +返回类型按以下规则合并： + +- 只有函数声明时，使用声明中的拼写 +- 只有 `@returns:` 时，去除首尾空白后使用指令中的拼写 +- 两处同时存在时，转换器使用与函数声明相同的词法规则拆分类型，忽略 token 之间的 + 空白，并按 TSL 标识符大小写无关的规则比较标识符 token；其他 token 必须一致。 + 校验通过后使用声明中的拼写，校验失败则定位 `@returns:` 行、报错并停止转换 +- 两处都不存在时，录入稿中的 `returns` 为空，不能直接生成 markdown + +转换器只校验类型的词法结构，不判断类型别名等语义等价。 + +#### 枚举值表 + +枚举值表使用分组格式，每一项在 `///` 标记后额外缩进两个空格： + +```text +/// @param: mode 处理模式，默认 0 +/// @values: mode +/// 0: 原样返回 +/// 1: 去重 +/// "auto": 自动判断 +``` + +规则： + +- 枚举值使用 json 标量写法：数字直接写，字符串使用双引号，布尔值使用 + `true`/`false`；`nil` 等符号值按字符串写成 `"nil"` +- 不接受数组、对象或 json `null` +- 值和说明以值后的第一个分隔冒号分开；说明不得为空 +- 保留枚举项的声明顺序 +- 同一参数最多有一个非空值表，重复值视为错误；不同 json 类型的值不视为重复， + 例如数字 `1` 与字符串 `"1"` 是两个值 +- `@values:` 引用的参数必须存在 +- 转换器不推断枚举值是否与参数类型兼容，该语义由 tsf 作者负责 + +#### 示例组 + +每个示例组以非空的 `@example: 示例说明` 开始。其后的代码行必须在 `///` 标记后 +额外缩进两个空格。可选的 `@output:` 结束代码部分，其后的缩进行记录原始输出；下一个 +`@example:` 开始新的示例组。第一个示例组出现后，不得再写参数、返回类型等其他函数 +级指令。 + +转换时移除代码和输出的两个结构缩进，保留其余空白和换行。示例说明、代码和输出 +分别映射为 `examples[].desc`、`examples[].code` 和 `examples[].output`。`desc` 与 +`code` 必填且非空；`output` 可选，但出现时必须包含至少一个非空行。示例顺序保持 +不变。 + +`code` 只记录示例源码，不得包含标准输出标记 `// 输出：`。`output` 只记录原始输出， +不写 `//` 注释标记。生成 markdown 时： + +- 所有示例共用一个 `### 示例` 标题 +- 每项按顺序生成“范例01：说明”“范例02：说明” +- 每项生成一个独立的 `tsl` 代码块 +- 没有 `output` 时，代码块只包含 `code` +- 单行 `output` 在代码末尾生成 `// 输出：<值>` +- 多行 `output` 先生成 `// 输出：`，再为每个输出行添加 `//` 和一个空格；空输出行生成 + 单独的 `//` + +#### 完整 tsf 示例 + +以下内容应保存为 `Normalize.tsf`： + +```tsl +function Normalize(mode: integer = 0): integer; +begin + /// 按指定模式处理并返回模式值 + /// @tags: 示例 枚举 + /// @param: mode 处理模式，默认 0 + /// @values: mode + /// 0: 原样返回 + /// 1: 去重 + /// 2: 排序 + /// @returns: integer + /// @example: 使用默认模式 + /// return Normalize(); + /// @output: + /// 0 + /// @example: 指定模式 + /// return Normalize(1); + return mode; +end; +``` + +对应的顶级 function 录入对象为： + +```json +{ + "kind": "function", + "name": "Normalize", + "signature": "Normalize(mode)", + "desc": "按指定模式处理并返回模式值", + "tags": ["示例", "枚举"], + "params": [ + { + "name": "mode", + "type": "integer", + "optional": true, + "desc": "处理模式，默认 0", + "values": [ + {"value": 0, "desc": "原样返回"}, + {"value": 1, "desc": "去重"}, + {"value": 2, "desc": "排序"} + ] + } + ], + "returns": "integer", + "examples": [ + { + "desc": "使用默认模式", + "code": "return Normalize();", + "output": "0" + }, + { + "desc": "指定模式", + "code": "return Normalize(1);" + } + ] +} +``` + +### class + +`Counter.tsf` + +```tsl +type Counter = class +/// 表示计数器 +/// @tags: 示例 计数 +public + /// 创建计数器 + /// @param: initial_value 初始值 + function create(initial_value: integer); + + /// 计算两个整数之和 + /// @tags: 示例 加法 + /// @param: x 第一个整数 + /// @param: y 第二个整数 + /// @returns: integer + class function Add(x: integer; y: integer): integer; + begin + return x + y; + end; + + /// 增加当前值 + /// @param: step 增量 + /// @returns: integer + /// @example: 增加计数 + /// counter := new Counter(1); + /// return counter.Increase(2); + /// @output: + /// 3 + function Increase(step: integer): integer; + + /// 当前值 + /// @tags: 状态 + property Value read value_ write value_; + + /// 显示名称 + Label: string; + + /// 最小值 + const Minimum = 0; + +protected + /// 已创建的计数器数量 + static CreatedCount: integer; + + /// 最大值 + static const Maximum = 100; + +private + /// 内部值 + value_: integer; +end; + +type CounterState = class +/// 表示内部状态 +public + /// 状态码 + Code: integer; +end; + +function Counter.create(initial_value: integer); +begin + value_ := initial_value; +end; + +function Counter.Increase(step: integer): integer; +begin + value_ := value_ + step; + return value_; +end; +``` + +对应的 class 录入对象为： + +```json +{ + "kind": "class", + "name": "Counter", + "desc": "表示计数器", + "tags": ["示例", "计数"], + "members": [ + { + "kind": "method", + "name": "create", + "visibility": "public", + "binding": "instance", + "signature": "create(initial_value)", + "desc": "创建计数器", + "params": [ + { + "name": "initial_value", + "type": "integer", + "desc": "初始值" + } + ] + }, + { + "kind": "method", + "name": "Add", + "visibility": "public", + "binding": "class", + "signature": "Add(x, y)", + "desc": "计算两个整数之和", + "tags": ["示例", "加法"], + "params": [ + { + "name": "x", + "type": "integer", + "desc": "第一个整数" + }, + { + "name": "y", + "type": "integer", + "desc": "第二个整数" + } + ], + "returns": "integer" + }, + { + "kind": "method", + "name": "Increase", + "visibility": "public", + "binding": "instance", + "signature": "Increase(step)", + "desc": "增加当前值", + "params": [ + { + "name": "step", + "type": "integer", + "desc": "增量" + } + ], + "returns": "integer", + "examples": [ + { + "desc": "增加计数", + "code": "counter := new Counter(1);\nreturn counter.Increase(2);", + "output": "3" + } + ] + }, + { + "kind": "property", + "name": "Value", + "visibility": "public", + "desc": "当前值", + "tags": ["状态"], + "access": "readwrite" + }, + { + "kind": "field", + "name": "Label", + "visibility": "public", + "desc": "显示名称", + "type": "string" + }, + { + "kind": "constant", + "name": "Minimum", + "visibility": "public", + "desc": "最小值", + "value": "0" + }, + { + "kind": "field", + "name": "CreatedCount", + "visibility": "protected", + "desc": "已创建的计数器数量", + "type": "integer", + "static": true + }, + { + "kind": "constant", + "name": "Maximum", + "visibility": "protected", + "desc": "最大值", + "value": "100", + "static": true + } + ] +} +``` + +### unit + +unit 只接受包含显式 `interface`、`implementation` 并以 `end.` 结束的完整形态。 +unit 名称必须与文件名大小写无关地一致；简写 unit 明确报错。 + +unit 文档块位于 `unit Name;` 之后、`interface` 之前，只允许描述和可选 +`@tags:`。转换器按源码顺序收录 interface 中的 function、var、const 和 class； +`uses` 只表示依赖，不进入 API。interface class 完整复用独立 class 的成员规则， +但 interface 中的所有 class 都进入文档。受支持区域中无法绑定到 unit 或 interface +成员的 `///` 文档块按原始行号报错。 + +进入 implementation 后停止收集 API；其中的函数、类、变量、常量和 `///` 文档块 +全部忽略。interface 中的 procedure 和非 class type 不支持，必须在声明行报错， +不能静默遗漏。interface function 复用 function 文档指令并要求最终返回类型； +var/const 只允许描述和可选 `@tags:`，且一项一条声明。 + ## 录入数据结构 -一个录入文件对应一个 Markdown 叶子页 +一个录入文件对应一个 markdown 叶子页。录入根只允许 `module`、`path`、 +`declarations`。`declarations` 必须是非空有序列表，function、class、unit 可以 +任意混合，生成器严格保持数组顺序。 顶层字段： -| 字段 | 必填 | 说明 | -| ----------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `module` | 是 | Markdown 一级标题内容；不是目录名或文件名。例如 `module: 示例 / 数组` 生成 `# 示例 / 数组` | -| `path` | 是 | 目标 Markdown 在 scope 目录下的相对路径，包含子目录和文件名，使用 `/` 分隔且不含 `.md` 后缀。例如 `path: base/example` 在默认 `project` scope 下生成 `references/codegen/project/base/example.md` | -| `functions` | 是 | 非空函数列表 | +| 字段 | 必填 | 说明 | +| -------------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `module` | 是 | markdown 一级标题内容；不是目录名或文件名。例如 `module: 示例 / 数组` 生成 `# 示例 / 数组` | +| `path` | 是 | 目标 markdown 在 scope 目录下的相对路径，包含子目录和文件名，使用 `/` 分隔且不含 `.md` 后缀。例如 `path: base/example` 在默认 `project` scope 下生成 `references/codegen/project/base/example.md` | +| `declarations` | 是 | 非空顶级声明列表；每项由 `kind` 判别并要求 `name`，按数组顺序生成 H2 | -函数字段： +### function + +顶级 function 对象字段： | 字段 | 必填 | 说明 | | ----------- | ------ | ------------------------------------------------------ | -| `signature` | 是 | 维护者提供的完整函数签名 | +| `kind` | 是 | 固定为 `function` | +| `name` | 是 | function 简单名称；必须与 `signature` 中的名称一致 | +| `signature` | 是 | 维护者提供的调用签名 | | `desc` | 是 | 函数描述，可包含多行 | | `tags` | 否 | 检索关键词列表；推荐填写，有助于更准确地识别和检索函数 | | `params` | 有参时 | 参数列表；无参函数省略 | | `returns` | 是 | 返回类型 | -| `example` | 否 | 不含代码围栏的 TSL 示例 | +| `examples` | 否 | 示例列表；按顺序生成独立的 TSL 代码块 | 参数字段： @@ -180,27 +1010,127 @@ return demoNow(); | `value` | 是 | 枚举值 | | `desc` | 是 | 枚举含义 | -## YAML 录入格式 +`examples` 每项包含： -YAML 适合包含多行示例的页面。解析 YAML 需要安装 `pyyaml` +| 字段 | 必填 | 说明 | +| -------- | ---- | ---------------------------------------------------- | +| `desc` | 是 | 示例场景说明；生成“范例NN：说明” | +| `code` | 是 | 不含代码围栏和标准输出注释的 TSL 代码 | +| `output` | 否 | 原始输出；生成器按单行或多行规则转换为 `//` 输出注释 | + +### class + +class 对象字段： + +| 字段 | 必填 | 说明 | +| --------- | ---- | ----------------------------------------- | +| `kind` | 是 | 固定为 `class` | +| `name` | 是 | class 简单名称 | +| `desc` | 是 | class 描述 | +| `tags` | 否 | 检索关键词列表 | +| `bases` | 否 | 按声明顺序保存的直接父类列表 | +| `members` | 是 | method/property/field/constant 的有序列表 | + +所有 class member 都要求 `name`、`desc` 和 `visibility`。 +所有 class member 都可以包含可选的 `tags`。visibility 只能是 `public` 或 +`protected`。成员判别字段如下： + +| kind | 关键字段 | +| ---------- | --------------------------------------------------------------------------------- | +| `method` | `signature`、`binding: instance\|class`；可选 `params/returns/modifiers/examples` | +| `property` | `access: read\|write\|readwrite`；可选 `type`，参数化时包含 `params` | +| `field` | `type`；可选 `static: true` | +| `constant` | `value`；可选 `type`、`static: true` | + +`kind: method` 对象不使用 `static` 字段；class function 由 `binding: class` 唯一 +表达。`value` 的存在性与真假值分开判断，因此数字 `0` 和布尔值 `false` 都是合法 +常量值。 +这里的 `method` 是录入结构内部用于归一化实例 function 和 class function 的 kind， +不会写进 markdown 标题或 `声明：...` 行。 + +### unit + +顶级 unit 对象包含固定的 `kind: unit`，以及必填的 `name`、`desc`、`members` +和可选 `tags`。direct member 按 interface 源码顺序保存，kind 只允许： + +| kind | 关键字段 | +| ---------- | -------------------------------------------------------- | +| `function` | 复用顶层 function 数据；`returns` 必填 | +| `variable` | `name`、`desc`、`type`，可选 `tags` | +| `constant` | `name`、`desc`、`value`，可选 `tags/type` | +| `class` | 复用完整 class 数据，并以 `kind: class` 作为成员判别字段 | + +录入数据不保存 unit 的 `uses`、implementation、initialization 或 finalization。 + +## 录入格式（用户必看） + +### yaml + +yaml 适合包含多行示例的页面。解析 yaml 需要安装 `pyyaml` 注意： -- `example` 使用 `|` 块标量 +- `examples[].code` 和多行 `examples[].output` 使用 `|` 块标量 - 参数名 `...` 必须加引号 - `nil|array` 可直接作为普通字符串值 完整例子：[examples/example.yaml](examples/example.yaml) -## JSON 录入格式 +### json -JSON 使用 Python 标准库解析，无额外依赖 +json 使用 Python 标准库解析，无额外依赖 注意： -- JSON 不支持注释 -- 多行示例使用 `\n` +- json 不支持注释 +- 多行 `examples[].code` 和 `examples[].output` 使用 `\n` - 字符串内部的双引号使用 `\"` - 结构标点必须使用半角字符 完整例子：[examples/example.json](examples/example.json) + +## 统一 API 索引 + +`function_index.tsv` 固定为 13 列： + +```text +name scope module signature page anchor tags summary kind binding visibility owner qualified_name +``` + +字段含义： + +- `kind`：`function|class|method|property|field|constant|unit|variable` +- `binding`：不适用时为空，否则为 `instance|class|static|unit` +- `visibility`：class member 为 `public|protected`，unit direct member 为 `public` +- `owner`：不含该 API 名称的完整所属路径 +- `qualified_name`：点分隔的稳定文档身份 + +索引的 `kind` 和 `binding` 由声明行和所属层级按下表确定： + +| markdown 位置 | `声明：...` | 录入对象表达 | 索引 `kind` | 索引 `binding` | +| ---------------------------- | ---------------- | --------------------------------- | ----------- | -------------- | +| H2 顶级声明 | `function` | `kind: function` | `function` | 空 | +| H2 顶级声明 | `class` | `kind: class` | `class` | 空 | +| H2 顶级声明 | `unit` | `kind: unit` | `unit` | 空 | +| 顶级/interface class 成员 | `function` | `kind: method, binding: instance` | `method` | `instance` | +| 顶级/interface class 成员 | `class function` | `kind: method, binding: class` | `method` | `class` | +| 顶级/interface class 成员 | `property` | `kind: property` | `property` | `instance` | +| 顶级/interface class 成员 | `field` | `kind: field` | `field` | `instance` | +| 顶级/interface class 成员 | `static field` | `kind: field, static: true` | `field` | `static` | +| 顶级/interface class 成员 | `const` | `kind: constant` | `constant` | `instance` | +| 顶级/interface class 成员 | `static const` | `kind: constant, static: true` | `constant` | `static` | +| unit interface direct member | `function` | `kind: function` | `function` | `unit` | +| unit interface direct member | `var` | `kind: variable` | `variable` | `unit` | +| unit interface direct member | `const` | `kind: constant` | `constant` | `unit` | +| unit interface direct member | `class` | `kind: class` | `class` | `unit` | + +表中的 interface class 成员指 unit interface class 的 H4 成员；其允许的声明值和 +派生规则与顶级 class 的 H3 成员完全相同。 + +例如 class function `Widget.Create` 的 owner 是 `Widget`；unit interface class +function `DemoUnit.Document.Save` 的 owner 是 `DemoUnit.Document`。重载共享 +`qualified_name`，由 `signature` 和唯一的 `page#anchor` 区分。 + +lookup 的 `--name` 同时精确匹配简单名称与 `qualified_name`，比较大小写不敏感。 +简单成员名会返回所有 owner 下的同名 API；完全限定名称用于缩小到指定 class/unit。 +`--kw` 还会搜索 kind、binding、visibility、owner 和 qualified_name。 diff --git a/tools/tsl-codegen/examples/example.json b/tools/tsl-codegen/examples/example.json index 22bbf336..9a360562 100644 --- a/tools/tsl-codegen/examples/example.json +++ b/tools/tsl-codegen/examples/example.json @@ -1,53 +1,213 @@ { - "module": "示例 / 数组", - "path": "base/example", - "functions": [ + "module": "OfficeXml / OpenXml", + "path": "officexml/openxml/elements", + "declarations": [ { - "signature": "demoNow()", - "desc": "返回示例值。", - "returns": "integer", - "example": "return demoNow();\n// 输出：1" - }, - { - "signature": "demoFn(src, mode, factor, ...)", - "tags": ["示例", "数组"], - "desc": "按指定模式处理数组并返回结果。", - "params": [ + "kind": "class", + "name": "OpenXmlAttribute", + "desc": "表示 OpenXml 属性", + "tags": ["OpenXml", "XML", "属性"], + "members": [ { - "name": "src", - "type": "array", - "desc": "待处理数组" - }, - { - "name": "mode", - "type": "integer", - "desc": "处理模式，取值见下。", - "values": [ + "kind": "method", + "name": "create", + "visibility": "public", + "binding": "instance", + "signature": "create(_prefix, _local_name)", + "desc": "创建 OpenXml 属性", + "tags": ["OpenXml", "属性", "创建"], + "params": [ { - "value": 0, - "desc": "原样返回" + "name": "_prefix", + "type": "string", + "desc": "命名空间前缀" }, { - "value": 1, - "desc": "去重" + "name": "_local_name", + "type": "string", + "desc": "本地名称" } ] }, { - "name": "factor", - "type": "float", - "optional": true, - "desc": "默认 1.0，结果乘以该系数。" + "kind": "method", + "name": "CreateVirtual", + "visibility": "public", + "binding": "class", + "signature": "CreateVirtual(position, row_index, _story)", + "desc": "创建虚段落", + "params": [ + { + "name": "position", + "type": "integer", + "desc": "段落位置" + }, + { + "name": "row_index", + "type": "integer", + "desc": "行索引" + }, + { + "name": "_story", + "type": "StoryNode", + "desc": "故事节点" + } + ], + "returns": "ParagraphSegment" }, { - "name": "...", - "type": "nil|array", - "optional": true, - "desc": "需要追加处理的其他数组。" + "kind": "property", + "name": "Prefix", + "visibility": "public", + "desc": "命名空间前缀", + "type": "string", + "access": "readwrite" + }, + { + "kind": "property", + "name": "NamespaceUri", + "visibility": "public", + "desc": "命名空间 URI", + "access": "read" + }, + { + "kind": "field", + "name": "value_", + "visibility": "protected", + "desc": "属性值存储", + "type": "any" + }, + { + "kind": "field", + "name": "Count", + "visibility": "public", + "desc": "属性实例数量", + "type": "integer", + "static": true + }, + { + "kind": "constant", + "name": "DefaultPrefix", + "visibility": "public", + "desc": "默认命名空间前缀", + "type": "string", + "value": "xml" + }, + { + "kind": "constant", + "name": "MaximumAttributes", + "visibility": "public", + "desc": "最大属性数量", + "type": "integer", + "value": 256, + "static": true + } + ] + }, + { + "kind": "function", + "name": "ParseOpenXml", + "signature": "ParseOpenXml(xml)", + "desc": "解析 OpenXml 文本", + "tags": ["OpenXml", "解析"], + "params": [ + { + "name": "xml", + "type": "string", + "desc": "OpenXml 文本" } ], - "returns": "array", - "example": "src := array(1, 1, 2);\nreturn demoFn(src, 1, 2.0);\n// 输出：array(2,4)" + "returns": "OpenXmlElement", + "examples": [ + { + "desc": "解析根元素", + "code": "xml := '';\nreturn ParseOpenXml(xml);" + } + ] + }, + { + "kind": "unit", + "name": "OpenXmlRuntime", + "desc": "提供 OpenXml 运行时接口", + "tags": ["OpenXml", "运行时"], + "members": [ + { + "kind": "constant", + "name": "DefaultSize", + "desc": "默认文档容量", + "type": "integer", + "value": 100 + }, + { + "kind": "variable", + "name": "CurrentDocument", + "desc": "当前文档", + "type": "Document" + }, + { + "kind": "function", + "name": "OpenDocument", + "signature": "OpenDocument(path)", + "desc": "打开文档", + "tags": ["OpenXml", "文档", "打开"], + "params": [ + { + "name": "path", + "type": "string", + "desc": "文档路径" + } + ], + "returns": "Document" + }, + { + "kind": "class", + "name": "Document", + "desc": "表示 OpenXml 文档", + "tags": ["OpenXml", "文档"], + "members": [ + { + "kind": "method", + "name": "Save", + "visibility": "public", + "binding": "instance", + "signature": "Save(path)", + "desc": "保存文档", + "params": [ + { + "name": "path", + "type": "string", + "desc": "保存路径" + } + ], + "returns": "boolean" + }, + { + "kind": "method", + "name": "Create", + "visibility": "public", + "binding": "class", + "signature": "Create(name)", + "desc": "创建文档", + "params": [ + { + "name": "name", + "type": "string", + "desc": "文档名称" + } + ], + "returns": "Document" + }, + { + "kind": "property", + "name": "Title", + "visibility": "public", + "desc": "文档标题", + "type": "string", + "access": "readwrite" + } + ] + } + ] } ] } diff --git a/tools/tsl-codegen/examples/example.yaml b/tools/tsl-codegen/examples/example.yaml index f510926c..0c2a6fc5 100644 --- a/tools/tsl-codegen/examples/example.yaml +++ b/tools/tsl-codegen/examples/example.yaml @@ -1,39 +1,160 @@ -module: 示例 / 数组 -path: base/example +module: OfficeXml / OpenXml +path: officexml/openxml/elements +declarations: + - kind: class + name: OpenXmlAttribute + desc: 表示 OpenXml 属性 + tags: [OpenXml, XML, 属性] + members: + - kind: method + name: create + visibility: public + binding: instance + signature: create(_prefix, _local_name) + desc: 创建 OpenXml 属性 + tags: [OpenXml, 属性, 创建] + params: + - name: _prefix + type: string + desc: 命名空间前缀 + - name: _local_name + type: string + desc: 本地名称 -functions: - - signature: demoNow() - desc: 返回示例值。 - returns: integer - example: | - return demoNow(); - // 输出：1 + - kind: method + name: CreateVirtual + visibility: public + binding: class + signature: CreateVirtual(position, row_index, _story) + desc: 创建虚段落 + params: + - name: position + type: integer + desc: 段落位置 + - name: row_index + type: integer + desc: 行索引 + - name: _story + type: StoryNode + desc: 故事节点 + returns: ParagraphSegment - - signature: demoFn(src, mode, factor, ...) - tags: [示例, 数组] - desc: 按指定模式处理数组并返回结果。 - params: - - name: src - type: array - desc: 待处理数组 - - name: mode + - kind: property + name: Prefix + visibility: public + desc: 命名空间前缀 + type: string + access: readwrite + + - kind: property + name: NamespaceUri + visibility: public + desc: 命名空间 URI + access: read + + - kind: field + name: value_ + visibility: protected + desc: 属性值存储 + type: any + + - kind: field + name: Count + visibility: public + desc: 属性实例数量 type: integer - desc: 处理模式，取值见下。 - values: - - value: 0 - desc: 原样返回 - - value: 1 - desc: 去重 - - name: factor - type: float - optional: true - desc: 默认 1.0，结果乘以该系数。 - - name: "..." - type: nil|array - optional: true - desc: 需要追加处理的其他数组。 - returns: array - example: | - src := array(1, 1, 2); - return demoFn(src, 1, 2.0); - // 输出：array(2,4) + static: true + + - kind: constant + name: DefaultPrefix + visibility: public + desc: 默认命名空间前缀 + type: string + value: xml + + - kind: constant + name: MaximumAttributes + visibility: public + desc: 最大属性数量 + type: integer + value: 256 + static: true + + - kind: function + name: ParseOpenXml + signature: ParseOpenXml(xml) + desc: 解析 OpenXml 文本 + tags: [OpenXml, 解析] + params: + - name: xml + type: string + desc: OpenXml 文本 + returns: OpenXmlElement + examples: + - desc: 解析根元素 + code: |- + xml := ''; + return ParseOpenXml(xml); + + - kind: unit + name: OpenXmlRuntime + desc: 提供 OpenXml 运行时接口 + tags: [OpenXml, 运行时] + members: + - kind: constant + name: DefaultSize + desc: 默认文档容量 + type: integer + value: 100 + + - kind: variable + name: CurrentDocument + desc: 当前文档 + type: Document + + - kind: function + name: OpenDocument + signature: OpenDocument(path) + desc: 打开文档 + tags: [OpenXml, 文档, 打开] + params: + - name: path + type: string + desc: 文档路径 + returns: Document + + - kind: class + name: Document + desc: 表示 OpenXml 文档 + tags: [OpenXml, 文档] + members: + - kind: method + name: Save + visibility: public + binding: instance + signature: Save(path) + desc: 保存文档 + params: + - name: path + type: string + desc: 保存路径 + returns: boolean + + - kind: method + name: Create + visibility: public + binding: class + signature: Create(name) + desc: 创建文档 + params: + - name: name + type: string + desc: 文档名称 + returns: Document + + - kind: property + name: Title + visibility: public + desc: 文档标题 + type: string + access: readwrite diff --git a/tools/tsl-codegen/scripts/api_markdown.py b/tools/tsl-codegen/scripts/api_markdown.py new file mode 100644 index 00000000..bf98ff67 --- /dev/null +++ b/tools/tsl-codegen/scripts/api_markdown.py @@ -0,0 +1,359 @@ +"""Shared recognition of TSL codegen API headings and declaration lines.""" + +from bisect import bisect_right +from dataclasses import dataclass +import re + + +TOP_LEVEL_RE = re.compile(r"^##(?!#)\s+`(.+?)`\s*$") +PLAIN_H2_RE = re.compile(r"^##(?!#)\s+") +BARE_API_RE = re.compile(r"^(#{3,5})(?!#)\s+`(.+?)`\s*$") +LEGACY_TYPED_RE = re.compile( + r"^(#{3,5})(?!#)\s+" + r"(class function|static function|static field|static const|" + r"function|property|field|const|var|class)\s+`(.+?)`\s*$" +) +DECLARATION_LINE_RE = re.compile(r"^声明：(.*?)\s*$") +DECLARATION_PREFIX_RE = re.compile(r"^声明[:：]", re.IGNORECASE) +LOOSE_DECLARATION_RE = re.compile(r"^声明[:：]\s*(.*?)\s*$", re.IGNORECASE) +TOP_LEVEL_TYPES = {"function", "class", "unit"} +CLASS_DECLARATIONS = { + "function": ("method", "instance"), + "class function": ("method", "class"), + "property": ("property", "instance"), + "field": ("field", "instance"), + "static field": ("field", "static"), + "const": ("constant", "instance"), + "static const": ("constant", "static"), +} +UNIT_DECLARATIONS = { + "function": ("function", "unit"), + "var": ("variable", "unit"), + "const": ("constant", "unit"), + "class": ("class", "unit"), +} +HEADING_RE = re.compile(r"^(#{1,6})(?!#)\s+") +FENCE_RE = re.compile(r"^\s*(```|~~~)") + + +@dataclass(frozen=True) +class ApiHeading: + level: int + kind: str + binding: str + name: str + signature: str + visible_title: str + root_kind: str + valid: bool = True + error: str = "" + + +@dataclass(frozen=True) +class ApiEntry: + heading: ApiHeading + start: int + end: int + + +def simple_name(signature): + return signature.split("(", 1)[0].strip() + + +def fence_flags(lines): + flags = [] + in_fence = False + marker = "" + for line in lines: + stripped = line.lstrip() + if not in_fence and FENCE_RE.match(line): + marker = stripped[:3] + flags.append(True) + in_fence = True + continue + flags.append(in_fence) + if in_fence and stripped.startswith(marker): + in_fence = False + marker = "" + return flags + + +def heading_body_end(lines, start, flags=None): + if flags is None: + flags = fence_flags(lines) + for index in range(start + 1, len(lines)): + if flags[index]: + continue + if HEADING_RE.match(lines[index]): + return index + return len(lines) + + +def top_level_header_end(lines, start, flags=None): + return heading_body_end(lines, start, flags) + + +def _declaration(lines, start, end, flags): + content = [] + declaration_lines = [] + for index in range(start + 1, end): + if flags[index]: + continue + stripped = lines[index].strip() + if not stripped: + continue + content.append((index, stripped)) + if DECLARATION_PREFIX_RE.match(stripped): + declaration_lines.append((index, stripped)) + + if not content: + return "", "API 标题后的第一条非空正文必须是 `声明：...`" + first_index, first = content[0] + match = DECLARATION_LINE_RE.fullmatch(first) + if not match: + loose = LOOSE_DECLARATION_RE.fullmatch(first) + if loose and loose.group(1).strip(): + expected = f"声明：{loose.group(1).strip().casefold()}" + return "", f"声明行必须精确写为 `{expected}`" + return ( + "", + "API 标题后的第一条非空正文必须是规范的 `声明：...` 行", + ) + value = match.group(1).strip().casefold() + expected = f"声明：{value}" + if first != expected: + return "", f"声明行必须精确写为 `{expected}`" + if len(declaration_lines) != 1 or declaration_lines[0][0] != first_index: + return "", "每个 API 必须且只能包含一行声明" + return value, "" + + +def top_level_type(lines, start, end): + return _declaration(lines, start, end, fence_flags(lines)) + + +def invalid_heading(level, name, signature, visible_title, root_kind, message): + return ApiHeading( + level, + "invalid", + "", + name, + signature, + visible_title, + root_kind, + False, + message, + ) + + +def _valid_heading(level, kind, binding, signature, root_kind): + return ApiHeading( + level, + kind, + binding, + simple_name(signature), + signature, + signature, + root_kind, + ) + + +def top_level_heading(signature, declaration, error): + if error: + return invalid_heading( + 2, + simple_name(signature), + signature, + signature, + "", + error, + ) + if declaration not in TOP_LEVEL_TYPES: + return invalid_heading( + 2, + simple_name(signature), + signature, + signature, + "", + f"顶级 API 声明必须是 function、class 或 unit，实为 {declaration}", + ) + return _valid_heading(2, declaration, "", signature, declaration) + + +def member_heading( + level, + declaration, + signature, + root_kind, + unit_class_open, + error, +): + name = simple_name(signature) + if error: + return invalid_heading( + level, name, signature, signature, root_kind, error + ) + if declaration == "static function": + return invalid_heading( + level, + name, + signature, + signature, + root_kind, + "不存在 static function，请使用 class function", + ) + + if root_kind == "class": + if level != 3 or declaration not in CLASS_DECLARATIONS: + return invalid_heading( + level, + name, + signature, + signature, + root_kind, + "class API 成员必须使用约定的 H3 声明", + ) + kind, binding = CLASS_DECLARATIONS[declaration] + return _valid_heading(level, kind, binding, signature, root_kind) + + if root_kind == "unit": + if level == 3: + if declaration not in UNIT_DECLARATIONS: + return invalid_heading( + level, + name, + signature, + signature, + root_kind, + "unit interface 成员必须使用约定的 H3 声明", + ) + kind, binding = UNIT_DECLARATIONS[declaration] + return _valid_heading(level, kind, binding, signature, root_kind) + if ( + level == 4 + and unit_class_open + and declaration in CLASS_DECLARATIONS + ): + kind, binding = CLASS_DECLARATIONS[declaration] + return _valid_heading(level, kind, binding, signature, root_kind) + return invalid_heading( + level, + name, + signature, + signature, + root_kind, + "unit class 成员必须位于所属 class 的 H4", + ) + + return invalid_heading( + level, + name, + signature, + signature, + root_kind, + "function 顶级 API 不能包含 API 成员", + ) + + +def collect_headings_and_boundaries(lines): + flags = fence_flags(lines) + headings = [] + h2_boundaries = [] + root_kind = "" + unit_class_open = False + + for index, line in enumerate(lines): + if flags[index]: + continue + + if PLAIN_H2_RE.match(line): + h2_boundaries.append(index) + root_kind = "" + unit_class_open = False + top_level = TOP_LEVEL_RE.match(line) + if not top_level: + continue + signature = top_level.group(1) + end = heading_body_end(lines, index, flags) + declaration, error = _declaration( + lines, index, end, flags + ) + heading = top_level_heading(signature, declaration, error) + if heading.valid: + root_kind = heading.kind + headings.append((index, heading)) + continue + + if not root_kind: + continue + + heading_match = HEADING_RE.match(line) + if root_kind == "unit" and heading_match: + if len(heading_match.group(1)) == 3: + unit_class_open = False + + legacy = LEGACY_TYPED_RE.match(line) + if legacy: + level = len(legacy.group(1)) + signature = legacy.group(3) + headings.append( + ( + index, + invalid_heading( + level, + simple_name(signature), + signature, + line.lstrip("# "), + root_kind, + "API 标题只写名称或调用签名，声明种类写在声明行", + ), + ) + ) + continue + + bare = BARE_API_RE.match(line) + if not bare: + continue + level = len(bare.group(1)) + signature = bare.group(2) + end = heading_body_end(lines, index, flags) + declaration, error = _declaration(lines, index, end, flags) + heading = member_heading( + level, + declaration, + signature, + root_kind, + unit_class_open, + error, + ) + if root_kind == "unit" and level == 3: + unit_class_open = heading.valid and heading.kind == "class" + headings.append((index, heading)) + + return headings, h2_boundaries + + +def collect_headings(lines): + headings, _ = collect_headings_and_boundaries(lines) + return headings + + +def iter_api_entries(lines): + headings, h2_boundaries = collect_headings_and_boundaries(lines) + for index, (start, heading) in enumerate(headings): + next_heading = ( + headings[index + 1][0] + if index + 1 < len(headings) + else len(lines) + ) + boundary_index = bisect_right(h2_boundaries, start) + next_h2 = ( + h2_boundaries[boundary_index] + if boundary_index < len(h2_boundaries) + else len(lines) + ) + yield ApiEntry(heading, start, min(next_heading, next_h2)) + + +def slug(text): + return re.sub(r"[^a-z0-9_]", "", text.casefold()) diff --git a/tools/tsl-codegen/scripts/build_index.py b/tools/tsl-codegen/scripts/build_index.py index d10660e0..73ef3b36 100644 --- a/tools/tsl-codegen/scripts/build_index.py +++ b/tools/tsl-codegen/scripts/build_index.py @@ -1,23 +1,28 @@ #!/usr/bin/env python3 -"""Rebuild the bundled TSL API function index from the codegen markdown tree. +"""Rebuild the bundled TSL API index from the codegen markdown tree. -The markdown tree is the source of truth: each `## `sig`` / `### `sig`` heading -is one function entry. The TSV is a derived product; regenerate it whenever the -leaf Markdown changes rather than editing it by hand. +The markdown tree is the source of truth. The TSV is a derived product; +regenerate it whenever the leaf Markdown changes rather than editing it by +hand. Columns (tab-separated, LF line endings, UTF-8): name scope module signature page anchor tags summary + kind binding visibility owner qualified_name - name: signature text up to the first '(' - scope: first path segment under the codegen root (for example project) - module: second path segment for nested pages, else the flat file stem - signature: verbatim from the heading, backticks stripped - page: POSIX path relative to the codegen root -- anchor: GitHub-style slug of the name (lowercased, chars outside - [a-z0-9_] removed); per-page duplicate slugs get -1/-2 suffixes - in document order, matching the rendered heading anchors. +- anchor: top-level declarations use the historic name slug; typed members + slug the complete visible API title. Per-page duplicate slugs get + -1/-2 suffixes in document order. - tags: space-separated keywords from `` -- summary: first prose line under the entry heading, empty for table, - heading, or standalone return-type lines +- summary: first prose line under the entry heading +- kind: function/class/method/property/field/constant/unit/variable +- binding: instance/class/static/unit, or empty when not applicable +- visibility: public/protected for class members; public for unit interface +- owner: dot-separated containing API path, excluding the entry name +- qualified_name: dot-separated stable API identity Usage (run from repo root; --skill-dir is required): SKILL=skills/tsl-api-reference @@ -31,9 +36,22 @@ import sys from pathlib import Path import re -ENTRY_RE = re.compile(r"^#{2,3}(?!#)\s+`(.+?)`\s*$") +SCRIPT_DIR = Path(__file__).resolve().parent +if str(SCRIPT_DIR) not in sys.path: + sys.path.insert(0, str(SCRIPT_DIR)) + +from api_markdown import DECLARATION_LINE_RE, iter_api_entries, slug + + RETURN_RE = re.compile(r"^返回[:：]") +DECLARATION_TYPE_RE = re.compile( + r"^类型[:：]\s*(function|class|unit)\s*$", re.IGNORECASE +) TAGS_RE = re.compile(r"^$") +VISIBILITY_RE = re.compile( + r"^可见性[:：]\s*`?(public|protected|private)`?\s*$", + re.IGNORECASE, +) HEADER = [ "name", "scope", @@ -43,29 +61,56 @@ HEADER = [ "anchor", "tags", "summary", + "kind", + "binding", + "visibility", + "owner", + "qualified_name", ] -def slug(name): - """GitHub-style anchor slug: lowercase, keep [a-z0-9_], drop the rest.""" - return re.sub(r"[^a-z0-9_]", "", name.lower()) - - -def extract_metadata(lines, heading_idx): - """Return tags and the first prose line under a function entry heading.""" +def extract_metadata(lines, heading_idx, end_idx=None): + """Return tags and the first prose line under one API heading.""" tags = "" - for line in lines[heading_idx + 1:]: + summary = "" + summary_open = True + for line in lines[heading_idx + 1:end_idx]: text = line.strip() if not text: continue + if DECLARATION_LINE_RE.fullmatch(text): + continue tag_match = TAGS_RE.match(text) if tag_match: tags = " ".join(tag_match.group(1).split()).replace("\t", " ") continue - if text.startswith("|") or text.startswith("#") or RETURN_RE.match(text): - return tags, "" - return tags, text.replace("\t", " ") - return tags, "" + if summary: + continue + if ( + text.startswith("|") + or text.startswith("#") + or RETURN_RE.match(text) + or DECLARATION_TYPE_RE.match(text) + ): + summary_open = False + continue + if summary_open: + summary = text.replace("\t", " ") + return tags, summary + + +def extract_visibility(lines, start, end): + for line in lines[start + 1:end]: + match = VISIBILITY_RE.match(line.strip()) + if match: + return match.group(1).casefold() + return "" + + +def next_anchor(base, seen): + count = seen.get(base, 0) + seen[base] = count + 1 + return base if count == 0 else f"{base}-{count}" def parse_page(codegen_root, md): @@ -75,18 +120,69 @@ def parse_page(codegen_root, md): seen = {} rows = [] lines = md.read_text(encoding="utf-8").splitlines() - for idx, line in enumerate(lines): - m = ENTRY_RE.match(line) - if not m: + entries = list(iter_api_entries(lines)) + root_kind = "" + root_name = "" + unit_class_owner = "" + + for entry in entries: + heading = entry.heading + if not heading.valid: continue - sig = m.group(1) - name = sig.split("(", 1)[0] - base = slug(name) - n = seen.get(base, 0) - seen[base] = n + 1 - anchor = base if n == 0 else f"{base}-{n}" - tags, summary = extract_metadata(lines, idx) - rows.append([name, scope, module, sig, page, anchor, tags, summary]) + + if heading.level == 2: + root_kind = heading.kind + root_name = heading.name + unit_class_owner = "" + owner = "" + anchor_base = slug(heading.name) + elif root_kind == "class": + owner = root_name + anchor_base = slug(heading.visible_title) + elif root_kind == "unit" and heading.level == 3: + anchor_base = slug(heading.visible_title) + owner = root_name + unit_class_owner = ( + f"{root_name}.{heading.name}" + if heading.kind == "class" + else "" + ) + elif root_kind == "unit" and heading.level == 4: + owner = unit_class_owner + anchor_base = slug(heading.visible_title) + else: + continue + + qualified_name = ( + f"{owner}.{heading.name}" if owner else heading.name + ) + if root_kind == "unit" and heading.level == 3: + visibility = "public" + elif ( + root_kind == "class" and heading.level == 3 + ) or (root_kind == "unit" and heading.level == 4): + visibility = extract_visibility(lines, entry.start, entry.end) + else: + visibility = "" + + tags, summary = extract_metadata(lines, entry.start, entry.end) + rows.append( + [ + heading.name, + scope, + module, + heading.signature, + page, + next_anchor(anchor_base, seen), + tags, + summary, + heading.kind, + heading.binding, + visibility, + owner, + qualified_name, + ] + ) return rows @@ -120,7 +216,16 @@ def read_tsv(tsv_path): def main(argv=None): if hasattr(sys.stdout, "reconfigure"): sys.stdout.reconfigure(encoding="utf-8") - parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser = argparse.ArgumentParser( + description=__doc__.splitlines()[0], + add_help=False, + allow_abbrev=False, + ) + parser.add_argument( + "--help", + action="help", + help="show this help message and exit (no -h short option)", + ) parser.add_argument( "--skill-dir", required=True, diff --git a/tools/tsl-codegen/scripts/convert_tsf.py b/tools/tsl-codegen/scripts/convert_tsf.py new file mode 100644 index 00000000..4df518f8 --- /dev/null +++ b/tools/tsl-codegen/scripts/convert_tsf.py @@ -0,0 +1,1643 @@ +#!/usr/bin/env python3 +"""Convert documented TSF function, class, and unit files to declarations.""" + +import argparse +import json +import os +import re +import sys +import tempfile +from dataclasses import dataclass +from pathlib import Path + +STRUCTURE_KEYWORDS = {"function", "procedure", "type", "unit"} +PARAM_MODIFIERS = {"const", "var"} +DOCUMENT_LINE_RE = re.compile(r"^\s*///(.*)$") +DIRECTIVE_RE = re.compile(r"^@([a-z]+):(.*)$") + + +class ConversionError(Exception): + def __init__(self, path, line, message): + self.path = Path(path) + self.line = line + self.message = message + super().__init__(str(self)) + + def __str__(self): + location = str(self.path) + if self.line is not None: + location += f":{self.line}" + return f"{location}: {self.message}" + + +@dataclass(frozen=True) +class Token: + value: str + start: int + end: int + line: int + kind: str + + +@dataclass(frozen=True) +class Parameter: + name: str + type: str + optional: bool + line: int + + +@dataclass(frozen=True) +class DocumentLine: + text: str + line: int + + +class ChineseHelpFormatter(argparse.HelpFormatter): + def add_usage(self, usage, actions, groups, prefix=None): + super().add_usage(usage, actions, groups, prefix or "用法：") + + def format_help(self): + return ( + super() + .format_help() + .replace("位置参数:\n", "位置参数：\n") + .replace("选项:\n", "选项：\n") + ) + + +def fail(path, line, message): + raise ConversionError(path, line, message) + + +def is_identifier_start(char): + return char == "_" or char.isalpha() + + +def is_identifier_part(char): + return char == "_" or char.isalnum() + + +def tokenize(source): + tokens = [] + index = 0 + line = 1 + length = len(source) + + while index < length: + char = source[index] + if char.isspace(): + if char == "\n": + line += 1 + index += 1 + continue + + if source.startswith("//", index): + newline = source.find("\n", index + 2) + if newline == -1: + break + index = newline + continue + + if source.startswith("(*", index): + end = source.find("*)", index + 2) + if end == -1: + end = length - 2 + line += source.count("\n", index, end + 2) + index = end + 2 + continue + + if char == "{": + end = source.find("}", index + 1) + if end == -1: + end = length - 1 + line += source.count("\n", index, end + 1) + index = end + 1 + continue + + if char in {"'", '"'}: + quote = char + start = index + token_line = line + index += 1 + while index < length: + if source[index] == "\n": + line += 1 + if source[index] == quote: + if index + 1 < length and source[index + 1] == quote: + index += 2 + continue + index += 1 + break + if source[index] == "\\" and index + 1 < length: + index += 2 + continue + index += 1 + tokens.append( + Token(source[start:index], start, index, token_line, "string") + ) + continue + + if is_identifier_start(char): + start = index + token_line = line + index += 1 + while index < length and is_identifier_part(source[index]): + index += 1 + tokens.append( + Token(source[start:index], start, index, token_line, "identifier") + ) + continue + + start = index + if source.startswith("...", index): + index += 3 + else: + index += 1 + tokens.append(Token(source[start:index], start, index, line, "symbol")) + + return tokens + + +def first_structure_token(tokens, path): + for token in tokens: + if token.kind != "identifier": + continue + keyword = token.value.casefold() + if keyword not in STRUCTURE_KEYWORDS: + continue + return token + fail(path, 1, "没有找到受支持的顶层声明") + + +def matching_parenthesis(tokens, open_index, path): + depth = 0 + for index in range(open_index, len(tokens)): + value = tokens[index].value + if value == "(": + depth += 1 + elif value == ")": + depth -= 1 + if depth == 0: + return index + fail(path, tokens[open_index].line, "函数参数列表缺少右括号") + + +def declaration_end(tokens, start_index, path): + depth = 0 + for index in range(start_index, len(tokens)): + value = tokens[index].value + if value in {"(", "["}: + depth += 1 + elif value in { + ")", + "]", + }: + depth -= 1 + elif value == ";" and depth == 0: + return index + fail(path, tokens[start_index].line, "function 声明缺少分号") + + +def split_parameter_ranges(tokens, start, end): + ranges = [] + range_start = start + depth = 0 + for token in tokens: + if token.start < start or token.end > end: + continue + if token.value in {"(", "["}: + depth += 1 + elif token.value in { + ")", + "]", + }: + depth -= 1 + elif token.value in {",", ";"} and depth == 0: + ranges.append((range_start, token.start)) + range_start = token.end + ranges.append((range_start, end)) + return ranges + + +def top_level_separator(tokens, start, end, value): + depth = 0 + for token in tokens: + if token.start < start or token.end > end: + continue + if token.value in {"(", "["}: + depth += 1 + elif token.value in { + ")", + "]", + }: + depth -= 1 + elif token.value == value and depth == 0: + return token + return None + + +def tokens_in_range(tokens, start, end): + return [token for token in tokens if token.start >= start and token.end <= end] + + +def parse_parameter(source, tokens, start, end, path, fallback_line): + if not source[start:end].strip(): + fail(path, fallback_line, "参数声明不能为空") + + equals = top_level_separator(tokens, start, end, "=") + declaration_end_offset = equals.start if equals else end + colon = top_level_separator(tokens, start, declaration_end_offset, ":") + name_end = colon.start if colon else declaration_end_offset + name_tokens = tokens_in_range(tokens, start, name_end) + if name_tokens and name_tokens[0].value.casefold() in PARAM_MODIFIERS: + name_tokens = name_tokens[1:] + if len(name_tokens) != 1: + fail(path, fallback_line, "参数声明必须包含一个参数名") + name_token = name_tokens[0] + if name_token.kind != "identifier" and name_token.value != "...": + fail(path, name_token.line, "参数名不是有效标识符") + + param_type = "" + if colon: + param_type = source[colon.end : declaration_end_offset].strip() + return Parameter(name_token.value, param_type, equals is not None, name_token.line) + + +def parse_function_signature(source, tokens, function_index, path): + function_token = tokens[function_index] + if function_index + 1 >= len(tokens): + fail(path, function_token.line, "function 声明缺少函数名") + name_token = tokens[function_index + 1] + if name_token.kind != "identifier": + fail(path, name_token.line, "function 声明缺少有效函数名") + + end_index = declaration_end(tokens, function_index + 2, path) + end_token = tokens[end_index] + between = tokens[function_index + 2 : end_index] + open_index = next( + ( + function_index + 2 + index + for index, token in enumerate(between) + if token.value == "(" + ), + None, + ) + + parameters = [] + return_search_start = name_token.end + if open_index is not None: + close_index = matching_parenthesis(tokens, open_index, path) + if close_index >= end_index: + fail(path, tokens[open_index].line, "函数参数列表没有在声明分号前结束") + open_token = tokens[open_index] + close_token = tokens[close_index] + if source[open_token.end : close_token.start].strip(): + for start, end in split_parameter_ranges( + tokens, open_token.end, close_token.start + ): + parameters.append( + parse_parameter( + source, + tokens, + start, + end, + path, + open_token.line, + ) + ) + return_search_start = close_token.end + + seen_names = set() + for parameter in parameters: + folded = parameter.name.casefold() + if folded in seen_names: + fail(path, parameter.line, f"参数名重复：{parameter.name}") + seen_names.add(folded) + + return_colon = top_level_separator( + tokens, return_search_start, end_token.start, ":" + ) + return_type = "" + if return_colon: + return_type = source[return_colon.end : end_token.start].strip() + + return name_token.value, parameters, return_type, end_index + + +def parse_declaration(source, tokens, function_token, path): + function_index = tokens.index(function_token) + name, parameters, return_type, end_index = parse_function_signature( + source, tokens, function_index, path + ) + end_token = tokens[end_index] + + begin_token = next( + ( + token + for token in tokens[end_index + 1 :] + if token.kind == "identifier" and token.value.casefold() == "begin" + ), + None, + ) + if begin_token is None: + fail(path, end_token.line, "function 缺少 begin") + return name, parameters, return_type, begin_token + + +def extract_document_lines(source, begin_token, path): + physical_lines = source.splitlines() + begin_index = begin_token.line - 1 + line_end = source.find("\n", begin_token.end) + if line_end == -1: + line_end = len(source) + if source[begin_token.end : line_end].strip(): + fail(path, begin_token.line, "begin 后第一段内容必须是 /// 文档块") + + first_content = None + for index in range(begin_index + 1, len(physical_lines)): + if physical_lines[index].strip(): + first_content = index + break + if first_content is None: + fail(path, begin_token.line, "function 缺少 /// 文档块") + if not DOCUMENT_LINE_RE.match(physical_lines[first_content]): + fail( + path, + first_content + 1, + "begin 后第一段内容必须是 /// 文档块", + ) + + document = [] + for index in range(first_content, len(physical_lines)): + match = DOCUMENT_LINE_RE.match(physical_lines[index]) + if not match: + break + text = match.group(1) + if text.startswith(" "): + text = text[1:] + document.append(DocumentLine(text, index + 1)) + return document + + +def trim_blank_lines(lines): + start = 0 + end = len(lines) + while start < end and not lines[start]: + start += 1 + while end > start and not lines[end - 1]: + end -= 1 + return lines[start:end] + + +def parse_directive(line, path): + match = DIRECTIVE_RE.match(line.text) + if not match: + fail(path, line.line, "未知或格式错误的文档指令") + return match.group(1), match.group(2).strip() + + +def collect_indented(lines, start, path, label): + collected = [] + index = start + while index < len(lines) and not lines[index].text.startswith("@"): + item = lines[index] + if item.text: + if not item.text.startswith(" "): + fail(path, item.line, f"{label}必须在 /// 后额外缩进两个空格") + collected.append(item.text[2:]) + else: + collected.append("") + index += 1 + return trim_blank_lines(collected), index + + +def parse_param_directive(argument, line, path): + parts = argument.split(None, 1) + if len(parts) != 2 or not parts[1].strip(): + fail(path, line, "@param: 必须包含参数名和说明") + name = parts[0] + remainder = parts[1].strip() + documented_type = "" + if remainder.startswith("{"): + close = remainder.find("}", 1) + if close == -1: + fail(path, line, "@param: 参数类型缺少右花括号") + documented_type = remainder[1:close].strip() + if not documented_type: + fail(path, line, "@param: 参数类型不能为空") + remainder = remainder[close + 1 :].strip() + if not remainder: + fail(path, line, "@param: 参数说明不能为空") + return name, documented_type, remainder + + +def parse_enum_item(text, path, line): + decoder = json.JSONDecoder() + stripped = text.lstrip() + try: + value, end = decoder.raw_decode(stripped) + except json.JSONDecodeError: + fail(path, line, "枚举值必须使用 json 标量写法") + if isinstance(value, (list, dict)) or value is None: + fail(path, line, "枚举值只允许数字、字符串或布尔值") + remainder = stripped[end:].lstrip() + if not remainder.startswith(":") or not remainder[1:].strip(): + fail(path, line, "枚举项必须写成“值: 说明”") + return value, remainder[1:].strip() + + +def enum_value_key(value): + return type(value).__name__, json.dumps(value, ensure_ascii=False, sort_keys=True) + + +def normalized_type_tokens(value): + normalized = [] + index = 0 + while index < len(value): + char = value[index] + if char.isspace(): + index += 1 + continue + if is_identifier_start(char): + end = index + 1 + while end < len(value) and is_identifier_part(value[end]): + end += 1 + normalized.append(("identifier", value[index:end].casefold())) + index = end + continue + normalized.append(("symbol", char)) + index += 1 + return normalized + + +def parameter_index(parameters, name): + folded = name.casefold() + for index, parameter in enumerate(parameters): + if parameter.name.casefold() == folded: + return index + return None + + +def parse_document(document, parameters, declared_return, path): + description_lines = [] + index = 0 + while index < len(document) and not document[index].text.startswith("@"): + description_lines.append(document[index].text) + index += 1 + description_lines = trim_blank_lines(description_lines) + if not any(line.strip() for line in description_lines): + fail(path, document[0].line, "函数描述不能为空") + + tags = None + param_docs = {} + param_types = {} + param_values = {} + documented_param_indexes = [] + documented_return = None + documented_return_line = None + examples = [] + phase = "tags" + last_param = None + + while index < len(document): + line = document[index] + directive, argument = parse_directive(line, path) + + if directive == "tags": + if phase != "tags" or tags is not None: + fail(path, line.line, "@tags: 重复或顺序错误") + tags = argument.split() + if not tags: + fail(path, line.line, "@tags: 至少需要一个标签") + index += 1 + continue + + if directive == "param": + if phase == "examples": + fail(path, line.line, "示例组之后不能再写函数级指令") + if phase == "returns": + fail(path, line.line, "@param: 必须写在 @returns: 之前") + phase = "params" + name, documented_type, desc = parse_param_directive( + argument, line.line, path + ) + param_idx = parameter_index(parameters, name) + if param_idx is None: + fail(path, line.line, f"@param: 引用了不存在的参数 {name}") + if documented_param_indexes and param_idx <= documented_param_indexes[-1]: + fail(path, line.line, "@param: 必须按函数声明中的参数顺序书写") + documented_param_indexes.append(param_idx) + canonical_name = parameters[param_idx].name + key = canonical_name.casefold() + declared_type = parameters[param_idx].type + if ( + declared_type + and documented_type + and normalized_type_tokens(declared_type) + != normalized_type_tokens(documented_type) + ): + fail( + path, + line.line, + f"参数 {canonical_name} 的类型与函数声明不一致：" + f"注释为 {documented_type}，声明为 {declared_type}", + ) + param_docs[key] = desc + if documented_type: + param_types[key] = documented_type + last_param = canonical_name + index += 1 + continue + + if directive == "values": + if phase == "examples": + fail(path, line.line, "示例组之后不能再写函数级指令") + if phase != "params" or last_param is None: + fail(path, line.line, "@values: 必须紧跟对应的 @param:") + if not argument or len(argument.split()) != 1: + fail(path, line.line, "@values: 只能包含一个参数名") + param_idx = parameter_index(parameters, argument) + if param_idx is None: + fail(path, line.line, f"@values: 引用了不存在的参数 {argument}") + canonical_name = parameters[param_idx].name + if canonical_name.casefold() != last_param.casefold(): + fail(path, line.line, "@values: 必须紧跟对应的 @param:") + key = canonical_name.casefold() + if key in param_values: + fail(path, line.line, f"参数 {canonical_name} 的 @values: 重复") + raw_items, index = collect_indented(document, index + 1, path, "枚举项") + if not any(item.strip() for item in raw_items): + fail(path, line.line, "@values: 必须包含至少一个枚举项") + values = [] + seen = set() + item_line_index = line.line + 1 + for raw_item in raw_items: + if not raw_item.strip(): + item_line_index += 1 + continue + value, desc = parse_enum_item(raw_item, path, item_line_index) + value_key = enum_value_key(value) + if value_key in seen: + fail(path, item_line_index, f"枚举值重复：{value!r}") + seen.add(value_key) + values.append({"value": value, "desc": desc}) + item_line_index += 1 + param_values[key] = values + last_param = None + continue + + if directive == "returns": + if phase == "examples": + fail(path, line.line, "示例组之后不能再写函数级指令") + if documented_return is not None: + fail(path, line.line, "@returns: 不能重复") + if not argument: + fail(path, line.line, "@returns: 类型不能为空") + phase = "returns" + documented_return = argument + documented_return_line = line.line + last_param = None + index += 1 + continue + + if directive == "example": + if not argument: + fail(path, line.line, "@example: 示例说明不能为空") + phase = "examples" + last_param = None + code, next_index = collect_indented(document, index + 1, path, "示例代码") + if not any(code_line.strip() for code_line in code): + fail(path, line.line, "@example: 必须包含示例代码") + if any(code_line.lstrip().startswith("// 输出：") for code_line in code): + fail(path, line.line, "示例代码不能包含 // 输出：，请改用 @output:") + example = {"desc": argument, "code": "\n".join(code)} + index = next_index + if index < len(document): + next_directive, next_argument = parse_directive(document[index], path) + if next_directive == "output": + if next_argument: + fail(path, document[index].line, "@output: 后不能写行内内容") + output, index = collect_indented( + document, index + 1, path, "示例输出" + ) + if not any(output_line.strip() for output_line in output): + fail(path, document[index - 1].line, "@output: 不能为空") + if any( + output_line.lstrip().startswith("//") for output_line in output + ): + fail( + path, + document[index - 1].line, + "@output: 中不写 // 注释标记", + ) + example["output"] = "\n".join(output) + examples.append(example) + continue + + if directive == "output": + fail(path, line.line, "@output: 必须位于 @example: 的代码之后") + fail(path, line.line, f"未知文档指令：@{directive}:") + + if declared_return and documented_return: + if normalized_type_tokens(declared_return) != normalized_type_tokens( + documented_return + ): + fail( + path, + documented_return_line, + "@returns: 返回类型与函数声明不一致：" + f"注释为 {documented_return}，声明为 {declared_return}", + ) + return_type = declared_return or documented_return or "" + + function = {"desc": "\n".join(description_lines)} + if tags: + function["tags"] = tags + if parameters: + converted_params = [] + for parameter in parameters: + key = parameter.name.casefold() + converted = { + "name": parameter.name, + "type": parameter.type or param_types.get(key, ""), + } + if parameter.optional: + converted["optional"] = True + converted["desc"] = param_docs.get(key, "") + if key in param_values: + converted["values"] = param_values[key] + converted_params.append(converted) + function["params"] = converted_params + function["returns"] = return_type + if examples: + function["examples"] = examples + return function + + +def keyword_at(tokens, index, value): + return ( + index < len(tokens) + and tokens[index].kind == "identifier" + and tokens[index].value.casefold() == value + ) + + +def document_before(source, declaration_line, lower_bound_line=1): + """Return the /// block immediately before a declaration, allowing blanks.""" + lines = source.splitlines() + index = declaration_line - 2 + while index >= lower_bound_line - 1 and not lines[index].strip(): + index -= 1 + if index < lower_bound_line - 1 or not DOCUMENT_LINE_RE.match(lines[index]): + return [] + end = index + while index >= lower_bound_line - 1 and DOCUMENT_LINE_RE.match(lines[index]): + index -= 1 + document = [] + for line_index in range(index + 1, end + 1): + match = DOCUMENT_LINE_RE.match(lines[line_index]) + text = match.group(1) + if text.startswith(" "): + text = text[1:] + document.append(DocumentLine(text, line_index + 1)) + return document + + +def trailing_line_comment(source, declaration_end_token): + line_end = source.find("\n", declaration_end_token.end) + if line_end == -1: + line_end = len(source) + remainder = source[declaration_end_token.end : line_end].lstrip() + if not remainder.startswith("//") or remainder.startswith("///"): + return "" + return remainder[2:].strip() + + +def parse_simple_document(document, path, label): + if not document: + return {"desc": ""} + description = [] + index = 0 + while index < len(document) and not document[index].text.startswith("@"): + description.append(document[index].text) + index += 1 + description = trim_blank_lines(description) + result = {"desc": "\n".join(description)} + if index < len(document): + directive, argument = parse_directive(document[index], path) + if directive != "tags" or not argument: + fail(path, document[index].line, f"{label}只支持 @tags: 指令") + result["tags"] = argument.split() + index += 1 + if index != len(document): + fail(path, document[index].line, f"{label}文档包含不支持的指令") + return result + + +def converted_parameters(parameters, descriptions=None, values=None): + descriptions = descriptions or {} + values = values or {} + result = [] + for parameter in parameters: + key = parameter.name.casefold() + item = { + "name": parameter.name, + "type": parameter.type, + } + if parameter.optional: + item["optional"] = True + item["desc"] = descriptions.get(key, "") + if key in values: + item["values"] = values[key] + result.append(item) + return result + + +def skip_begin_block(tokens, begin_index, path): + depth = 0 + for index in range(begin_index, len(tokens)): + if keyword_at(tokens, index, "begin"): + depth += 1 + elif keyword_at(tokens, index, "end"): + depth -= 1 + if depth == 0: + if index + 1 < len(tokens) and tokens[index + 1].value == ";": + return index + 2 + return index + 1 + fail(path, tokens[begin_index].line, "内联方法缺少 end") + + +def parse_method_member(source, tokens, function_index, path, visibility, binding): + name, parameters, declared_return, end_index = parse_function_signature( + source, tokens, function_index, path + ) + document = ( + document_before(source, tokens[function_index].line) + if visibility != "private" + else [] + ) + if document: + details = parse_document(document, parameters, declared_return, path) + else: + details = {"desc": ""} + if parameters: + details["params"] = converted_parameters(parameters) + if declared_return: + details["returns"] = declared_return + if not details.get("returns"): + details.pop("returns", None) + + modifiers = [] + next_index = end_index + 1 + declaration_end_token = tokens[end_index] + while next_index + 1 < len(tokens): + modifier = tokens[next_index].value.casefold() + if modifier not in {"overload", "virtual", "override"}: + break + if tokens[next_index + 1].value != ";": + break + if modifier in modifiers: + fail(path, tokens[next_index].line, f"方法修饰符重复：{modifier}") + modifiers.append(modifier) + declaration_end_token = tokens[next_index + 1] + next_index += 2 + if visibility != "private" and not details["desc"]: + details["desc"] = trailing_line_comment(source, declaration_end_token) + if keyword_at(tokens, next_index, "begin"): + next_index = skip_begin_block(tokens, next_index, path) + + names = ", ".join(parameter.name for parameter in parameters) + member = { + "kind": "method", + "name": name, + "visibility": visibility, + "binding": binding, + "signature": f"{name}({names})", + **details, + } + if modifiers: + member["modifiers"] = modifiers + return member, next_index + + +def parse_property_member(source, tokens, property_index, path, visibility): + if property_index + 1 >= len(tokens): + fail(path, tokens[property_index].line, "property 声明缺少名称") + name_token = tokens[property_index + 1] + if name_token.kind != "identifier": + fail(path, name_token.line, "property 声明缺少有效名称") + end_index = declaration_end(tokens, property_index + 2, path) + end_token = tokens[end_index] + cursor = property_index + 2 + parameters = [] + type_start = name_token.end + if cursor < end_index and tokens[cursor].value == "(": + close_index = matching_parenthesis(tokens, cursor, path) + if close_index >= end_index: + fail(path, tokens[cursor].line, "property 参数列表没有在分号前结束") + if source[tokens[cursor].end : tokens[close_index].start].strip(): + for start, end in split_parameter_ranges( + tokens, tokens[cursor].end, tokens[close_index].start + ): + parameters.append( + parse_parameter( + source, tokens, start, end, path, tokens[cursor].line + ) + ) + cursor = close_index + 1 + type_start = tokens[close_index].end + + read_indexes = [ + index for index in range(cursor, end_index) if keyword_at(tokens, index, "read") + ] + write_indexes = [ + index + for index in range(cursor, end_index) + if keyword_at(tokens, index, "write") + ] + if not read_indexes and not write_indexes: + fail(path, name_token.line, "property 必须包含 read 或 write") + accessor_index = min(read_indexes + write_indexes) + colon_index = next( + ( + index + for index in range(cursor, accessor_index) + if tokens[index].value == ":" + ), + None, + ) + property_type = "" + if colon_index is not None: + property_type = source[ + tokens[colon_index].end : tokens[accessor_index].start + ].strip() + + document = ( + document_before(source, tokens[property_index].line) + if visibility != "private" + else [] + ) + if document: + details = parse_document(document, parameters, "", path) + if details.get("returns"): + fail(path, document[0].line, "property 文档不支持 @returns:") + if details.get("examples"): + fail(path, document[0].line, "property 文档不支持示例") + details.pop("returns", None) + else: + details = {"desc": ""} + if parameters: + details["params"] = converted_parameters(parameters) + access = "readwrite" if read_indexes and write_indexes else ( + "read" if read_indexes else "write" + ) + member = { + "kind": "property", + "name": name_token.value, + "visibility": visibility, + **details, + "access": access, + } + if property_type: + member["type"] = property_type + return member, end_index + 1 + + +def skip_attributes(tokens, index, end_index, path): + while index < end_index and tokens[index].value == "[": + depth = 1 + index += 1 + while index < end_index and depth: + if tokens[index].value == "[": + depth += 1 + elif tokens[index].value == "]": + depth -= 1 + index += 1 + if depth: + fail(path, tokens[index - 1].line, "成员属性缺少右方括号") + return index + + +def parse_field_member(source, tokens, start_index, path, visibility, is_static): + end_index = declaration_end(tokens, start_index, path) + cursor = start_index + (1 if is_static else 0) + cursor = skip_attributes(tokens, cursor, end_index, path) + if cursor >= end_index or tokens[cursor].kind != "identifier": + fail(path, tokens[start_index].line, "字段声明缺少有效名称") + name_token = tokens[cursor] + colon_index = next( + ( + index + for index in range(cursor + 1, end_index) + if tokens[index].value == ":" + ), + None, + ) + comma_limit = colon_index if colon_index is not None else end_index + if visibility != "private" and any( + tokens[index].value == "," for index in range(cursor + 1, comma_limit) + ): + fail(path, name_token.line, "对外字段必须一项一条声明") + equals_index = next( + ( + index + for index in range(cursor + 1, end_index) + if tokens[index].value == "=" + ), + None, + ) + field_type = "" + if colon_index is not None: + type_end = equals_index if equals_index is not None else end_index + field_type = source[ + tokens[colon_index].end : tokens[type_end].start + ].strip() + details = ( + parse_simple_document( + document_before(source, tokens[start_index].line), path, "字段" + ) + if visibility != "private" + else {"desc": ""} + ) + member = { + "kind": "field", + "name": name_token.value, + "visibility": visibility, + **details, + "type": field_type, + } + if is_static: + member["static"] = True + return member, end_index + 1 + + +def parse_constant_member(source, tokens, start_index, path, visibility, is_static): + const_index = start_index + (1 if is_static else 0) + end_index = declaration_end(tokens, const_index + 1, path) + if const_index + 1 >= end_index or tokens[const_index + 1].kind != "identifier": + fail(path, tokens[const_index].line, "const 声明缺少有效名称") + name_token = tokens[const_index + 1] + equals_index = next( + ( + index + for index in range(const_index + 2, end_index) + if tokens[index].value == "=" + ), + None, + ) + if equals_index is None: + fail(path, name_token.line, "const 声明缺少值") + colon_index = next( + ( + index + for index in range(const_index + 2, equals_index) + if tokens[index].value == ":" + ), + None, + ) + name_end_index = colon_index if colon_index is not None else equals_index + if visibility != "private" and any( + tokens[index].value == "," + for index in range(const_index + 2, name_end_index) + ): + fail(path, name_token.line, "对外常量必须一项一条声明") + constant_type = "" + if colon_index is not None: + constant_type = source[ + tokens[colon_index].end : tokens[equals_index].start + ].strip() + value = source[tokens[equals_index].end : tokens[end_index].start].strip() + details = ( + parse_simple_document( + document_before(source, name_token.line), path, "常量" + ) + if visibility != "private" + else {"desc": ""} + ) + member = { + "kind": "constant", + "name": name_token.value, + "visibility": visibility, + **details, + "value": value, + } + if constant_type: + member["type"] = constant_type + if is_static: + member["static"] = True + return member, end_index + 1 + + +def parse_bases(source, tokens, class_index, path): + if class_index + 1 >= len(tokens) or tokens[class_index + 1].value != "(": + return [], class_index + 1, tokens[class_index] + open_index = class_index + 1 + close_index = matching_parenthesis(tokens, open_index, path) + raw_start = tokens[open_index].end + raw_end = tokens[close_index].start + bases = [] + for start, end in split_parameter_ranges(tokens, raw_start, raw_end): + base = source[start:end].strip() + if base: + bases.append(base) + return bases, close_index + 1, tokens[close_index] + + +def first_unbound_document_line(source, start_line, end_line, allowed_lines): + for line_number, text in enumerate( + source.splitlines()[start_line - 1 : end_line], start=start_line + ): + if DOCUMENT_LINE_RE.match(text) and line_number not in allowed_lines: + return line_number + return None + + +def parse_class(source, tokens, type_index, path, *, require_filename_match=True): + if type_index + 3 >= len(tokens): + fail(path, tokens[type_index].line, "class 声明不完整") + name_token = tokens[type_index + 1] + if name_token.kind != "identifier" or tokens[type_index + 2].value != "=": + fail(path, tokens[type_index].line, "type class 声明格式错误") + class_index = type_index + 3 + if not keyword_at(tokens, class_index, "class"): + fail(path, tokens[type_index].line, "目前只支持 class type") + if ( + require_filename_match + and name_token.value.casefold() != Path(path).stem.casefold() + ): + fail(path, name_token.line, "对外 class 名称必须与文件名一致") + + bases, index, header_end = parse_bases(source, tokens, class_index, path) + class_document = ( + document_before(source, tokens[index].line, header_end.line + 1) + if index < len(tokens) + else [] + ) + class_details = parse_simple_document(class_document, path, "class") + used_class_doc_lines = {line.line for line in class_document} + allowed_document_lines = set(used_class_doc_lines) + visibility = "public" + members = [] + + while index < len(tokens): + token = tokens[index] + lowered = token.value.casefold() if token.kind == "identifier" else "" + if lowered == "end": + if index + 1 >= len(tokens) or tokens[index + 1].value != ";": + fail(path, token.line, "class 结尾必须是 end;") + unbound_line = first_unbound_document_line( + source, + header_end.line + 1, + token.line, + allowed_document_lines, + ) + if unbound_line is not None: + fail(path, unbound_line, "class 文档块无法绑定") + result = { + "name": name_token.value, + **class_details, + "members": members, + } + if bases: + result["bases"] = bases + return result, index + 2 + if lowered in {"public", "protected", "private"}: + visibility = lowered + index += 1 + continue + if lowered == "uses": + index = declaration_end(tokens, index + 1, path) + 1 + continue + + declaration_line = token.line + document = document_before(source, declaration_line) + if document and any(line.line in used_class_doc_lines for line in document): + document = [] + allowed_document_lines.update(line.line for line in document) + + if lowered == "class" and keyword_at(tokens, index + 1, "function"): + member, index = parse_method_member( + source, tokens, index + 1, path, visibility, "class" + ) + elif lowered == "function": + member, index = parse_method_member( + source, tokens, index, path, visibility, "instance" + ) + elif lowered == "procedure": + fail(path, token.line, "class 暂不支持 procedure") + elif lowered == "property": + member, index = parse_property_member( + source, tokens, index, path, visibility + ) + elif lowered == "static": + if keyword_at(tokens, index + 1, "function"): + fail(path, token.line, "不存在 static function，请使用 class function") + if keyword_at(tokens, index + 1, "const"): + member, index = parse_constant_member( + source, tokens, index, path, visibility, True + ) + elif any( + keyword_at(tokens, index + 1, keyword) + for keyword in {"class", "procedure", "property", "type", "var"} + ): + fail(path, token.line, f"class 不支持成员声明：static {tokens[index + 1].value}") + else: + member, index = parse_field_member( + source, tokens, index, path, visibility, True + ) + elif lowered == "const": + member, index = parse_constant_member( + source, tokens, index, path, visibility, False + ) + elif lowered in { + "class", + "constructor", + "destructor", + "finalization", + "implementation", + "initialization", + "interface", + "type", + "unit", + "var", + }: + fail(path, token.line, f"class 不支持成员声明：{token.value}") + else: + member, index = parse_field_member( + source, tokens, index, path, visibility, False + ) + + if visibility != "private": + if document and not member.get("desc"): + member_doc_lines = {line.line for line in document} + if member_doc_lines != used_class_doc_lines: + fail(path, document[0].line, "成员文档块无法绑定") + members.append(member) + fail(path, name_token.line, "class 缺少 end;") + + +def parse_interface_function(source, tokens, function_index, path): + name, parameters, declared_return, end_index = parse_function_signature( + source, tokens, function_index, path + ) + document = document_before(source, tokens[function_index].line) + if document: + details = parse_document(document, parameters, declared_return, path) + else: + details = {"desc": ""} + if parameters: + details["params"] = converted_parameters(parameters) + details["returns"] = declared_return + names = ", ".join(parameter.name for parameter in parameters) + return { + "kind": "function", + "name": name, + "signature": f"{name}({names})", + **details, + }, end_index + 1 + + +def parse_unit_constant(source, tokens, const_index, path): + member, next_index = parse_constant_member( + source, tokens, const_index, path, "public", False + ) + member.pop("visibility", None) + return member, next_index + + +def parse_unit_variable(source, tokens, var_index, path): + if var_index + 1 >= len(tokens): + fail(path, tokens[var_index].line, "var 声明缺少变量") + member, next_index = parse_field_member( + source, tokens, var_index + 1, path, "public", False + ) + member["kind"] = "variable" + member.pop("visibility", None) + return member, next_index + + +def parse_bare_unit_variable(source, tokens, start_index, path): + member, next_index = parse_field_member( + source, tokens, start_index, path, "public", False + ) + member["kind"] = "variable" + member.pop("visibility", None) + return member, next_index + + +def parse_bare_unit_constant(source, tokens, start_index, path): + end_index = declaration_end(tokens, start_index + 1, path) + name_token = tokens[start_index] + if name_token.kind != "identifier": + fail(path, name_token.line, "const 声明缺少有效名称") + equals_index = next( + ( + index + for index in range(start_index + 1, end_index) + if tokens[index].value == "=" + ), + None, + ) + if equals_index is None: + fail(path, name_token.line, "const 声明缺少值") + colon_index = next( + ( + index + for index in range(start_index + 1, equals_index) + if tokens[index].value == ":" + ), + None, + ) + name_end_index = colon_index if colon_index is not None else equals_index + if any( + tokens[index].value == "," + for index in range(start_index + 1, name_end_index) + ): + fail(path, name_token.line, "对外常量必须一项一条声明") + member = { + "kind": "constant", + "name": name_token.value, + **parse_simple_document( + document_before(source, name_token.line), path, "常量" + ), + "value": source[tokens[equals_index].end : tokens[end_index].start].strip(), + } + if colon_index is not None: + member["type"] = source[ + tokens[colon_index].end : tokens[equals_index].start + ].strip() + return member, end_index + 1 + + +def parse_unit(source, tokens, unit_index, path): + if unit_index + 2 >= len(tokens): + fail(path, tokens[unit_index].line, "unit 声明不完整") + name_token = tokens[unit_index + 1] + if name_token.kind != "identifier" or tokens[unit_index + 2].value != ";": + fail(path, tokens[unit_index].line, "unit 声明格式错误") + if name_token.value.casefold() != Path(path).stem.casefold(): + fail(path, name_token.line, "unit 名称必须与文件名一致") + + interface_index = next( + ( + index + for index in range(unit_index + 3, len(tokens)) + if keyword_at(tokens, index, "interface") + ), + None, + ) + if interface_index is None: + fail(path, name_token.line, "unit API 文档要求显式 interface") + implementation_index = next( + ( + index + for index in range(interface_index + 1, len(tokens)) + if keyword_at(tokens, index, "implementation") + ), + None, + ) + if implementation_index is None: + fail(path, tokens[interface_index].line, "完整 unit 缺少 implementation") + terminal_end_indexes = [ + index + for index in range(implementation_index + 1, len(tokens)) + if keyword_at(tokens, index, "end") + and index + 1 < len(tokens) + and tokens[index + 1].value == "." + ] + if not terminal_end_indexes: + fail(path, tokens[implementation_index].line, "完整 unit 缺少 end.") + terminal_end_index = terminal_end_indexes[-1] + if terminal_end_index + 2 != len(tokens): + fail( + path, + tokens[terminal_end_index + 2].line, + "unit 的 end. 必须结束整个文件", + ) + + unit_document = document_before( + source, tokens[interface_index].line, tokens[unit_index + 2].line + 1 + ) + details = parse_simple_document(unit_document, path, "unit") + members = [] + allowed_document_lines = {line.line for line in unit_document} + index = interface_index + 1 + declaration_section = None + while index < implementation_index: + token = tokens[index] + lowered = token.value.casefold() if token.kind == "identifier" else "" + if lowered == "uses": + declaration_section = None + index = declaration_end(tokens, index + 1, path) + 1 + continue + if lowered == "function": + declaration_section = None + allowed_document_lines.update( + line.line for line in document_before(source, token.line) + ) + member, index = parse_interface_function(source, tokens, index, path) + members.append(member) + continue + if lowered == "procedure": + fail(path, token.line, "unit interface 暂不支持 procedure") + if lowered == "type": + declaration_section = None + if index + 3 >= implementation_index or not keyword_at( + tokens, index + 3, "class" + ): + fail(path, token.line, "unit interface 只支持 class type") + class_start_line = token.line + class_data, index = parse_class( + source, + tokens, + index, + path, + require_filename_match=False, + ) + class_end_line = tokens[index - 1].line + allowed_document_lines.update( + line_number + for line_number, text in enumerate( + source.splitlines()[class_start_line - 1 : class_end_line], + start=class_start_line, + ) + if DOCUMENT_LINE_RE.match(text) + ) + members.append({"kind": "class", **class_data}) + continue + if lowered == "const": + declaration_section = "const" + if index + 1 < implementation_index: + allowed_document_lines.update( + line.line + for line in document_before(source, tokens[index + 1].line) + ) + member, index = parse_unit_constant(source, tokens, index, path) + members.append(member) + continue + if lowered == "var": + declaration_section = "var" + if index + 1 < implementation_index: + allowed_document_lines.update( + line.line + for line in document_before(source, tokens[index + 1].line) + ) + member, index = parse_unit_variable(source, tokens, index, path) + members.append(member) + continue + if token.kind == "identifier" and declaration_section == "var": + allowed_document_lines.update( + line.line for line in document_before(source, token.line) + ) + member, index = parse_bare_unit_variable(source, tokens, index, path) + members.append(member) + continue + if token.kind == "identifier" and declaration_section == "const": + allowed_document_lines.update( + line.line for line in document_before(source, token.line) + ) + member, index = parse_bare_unit_constant(source, tokens, index, path) + members.append(member) + continue + fail(path, token.line, f"unit interface 不支持声明：{token.value}") + + unbound_line = first_unbound_document_line( + source, + tokens[unit_index + 2].line + 1, + tokens[implementation_index].line - 1, + allowed_document_lines, + ) + if unbound_line is not None: + fail(path, unbound_line, "unit interface 文档块无法绑定") + + return { + "name": name_token.value, + **details, + "members": members, + } + + +def convert_source(source, path): + tokens = tokenize(source) + structure_token = first_structure_token(tokens, path) + structure_index = tokens.index(structure_token) + kind = structure_token.value.casefold() + if kind == "function": + name, parameters, declared_return, begin_token = parse_declaration( + source, tokens, structure_token, path + ) + document = extract_document_lines(source, begin_token, path) + function = parse_document(document, parameters, declared_return, path) + names = ", ".join(parameter.name for parameter in parameters) + return { + "kind": "function", + "name": name, + "signature": f"{name}({names})", + **function, + } + if kind == "type": + class_data, _ = parse_class(source, tokens, structure_index, path) + return {"kind": "class", **class_data} + if kind == "unit": + return { + "kind": "unit", + **parse_unit(source, tokens, structure_index, path), + } + fail(path, structure_token.line, "目前支持独立顶层 function、class 和完整 unit") + + +def read_tsf(path): + if path.suffix.lower() != ".tsf": + fail(path, None, "输入文件扩展名必须是 .tsf") + if not path.is_file(): + fail(path, None, "输入文件不存在") + source_bytes = path.read_bytes() + for encoding in ("utf-8-sig", "gb18030"): + try: + return source_bytes.decode(encoding) + except UnicodeDecodeError: + continue + fail(path, None, "文件不是有效的 utf-8 或 gb18030 编码") + + +def complete_draft_fields(item): + item.setdefault("desc", "") + kind = item["kind"] + + if kind in {"function", "method"}: + item.setdefault("params", []) + item.setdefault("returns", "") + for parameter in item["params"]: + parameter.setdefault("type", "") + parameter.setdefault("desc", "") + return item + + if kind == "property": + item.setdefault("type", "") + item.setdefault("params", []) + for parameter in item["params"]: + parameter.setdefault("type", "") + parameter.setdefault("desc", "") + return item + + if kind in {"field", "variable", "constant"}: + item.setdefault("type", "") + return item + + if kind in {"class", "unit"}: + item.setdefault("members", []) + for member in item["members"]: + complete_draft_fields(member) + return item + + +def serialize(data, output_format, output_path): + if output_format == "json": + return json.dumps(data, ensure_ascii=False, indent=2) + "\n" + try: + import yaml + except ImportError: + fail( + output_path, None, "生成 yaml 需要安装 pyyaml：python -m pip install pyyaml" + ) + return yaml.safe_dump( + data, + allow_unicode=True, + sort_keys=False, + default_flow_style=False, + ) + + +def validate_declaration_uniqueness(declarations, path): + function_signatures = set() + class_names = set() + unit_names = set() + for declaration in declarations: + kind = declaration["kind"] + if kind == "function": + key = declaration["signature"].casefold() + if key in function_signatures: + fail( + path, + None, + "重复 function signature：" + f"{declaration['signature']}", + ) + function_signatures.add(key) + continue + names = class_names if kind == "class" else unit_names + key = declaration["name"].casefold() + if key in names: + fail(path, None, f"重复 {kind}：{declaration['name']}") + names.add(key) + + +def atomic_write(path, text): + path.parent.mkdir(parents=True, exist_ok=True) + descriptor, temporary_name = tempfile.mkstemp( + prefix=f".{path.name}.", suffix=".tmp", dir=path.parent + ) + temporary_path = Path(temporary_name) + try: + with os.fdopen( + descriptor, "w", encoding="utf-8", newline="\n" + ) as handle: + handle.write(text) + handle.flush() + os.fsync(handle.fileno()) + os.replace(temporary_path, path) + finally: + if temporary_path.exists(): + temporary_path.unlink() + + +def build_parser(): + parser = argparse.ArgumentParser( + description=( + "把带标准文档块的 tsf function、class 和 unit " + "转换为 json 或 yaml declarations 录入文件" + ), + formatter_class=ChineseHelpFormatter, + add_help=False, + allow_abbrev=False, + ) + parser._positionals.title = "位置参数" + parser._optionals.title = "选项" + parser.add_argument( + "inputs", + nargs="+", + metavar="tsf文件", + help="一个或多个 tsf 文件；按给定顺序生成 declarations", + ) + parser.add_argument( + "--format", + required=True, + choices=["json", "yaml"], + metavar="格式", + help="输出格式，可选 json 或 yaml", + ) + parser.add_argument( + "--module", + required=True, + metavar="模块标题", + help="录入文件的 module 值，即 markdown 一级标题", + ) + parser.add_argument( + "--path", + required=True, + metavar="页面路径", + help="录入文件的 path 值，不包含 .md 后缀", + ) + parser.add_argument( + "--output", + required=True, + metavar="输出文件", + help="json 或 yaml 输出文件路径", + ) + parser.add_argument( + "--help", + action="help", + help="显示此帮助信息并退出（不提供 -h 短选项）", + ) + return parser + + +def main(argv=None): + if hasattr(sys.stdout, "reconfigure"): + sys.stdout.reconfigure(encoding="utf-8") + if hasattr(sys.stderr, "reconfigure"): + sys.stderr.reconfigure(encoding="utf-8") + args = build_parser().parse_args(argv) + output_path = Path(args.output) + + try: + input_paths = [Path(input_value) for input_value in args.inputs] + resolved_output = output_path.resolve() + if any(input_path.resolve() == resolved_output for input_path in input_paths): + fail(output_path, None, "输出文件不能覆盖输入 tsf") + declarations = [ + complete_draft_fields(convert_source(read_tsf(input_path), input_path)) + for input_path in input_paths + ] + validate_declaration_uniqueness(declarations, output_path) + data = { + "module": args.module, + "path": args.path, + "declarations": declarations, + } + output_text = serialize(data, args.format, output_path) + atomic_write(output_path, output_text) + except ConversionError as exc: + print(f"错误：{exc}", file=sys.stderr) + return 1 + except OSError as exc: + print(f"错误：{exc}", file=sys.stderr) + return 1 + + print(f"已写入 {output_path}", file=sys.stderr) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tools/tsl-codegen/scripts/generate.py b/tools/tsl-codegen/scripts/generate.py index 595f5208..10297eb8 100644 --- a/tools/tsl-codegen/scripts/generate.py +++ b/tools/tsl-codegen/scripts/generate.py @@ -2,17 +2,20 @@ """Generate compliant TSL codegen markdown from a YAML/JSON entry file. The recording format is one leaf page: a `module` title, a relative `path`, and -a `functions` list. This script renders it to the markdown the codegen tree -stores, matching tools/tsl-codegen/STANDARD.md. +an ordered `declarations` list containing function, class, or unit entries. This +script renders it to the markdown the codegen tree stores, matching +tools/tsl-codegen/STANDARD.md. -Tables are emitted as valid Markdown with single-space padding. Prettier may be -used optionally to align columns. +Rendered Markdown is passed through the repository-pinned Prettier before it is +written, keeping generated pages consistent with the existing codegen tree. Input dispatch is by extension: .json parses with the stdlib (keeping the toolchain dependency-free); .yml/.yaml needs pyyaml. If pyyaml is missing the script says so and points at the JSON path. -Entry schema (per function): +Function declaration fields: + kind required `function` + name required must match the signature name case-insensitively signature required verbatim, underscores/case untouched desc required description; may contain multiple lines tags optional list of Chinese keywords -> `` @@ -27,11 +30,20 @@ Usage (run from repo root): python tools/tsl-codegen/scripts/generate.py entry.json \ --scope my-project """ + import argparse import json +import os +import shutil +import subprocess import sys +import tempfile from pathlib import Path +REPO_ROOT = Path(__file__).resolve().parents[3] +PRETTIER_CONFIG = REPO_ROOT / ".prettierrc.json" + + def die(msg): print(f"ERROR: {msg}", file=sys.stderr) raise SystemExit(1) @@ -53,10 +65,7 @@ def resolve_format(path, fmt): return "json" if suffix in (".yml", ".yaml"): return "yaml" - die( - f"cannot infer format from extension '{suffix}'; " - f"pass --format json|yaml" - ) + die(f"cannot infer format from extension '{suffix}'; " f"pass --format json|yaml") def load_entries(path, fmt=None): @@ -96,6 +105,322 @@ def require(cond, msg): die(msg) +def require_mapping(value, where): + require(isinstance(value, dict), f"{where}: must be a mapping") + + +def reject_unknown(mapping, allowed, where): + unknown = sorted(set(mapping) - set(allowed)) + require(not unknown, f"{where}: unknown field(s): {', '.join(unknown)}") + + +def non_empty_string(value, where): + require(isinstance(value, str) and value.strip(), f"{where}: must be non-empty") + + +def optional_draft_string(value, where): + if value == "": + return + non_empty_string(value, where) + + +def validate_tags(tags, where): + if tags is None: + return + require(isinstance(tags, list) and tags, f"{where}: tags must be a non-empty list") + for index, tag in enumerate(tags): + non_empty_string(tag, f"{where}: tags[{index}]") + + +def signature_names(signature, where): + non_empty_string(signature, f"{where}: signature") + left = signature.find("(") + right = signature.rfind(")") + require(left > 0 and right == len(signature) - 1, f"{where}: invalid signature") + name = signature[:left] + non_empty_string(name, f"{where}: signature name") + raw = signature[left + 1 : right].strip() + if not raw: + return name, [] + names = [item.strip() for item in raw.split(",")] + require(all(names), f"{where}: signature contains an empty parameter") + require(len({item.casefold() for item in names}) == len(names), f"{where}: duplicate parameter name") + return name, names + + +def validate_values(values, where): + require(isinstance(values, list) and values, f"{where}: values must be a non-empty list") + for index, item in enumerate(values): + item_where = f"{where}[{index}]" + require_mapping(item, item_where) + reject_unknown(item, {"value", "desc"}, item_where) + require("value" in item, f"{item_where}: missing 'value'") + non_empty_string(item.get("desc"), f"{item_where}: desc") + + +def validate_params(params, expected_names, where): + if not expected_names: + require(not params, f"{where}: nullary signature must not have params") + return + require(isinstance(params, list), f"{where}: params must be a list") + require(len(params) == len(expected_names), f"{where}: params do not match signature") + actual_names = [] + for index, param in enumerate(params): + param_where = f"{where}: params[{index}]" + require_mapping(param, param_where) + reject_unknown(param, {"name", "type", "desc", "optional", "values"}, param_where) + name = param.get("name") + non_empty_string(name, f"{param_where}: name") + non_empty_string(param.get("type"), f"{param_where}: type") + non_empty_string(param.get("desc"), f"{param_where}: desc") + if "optional" in param: + require(isinstance(param["optional"], bool), f"{param_where}: optional must be boolean") + if "values" in param: + validate_values(param["values"], f"{param_where}: values") + actual_names.append(name) + require( + [name.casefold() for name in actual_names] + == [name.casefold() for name in expected_names], + f"{where}: params must follow signature order", + ) + + +def validate_examples(examples, where): + require(isinstance(examples, list) and examples, f"{where}: examples must be a non-empty list") + for index, example in enumerate(examples): + example_where = f"{where}: examples[{index}]" + require_mapping(example, example_where) + reject_unknown(example, {"desc", "code", "output"}, example_where) + non_empty_string(example.get("desc"), f"{example_where}: desc") + non_empty_string(example.get("code"), f"{example_where}: code") + if "output" in example: + non_empty_string(example["output"], f"{example_where}: output") + + +def validate_function(fn, where, *, returns_required, extra_fields=()): + require_mapping(fn, where) + allowed = { + "signature", + "desc", + "tags", + "params", + "returns", + "example", + "examples", + *extra_fields, + } + reject_unknown(fn, allowed, where) + name, names = signature_names(fn.get("signature"), where) + non_empty_string(fn.get("desc"), f"{where}: desc") + validate_tags(fn.get("tags"), where) + validate_params(fn.get("params"), names, where) + if returns_required: + non_empty_string(fn.get("returns"), f"{where}: missing 'returns'") + elif "returns" in fn: + optional_draft_string(fn["returns"], f"{where}: returns") + require(not ("example" in fn and "examples" in fn), f"{where}: use example or examples, not both") + if "example" in fn: + non_empty_string(fn["example"], f"{where}: example") + if "examples" in fn: + validate_examples(fn["examples"], where) + return name + + +def validate_class_member(member, where): + require_mapping(member, where) + kind = member.get("kind") + require(kind in {"method", "property", "field", "constant"}, f"{where}: unknown kind '{kind}'") + non_empty_string(member.get("name"), f"{where}: name") + visibility = member.get("visibility") + require(visibility in {"public", "protected"}, f"{where}: visibility must be public or protected") + non_empty_string(member.get("desc"), f"{where}: desc") + validate_tags(member.get("tags"), where) + + if kind == "method": + reject_unknown( + member, + { + "kind", "name", "visibility", "binding", "signature", "desc", + "tags", "params", "returns", "modifiers", "example", "examples", + }, + where, + ) + require(member.get("binding") in {"instance", "class"}, f"{where}: invalid binding") + parsed_name = validate_function( + member, + where, + returns_required=False, + extra_fields={"kind", "name", "visibility", "binding", "modifiers"}, + ) + require(parsed_name.casefold() == member["name"].casefold(), f"{where}: name and signature differ") + if "modifiers" in member: + modifiers = member["modifiers"] + require(isinstance(modifiers, list), f"{where}: modifiers must be a list") + allowed = {"overload", "virtual", "override"} + require(all(item in allowed for item in modifiers), f"{where}: invalid modifier") + require(len(set(modifiers)) == len(modifiers), f"{where}: duplicate modifier") + return + + common = {"kind", "name", "visibility", "desc", "tags"} + if kind == "property": + reject_unknown(member, common | {"type", "params", "access"}, where) + if "type" in member: + optional_draft_string(member["type"], f"{where}: type") + require(member.get("access") in {"read", "write", "readwrite"}, f"{where}: invalid access") + params = member.get("params") + if params: + expected = [param.get("name") for param in params] + validate_params(params, expected, where) + return + if kind == "field": + reject_unknown(member, common | {"type", "static"}, where) + non_empty_string(member.get("type"), f"{where}: type") + if "static" in member: + require(isinstance(member["static"], bool), f"{where}: static must be boolean") + return + reject_unknown(member, common | {"type", "value", "static"}, where) + require("value" in member and member["value"] is not None, f"{where}: missing 'value'") + if isinstance(member["value"], str): + non_empty_string(member["value"], f"{where}: value") + if "type" in member: + optional_draft_string(member["type"], f"{where}: type") + if "static" in member: + require(isinstance(member["static"], bool), f"{where}: static must be boolean") + + +def validate_class(cls, where): + require_mapping(cls, where) + reject_unknown(cls, {"kind", "name", "desc", "tags", "bases", "members"}, where) + require(cls.get("kind") == "class", f"{where}: kind must be class") + non_empty_string(cls.get("name"), f"{where}: name") + non_empty_string(cls.get("desc"), f"{where}: desc") + validate_tags(cls.get("tags"), where) + if "bases" in cls: + require(isinstance(cls["bases"], list), f"{where}: bases must be a list") + for index, base in enumerate(cls["bases"]): + non_empty_string(base, f"{where}: bases[{index}]") + require(isinstance(cls.get("members"), list), f"{where}: members must be a list") + for index, member in enumerate(cls["members"]): + validate_class_member(member, f"{where}: members[{index}]") + + +def validate_unit_member(member, where): + require_mapping(member, where) + kind = member.get("kind") + require(kind in {"function", "variable", "constant", "class"}, f"{where}: unknown kind '{kind}'") + if kind == "class": + validate_class(member, where) + return + non_empty_string(member.get("name"), f"{where}: name") + non_empty_string(member.get("desc"), f"{where}: desc") + validate_tags(member.get("tags"), where) + if kind == "function": + parsed_name = validate_function( + member, + where, + returns_required=True, + extra_fields={"kind", "name"}, + ) + require(parsed_name.casefold() == member["name"].casefold(), f"{where}: name and signature differ") + return + common = {"kind", "name", "desc", "tags", "type"} + if kind == "variable": + reject_unknown(member, common, where) + non_empty_string(member.get("type"), f"{where}: type") + return + reject_unknown(member, common | {"value"}, where) + require("value" in member and member["value"] is not None, f"{where}: missing 'value'") + if isinstance(member["value"], str): + non_empty_string(member["value"], f"{where}: value") + if "type" in member: + optional_draft_string(member["type"], f"{where}: type") + + +def validate_unit(unit, where): + require_mapping(unit, where) + reject_unknown(unit, {"kind", "name", "desc", "tags", "members"}, where) + require(unit.get("kind") == "unit", f"{where}: kind must be unit") + non_empty_string(unit.get("name"), f"{where}: name") + non_empty_string(unit.get("desc"), f"{where}: desc") + validate_tags(unit.get("tags"), where) + require( + isinstance(unit.get("members"), list), + f"{where}: members must be a list", + ) + for index, member in enumerate(unit["members"]): + validate_unit_member(member, f"{where}: members[{index}]") + + +def validate_top_level_function(declaration, where): + require( + declaration.get("kind") == "function", + f"{where}: kind must be function", + ) + non_empty_string(declaration.get("name"), f"{where}: name") + parsed_name = validate_function( + declaration, + where, + returns_required=True, + extra_fields={"kind", "name"}, + ) + require( + parsed_name.casefold() == declaration["name"].casefold(), + f"{where}: name and signature differ", + ) + + +def validate_declaration(declaration, where): + require_mapping(declaration, where) + kind = declaration.get("kind") + require( + kind in {"function", "class", "unit"}, + f"{where}: unknown kind '{kind}'", + ) + if kind == "function": + validate_top_level_function(declaration, where) + elif kind == "class": + validate_class(declaration, where) + else: + validate_unit(declaration, where) + + +def validate_declaration_uniqueness(declarations): + function_signatures = set() + class_names = set() + unit_names = set() + for index, declaration in enumerate(declarations): + where = f"declarations[{index}]" + kind = declaration["kind"] + if kind == "function": + key = declaration["signature"].casefold() + require( + key not in function_signatures, + f"{where}: duplicate function signature", + ) + function_signatures.add(key) + continue + names = class_names if kind == "class" else unit_names + key = declaration["name"].casefold() + require(key not in names, f"{where}: duplicate {kind} name") + names.add(key) + + +def validate_page(data): + require_mapping(data, "input root") + reject_unknown(data, {"module", "path", "declarations"}, "input root") + non_empty_string(data.get("module"), "input root: module") + non_empty_string(data.get("path"), "input root: path") + declarations = data.get("declarations") + require( + isinstance(declarations, list) and declarations, + "input root: declarations must be a non-empty list", + ) + for index, declaration in enumerate(declarations): + validate_declaration(declaration, f"declarations[{index}]") + validate_declaration_uniqueness(declarations) + return declarations + + def param_desc(param, where): """Description column text: prepend `可选。` for optional params.""" desc = param.get("desc") @@ -120,14 +445,17 @@ def render_param_table(params, where): return lines -def render_enum_sections(params): - """`**name 取值**` sections for every param carrying a `values` list.""" +def render_enum_sections(params, heading_level=None): + """Render value sections, preserving legacy function-page headings.""" lines = [] for param in params: values = param.get("values") if not values: continue - lines.append(f"**{param['name']} 取值**") + if heading_level is None: + lines.append(f"**{param['name']} 取值**") + else: + lines.append(f"{'#' * heading_level} `{param['name']}` 取值") lines.append("") for item in values: lines.append(f"- `{item['value']}` — {item['desc']}") @@ -135,23 +463,89 @@ def render_enum_sections(params): return lines -def render_function(fn, index): - """Render one function entry to a list of lines (no trailing blank).""" - where = f"functions[{index}]" - sig = fn.get("signature") - require(sig, f"{where}: missing 'signature'") - desc = fn.get("desc") - require(desc, f"{where} ({sig}): missing 'desc'") - returns = fn.get("returns") - require(returns, f"{where} ({sig}): missing 'returns'") +def render_examples(fn, heading_level): + lines = [] + if "examples" in fn: + lines.extend([f"{'#' * heading_level} 示例", ""]) + for index, example in enumerate(fn["examples"], start=1): + lines.append(f"范例{index:02d}：{example['desc']}") + lines.append("") + lines.append("```tsl") + lines.extend(example["code"].rstrip("\n").split("\n")) + output = example.get("output") + if output is not None: + output_lines = output.split("\n") + if len(output_lines) == 1: + lines.append(f"// 输出：{output_lines[0]}") + else: + lines.append("// 输出：") + lines.extend(f"// {line}" if line else "//" for line in output_lines) + lines.append("```") + lines.append("") + while lines and not lines[-1]: + lines.pop() + return lines + if fn.get("example"): + return [ + f"{'#' * heading_level} 示例", + "", + "```tsl", + *fn["example"].rstrip("\n").split("\n"), + "```", + ] + return [] - lines = [f"## `{sig}`"] - tags = fn.get("tags") + +def render_intro(heading, declaration, item): + lines = [heading, "", f"声明：{declaration}", "", item["desc"], ""] + tags = item.get("tags") if tags: - lines.append(f"") - lines.append("") - lines.append(desc) - lines.append("") + lines.extend([f"", ""]) + return lines + + +def render_callable( + fn, + signature, + declaration, + where, + *, + level, + returns_required, + show_visibility, +): + validate_function( + fn, + where, + returns_required=returns_required, + extra_fields={"kind", "name", "visibility", "binding", "modifiers"}, + ) + lines = render_intro( + f"{'#' * level} `{signature}`", declaration, fn + ) + if show_visibility: + lines.extend([f"可见性：`{fn['visibility']}`", ""]) + if fn.get("modifiers"): + rendered = "、".join(f"`{item}`" for item in fn["modifiers"]) + lines.extend([f"修饰符：{rendered}", ""]) + params = fn.get("params") or [] + if params: + lines.extend(render_param_table(params, where)) + lines.append("") + lines.extend(render_enum_sections(params, level + 1)) + if fn.get("returns"): + lines.append(f"返回：{fn['returns']}") + example_lines = render_examples(fn, level + 1) + if example_lines: + lines.extend(["", *example_lines]) + return lines + + +def render_top_level_function(fn, index): + """Render one function entry to a list of lines (no trailing blank).""" + where = f"declarations[{index}]" + sig = fn["signature"] + lines = render_intro(f"## `{sig}`", "function", fn) params = fn.get("params") or [] if params: @@ -159,34 +553,168 @@ def render_function(fn, index): lines.append("") lines.extend(render_enum_sections(params)) - lines.append(f"返回：{returns}") - - example = fn.get("example") - if example: - lines.append("") - lines.append("### 示例") - lines.append("") - lines.append("```tsl") - lines.extend(example.rstrip("\n").split("\n")) - lines.append("```") + lines.append(f"返回：{fn['returns']}") + examples = render_examples(fn, 3) + if examples: + lines.extend(["", *examples]) return lines -def render_page(data): - """Render a whole leaf page: H1 + every function entry.""" - require(isinstance(data, dict), "input root must be a mapping") - module = data.get("module") - require(module, "input missing 'module'") - functions = data.get("functions") - require(functions, "input missing non-empty 'functions'") +def render_class_member(member, level, where): + kind = member["kind"] + if kind == "method": + declaration = ( + "class function" + if member["binding"] == "class" + else "function" + ) + return render_callable( + member, + member["signature"], + declaration, + where, + level=level, + returns_required=False, + show_visibility=True, + ) + static_prefix = "static " if member.get("static") else "" + declaration = { + "property": "property", + "field": f"{static_prefix}field", + "constant": f"{static_prefix}const", + }[kind] + signature = member["name"] + if kind == "property" and member.get("params"): + signature += "(" + ", ".join(param["name"] for param in member["params"]) + ")" + lines = render_intro( + f"{'#' * level} `{signature}`", declaration, member + ) + lines.extend([f"可见性：`{member['visibility']}`", ""]) + if kind == "property": + if member.get("type"): + lines.extend([f"类型：{member['type']}", ""]) + access = {"read": "read", "write": "write", "readwrite": "read / write"}[member["access"]] + lines.append(f"访问：{access}") + params = member.get("params") or [] + if params: + lines.extend(["", *render_param_table(params, where), ""]) + lines.extend(render_enum_sections(params, level + 1)) + elif kind == "field": + lines.append(f"类型：{member['type']}") + else: + if member.get("type"): + lines.extend([f"类型：{member['type']}", ""]) + lines.append(f"值：`{render_scalar(member['value'])}`") + return lines - out = [f"# {module}", ""] - for index, fn in enumerate(functions): - out.extend(render_function(fn, index)) + +def render_class(cls, level, where, *, page_root): + lines = render_intro( + f"{'#' * level} `{cls['name']}`", "class", cls + ) + if cls.get("bases"): + lines.extend(["父类：" + "、".join(f"`{base}`" for base in cls["bases"]), ""]) + for index, member in enumerate(cls["members"]): + lines.extend(render_class_member(member, level + 1, f"{where}: members[{index}]")) + lines.append("") + while lines and not lines[-1]: + lines.pop() + return lines + + +def render_unit_member(member, index, unit_where): + where = f"{unit_where}: members[{index}]" + kind = member["kind"] + if kind == "function": + return render_callable( + member, + member["signature"], + "function", + where, + level=3, + returns_required=True, + show_visibility=False, + ) + if kind == "class": + return render_class(member, 3, where, page_root=False) + declaration = "var" if kind == "variable" else "const" + lines = render_intro( + f"### `{member['name']}`", declaration, member + ) + if kind == "variable": + lines.append(f"类型：{member['type']}") + else: + if member.get("type"): + lines.extend([f"类型：{member['type']}", ""]) + lines.append(f"值：`{render_scalar(member['value'])}`") + return lines + + +def render_unit(unit, where): + lines = render_intro(f"## `{unit['name']}`", "unit", unit) + for index, member in enumerate(unit["members"]): + lines.extend(render_unit_member(member, index, where)) + lines.append("") + while lines and not lines[-1]: + lines.pop() + return lines + + +def render_scalar(value): + if isinstance(value, str): + return value + return json.dumps(value, ensure_ascii=False) + + +def render_declaration(declaration, index): + kind = declaration["kind"] + where = f"declarations[{index}]" + if kind == "function": + return render_top_level_function(declaration, index) + if kind == "class": + return render_class(declaration, 2, where, page_root=True) + return render_unit(declaration, where) + + +def render_page(data): + """Validate and render a whole leaf page.""" + declarations = validate_page(data) + out = [f"# {data['module']}", ""] + for index, declaration in enumerate(declarations): + out.extend(render_declaration(declaration, index)) out.append("") return "\n".join(out).rstrip("\n") + "\n" +def format_markdown(text): + """Format generated Markdown with the repository-pinned Prettier.""" + npx = shutil.which("npx") + if not npx: + die("未找到 Prettier；请先在仓库根目录运行 `npm install`") + + result = subprocess.run( + [ + npx, + "--no-install", + "prettier", + "--config", + str(PRETTIER_CONFIG), + "--parser", + "markdown", + ], + input=text, + capture_output=True, + text=True, + encoding="utf-8", + cwd=REPO_ROOT, + check=False, + ) + if result.returncode != 0: + detail = result.stderr.strip() or "未知错误" + die(f"Prettier 格式化失败：{detail}") + return result.stdout + + def output_path(data, scope): """Build the leaf-page destination from the recording file's relative path.""" relative = data.get("path") @@ -204,10 +732,37 @@ def output_path(data, scope): ) +def atomic_write(path, text): + """Atomically replace path and remove the temporary file on failure.""" + path.parent.mkdir(parents=True, exist_ok=True) + descriptor, temporary_name = tempfile.mkstemp( + prefix=f".{path.name}.", suffix=".tmp", dir=path.parent + ) + temporary_path = Path(temporary_name) + try: + with os.fdopen(descriptor, "w", encoding="utf-8", newline="\n") as handle: + handle.write(text) + handle.flush() + os.fsync(handle.fileno()) + os.replace(temporary_path, path) + finally: + if temporary_path.exists(): + temporary_path.unlink() + + def main(argv=None): if hasattr(sys.stdout, "reconfigure"): sys.stdout.reconfigure(encoding="utf-8") - parser = argparse.ArgumentParser(description="从 YAML/JSON 录入文件生成 TSL 函数文档") + parser = argparse.ArgumentParser( + description="从 YAML/JSON 录入文件生成 TSL API 文档", + add_help=False, + allow_abbrev=False, + ) + parser.add_argument( + "--help", + action="help", + help="显示本帮助并退出（不提供 -h 短选项）", + ) parser.add_argument( "input", metavar="INPUT_FILE", @@ -231,10 +786,9 @@ def main(argv=None): die(f"input not found: {in_path}") data = load_entries(in_path, args.format) - text = render_page(data) + text = format_markdown(render_page(data)) out_path = output_path(data, args.scope) - out_path.parent.mkdir(parents=True, exist_ok=True) - out_path.write_text(text, encoding="utf-8", newline="\n") + atomic_write(out_path, text) print( f"wrote {out_path}", file=sys.stderr, diff --git a/tools/tsl-codegen/scripts/lint.py b/tools/tsl-codegen/scripts/lint.py index 6e8b656c..fefa7b71 100644 --- a/tools/tsl-codegen/scripts/lint.py +++ b/tools/tsl-codegen/scripts/lint.py @@ -1,17 +1,13 @@ #!/usr/bin/env python3 -"""Lint TSL codegen function-doc markdown against the house standard. +"""Lint TSL codegen API Markdown against the house standard. The standard lives in tools/tsl-codegen/STANDARD.md. -Each `## `sig`` / `### `sig`` heading starts one function entry. Rules split -into hard errors (CI-blocking) and soft warnings (style -convergence over the ~12k existing entries). +Each typed H2 starts one top-level declaration; typed H3/H4 headings describe +class or unit members. Rules split into hard errors (CI-blocking) and soft +warnings (style convergence over the ~12k existing entries). -Hard errors: - - missing/empty description (first prose line after the signature) - - missing `返回：类型` - - signature has parameters but the entry has no parameter table - - signature has no parameters but a parameter table is present - - parameter table header is not the fixed 参数 / 类型 / 说明 三列 +Hard errors include incomplete descriptions/metadata/parameter tables and +invalid class/unit API headings, visibility, or child-heading levels. Soft warnings: - optional-parameter wording not starting with `可选。` @@ -29,12 +25,26 @@ import re import sys from pathlib import Path -# Entry heading: `## `sig`` or `### `sig``. Matches the index generator's rule -# so the linter and the tsv agree on what a function entry is. -ENTRY_RE = re.compile(r"^(#{2,3})(?!#)\s+`(.+?)`\s*$") -RETURN_RE = re.compile(r"^返回[:：]") +SCRIPT_DIR = Path(__file__).resolve().parent +if str(SCRIPT_DIR) not in sys.path: + sys.path.insert(0, str(SCRIPT_DIR)) + +from api_markdown import DECLARATION_LINE_RE, iter_api_entries + +RETURN_LINE_RE = re.compile(r"^返回[:：]") +RETURN_RE = re.compile(r"^返回[:：]\s*\S") +TYPE_LINE_RE = re.compile(r"^类型[:：]\s*(.*?)\s*$", re.IGNORECASE) +TYPE_RE = re.compile(r"^类型[:：]\s*\S") +VALUE_LINE_RE = re.compile(r"^值[:：]") +VALUE_RE = re.compile(r"^值[:：]\s*(?:`[^`]+`|[^`\s])") +ACCESS_RE = re.compile(r"^访问[:：]\s*(read|write|read\s*/\s*write)\s*$") +VISIBILITY_RE = re.compile(r"^可见性[:：]\s*`?(public|protected|private)`?\s*$") +BASE_RE = re.compile(r"^父类[:：]") +MODIFIERS_RE = re.compile(r"^修饰符[:：]") TAGS_RE = re.compile(r"^\s*$") FENCE_RE = re.compile(r"^(```|~~~)") +HEADING_RE = re.compile(r"^(#{1,6})(?!#)\s+(.+?)\s*$") +VALUE_HEADING_RE = re.compile(r"^`.+?`\s+取值$") OPTIONAL_HINT_RE = re.compile(r"可选|可省略|省略") # Split a table row on unescaped pipes so `nil\|array` stays one cell. CELL_SPLIT_RE = re.compile(r"(?\n\n" + "声明：function\n\n" "返回示例值。\n\n" + "\n\n" "返回：integer\n", encoding="utf-8", ) @@ -50,6 +51,12 @@ class FunctionIndexTest(unittest.TestCase): result = self.module.main(["--skill-dir", str(self.skill_dir), *args]) return result, stdout.getvalue(), stderr.getvalue() + def write_page(self, relative_path, text): + page = self.codegen_root / relative_path + page.parent.mkdir(parents=True, exist_ok=True) + page.write_text(text, encoding="utf-8") + return page + def test_rebuild_writes_only_tsv(self): result, _, _ = self.run_main() @@ -64,6 +71,294 @@ class FunctionIndexTest(unittest.TestCase): self.assertEqual("数组 列表", row["tags"]) self.assertEqual("返回示例值。", row["summary"]) + def test_declaration_does_not_become_a_missing_description_summary(self): + self.write_page( + "project/missing_description.md", + "# Missing\n\n" + "## `Missing()`\n\n" + "声明：function\n\n" + "返回：nil\n", + ) + + rows = { + row[0]: dict(zip(self.module.HEADER, row)) + for row in self.module.build_rows(self.codegen_root) + } + + self.assertEqual("", rows["Missing"]["summary"]) + + def test_legacy_function_anchor_and_first_eight_columns_are_unchanged(self): + rows = self.module.build_rows(self.codegen_root) + row = dict(zip(self.module.HEADER, rows[0])) + + self.assertEqual( + [ + "name", + "scope", + "module", + "signature", + "page", + "anchor", + "tags", + "summary", + "kind", + "binding", + "visibility", + "owner", + "qualified_name", + ], + self.module.HEADER, + ) + self.assertEqual( + [ + "demo", + "builtin", + "base", + "demo()", + "builtin/base/array.md", + "demo", + "数组 列表", + "返回示例值。", + ], + rows[0][:8], + ) + self.assertEqual("function", row["kind"]) + self.assertEqual("", row["binding"]) + self.assertEqual("", row["visibility"]) + self.assertEqual("", row["owner"]) + self.assertEqual("demo", row["qualified_name"]) + + def test_mixed_page_resets_owner_for_every_h2(self): + self.write_page( + "project/mixed/api.md", + "# Mixed\n\n" + "## `Widget`\n\n声明：class\n\n组件。\n\n" + "### `Open()`\n\n声明：function\n\n打开。\n\n可见性：public\n\n" + "## `Parse()`\n\n声明：function\n\n解析。\n\n返回：Widget\n\n" + "## `Runtime`\n\n声明：unit\n\n接口。\n\n" + "### `Document`\n\n声明：class\n\n文档。\n\n" + "#### `Save()`\n\n声明：function\n\n保存。\n\n可见性：public\n", + ) + + rows = { + (row[0], row[3]): dict(zip(self.module.HEADER, row)) + for row in self.module.build_rows(self.codegen_root) + } + + self.assertEqual("", rows[("Widget", "Widget")]["owner"]) + self.assertEqual("Widget", rows[("Open", "Open()")]["owner"]) + self.assertEqual("", rows[("Parse", "Parse()")]["owner"]) + self.assertEqual("", rows[("Runtime", "Runtime")]["owner"]) + self.assertEqual("Runtime", rows[("Document", "Document")]["owner"]) + self.assertEqual( + "Runtime.Document", rows[("Save", "Save()")]["owner"] + ) + + def test_class_page_maps_members_to_unified_index_columns(self): + self.write_page( + "project/widgets/widget.md", + "# Project / Widgets\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "组件。\n\n" + "\n\n" + "### `Open(path)`\n\n" + "声明：function\n\n" + "打开组件。\n\n" + "可见性：`public`\n\n" + "### `Create()`\n\n" + "声明：class function\n\n" + "创建组件。\n\n" + "可见性：`protected`\n\n" + "### `Count`\n\n" + "声明：static field\n\n" + "组件数量。\n\n" + "可见性：`protected`\n\n" + "类型：integer\n\n" + "### `Items(index)`\n\n" + "声明：property\n\n" + "按索引读取组件。\n\n" + "可见性：`public`\n\n" + "类型：Widget\n\n" + "访问：read\n\n" + "### `DefaultName`\n\n" + "声明：const\n\n" + "默认名称。\n\n" + "可见性：`public`\n\n" + "值：`'widget'`\n\n" + "### `Maximum`\n\n" + "声明：static const\n\n" + "最大数量。\n\n" + "可见性：`protected`\n\n" + "值：`100`\n", + ) + + rows = { + row[self.module.HEADER.index("qualified_name")]: dict( + zip(self.module.HEADER, row) + ) + for row in self.module.build_rows(self.codegen_root) + } + + self.assertEqual( + { + "name": "Widget", + "scope": "project", + "module": "widgets", + "signature": "Widget", + "page": "project/widgets/widget.md", + "anchor": "widget", + "tags": "组件", + "summary": "组件。", + "kind": "class", + "binding": "", + "visibility": "", + "owner": "", + "qualified_name": "Widget", + }, + rows["Widget"], + ) + self.assertEqual("method", rows["Widget.Open"]["kind"]) + self.assertEqual("instance", rows["Widget.Open"]["binding"]) + self.assertEqual("public", rows["Widget.Open"]["visibility"]) + self.assertEqual("Widget", rows["Widget.Open"]["owner"]) + self.assertEqual("openpath", rows["Widget.Open"]["anchor"]) + self.assertEqual("class", rows["Widget.Create"]["binding"]) + self.assertEqual("create", rows["Widget.Create"]["anchor"]) + self.assertEqual("field", rows["Widget.Count"]["kind"]) + self.assertEqual("static", rows["Widget.Count"]["binding"]) + self.assertEqual("count", rows["Widget.Count"]["anchor"]) + self.assertEqual("property", rows["Widget.Items"]["kind"]) + self.assertEqual("instance", rows["Widget.Items"]["binding"]) + self.assertEqual("Items(index)", rows["Widget.Items"]["signature"]) + self.assertEqual("constant", rows["Widget.DefaultName"]["kind"]) + self.assertEqual("instance", rows["Widget.DefaultName"]["binding"]) + self.assertEqual("constant", rows["Widget.Maximum"]["kind"]) + self.assertEqual("static", rows["Widget.Maximum"]["binding"]) + self.assertEqual("maximum", rows["Widget.Maximum"]["anchor"]) + + def test_unit_page_derives_nested_owner_and_public_interface_visibility(self): + self.write_page( + "dotnet/runtime/demo_unit.md", + "# Dotnet / Runtime\n\n" + "## `DemoUnit`\n\n" + "声明：unit\n\n" + "接口。\n\n" + "### `Open()`\n\n" + "声明：function\n\n" + "打开。\n\n" + "返回：Document\n\n" + "### `Current`\n\n" + "声明：var\n\n" + "当前文档。\n\n" + "类型：Document\n\n" + "### `DefaultName`\n\n" + "声明：const\n\n" + "默认名称。\n\n" + "值：`'demo'`\n\n" + "### `Document`\n\n" + "声明：class\n\n" + "文档对象。\n\n" + "#### `Save()`\n\n" + "声明：function\n\n" + "保存。\n\n" + "可见性：`protected`\n", + ) + + rows = { + row[self.module.HEADER.index("qualified_name")]: dict( + zip(self.module.HEADER, row) + ) + for row in self.module.build_rows(self.codegen_root) + } + + self.assertEqual("unit", rows["DemoUnit"]["kind"]) + self.assertEqual("", rows["DemoUnit"]["owner"]) + self.assertEqual("unit", rows["DemoUnit.Open"]["binding"]) + self.assertEqual("public", rows["DemoUnit.Open"]["visibility"]) + self.assertEqual("DemoUnit", rows["DemoUnit.Open"]["owner"]) + self.assertEqual("variable", rows["DemoUnit.Current"]["kind"]) + self.assertEqual("constant", rows["DemoUnit.DefaultName"]["kind"]) + self.assertEqual("unit", rows["DemoUnit.DefaultName"]["binding"]) + self.assertEqual("public", rows["DemoUnit.DefaultName"]["visibility"]) + self.assertEqual("unit", rows["DemoUnit.Document"]["binding"]) + self.assertEqual("public", rows["DemoUnit.Document"]["visibility"]) + self.assertEqual("DemoUnit.Document", rows["DemoUnit.Document.Save"]["owner"]) + self.assertEqual("method", rows["DemoUnit.Document.Save"]["kind"]) + self.assertEqual("instance", rows["DemoUnit.Document.Save"]["binding"]) + self.assertEqual("protected", rows["DemoUnit.Document.Save"]["visibility"]) + + def test_class_overloads_share_qualified_name_but_have_distinct_anchors(self): + self.write_page( + "project/widgets/overloads.md", + "# Project / Widgets\n\n" + "## `Overloads`\n\n" + "声明：class\n\n" + "重载示例。\n\n" + "### `Open(path)`\n\n" + "声明：function\n\n" + "按路径打开。\n\n" + "可见性：`public`\n\n" + "### `Open(mode, flags)`\n\n" + "声明：function\n\n" + "按模式打开。\n\n" + "可见性：`public`\n", + ) + + rows = [ + dict(zip(self.module.HEADER, row)) + for row in self.module.build_rows(self.codegen_root) + if row[0] == "Open" + ] + + self.assertEqual(2, len(rows)) + self.assertEqual({"Overloads.Open"}, {row["qualified_name"] for row in rows}) + self.assertEqual( + {"openpath", "openmodeflags"}, + {row["anchor"] for row in rows}, + ) + self.assertEqual( + 2, + len({f"{row['page']}#{row['anchor']}" for row in rows}), + ) + + def test_top_level_overloads_and_cross_kind_names_get_unique_anchors(self): + page = "project/mixed/overloads.md" + self.write_page( + page, + "# Project / Mixed\n\n" + "## `Open(path)`\n\n声明：function\n\n按路径打开。\n\n" + "返回：nil\n\n" + "## `Open(mode)`\n\n声明：function\n\n按模式打开。\n\n" + "返回：nil\n\n" + "## `Open`\n\n声明：class\n\n打开器。\n\n" + "## `Open`\n\n声明：unit\n\n打开接口。\n", + ) + + rows = [ + dict(zip(self.module.HEADER, row)) + for row in self.module.build_rows(self.codegen_root) + if row[4] == page + ] + by_identity = { + (row["kind"], row["signature"]): row for row in rows + } + + self.assertEqual(4, len(rows)) + self.assertEqual({""}, {row["owner"] for row in rows}) + self.assertEqual({"Open"}, {row["qualified_name"] for row in rows}) + self.assertEqual( + "open", by_identity[("function", "Open(path)")]["anchor"] + ) + self.assertEqual( + "open-1", by_identity[("function", "Open(mode)")]["anchor"] + ) + self.assertEqual("open-2", by_identity[("class", "Open")]["anchor"]) + self.assertEqual("open-3", by_identity[("unit", "Open")]["anchor"]) + self.assertEqual( + 4, len({f"{row['page']}#{row['anchor']}" for row in rows}) + ) + def test_check_does_not_require_index_pages(self): self.data_dir.mkdir(parents=True) rows = self.module.build_rows(self.codegen_root) diff --git a/tools/tsl-codegen/tests/test_convert_tsf.py b/tools/tsl-codegen/tests/test_convert_tsf.py new file mode 100644 index 00000000..265a70ac --- /dev/null +++ b/tools/tsl-codegen/tests/test_convert_tsf.py @@ -0,0 +1,1590 @@ +import importlib.util +import json +import subprocess +import sys +import tempfile +import textwrap +import unittest +from pathlib import Path +from unittest import mock + +SCRIPT = Path(__file__).parents[1] / "scripts" / "convert_tsf.py" + + +def load_script(): + spec = importlib.util.spec_from_file_location( + "tsl_codegen_convert_tsf", SCRIPT + ) + module = importlib.util.module_from_spec(spec) + sys.modules[spec.name] = module + spec.loader.exec_module(module) + return module + + +class ConvertTsfCliTest(unittest.TestCase): + def setUp(self): + self.temp_dir = tempfile.TemporaryDirectory() + self.root = Path(self.temp_dir.name) + self.output = self.root / "entry.json" + + def tearDown(self): + self.temp_dir.cleanup() + + def write_tsf(self, name, source): + path = self.root / f"{name}.tsf" + path.write_text(textwrap.dedent(source).lstrip(), encoding="utf-8") + return path + + def write_nested_tsf(self, directory, name, source): + path = self.root / directory / f"{name}.tsf" + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(textwrap.dedent(source).lstrip(), encoding="utf-8") + return path + + def run_cli( + self, + *inputs, + fmt="json", + output=None, + module="示例 / 函数", + path="base/example", + ): + output = output or self.output + return subprocess.run( + [ + sys.executable, + str(SCRIPT), + *map(str, inputs), + "--format", + fmt, + "--module", + module, + "--path", + path, + "--output", + str(output), + ], + capture_output=True, + text=True, + encoding="utf-8", + ) + + def read_declarations(self): + return json.loads(self.output.read_text(encoding="utf-8"))[ + "declarations" + ] + + def read_declaration(self): + declarations = self.read_declarations() + self.assertEqual(1, len(declarations)) + return declarations[0] + + def test_json_converts_complete_function_document(self): + source = self.write_tsf( + "Normalize", + """ + function Normalize(mode: integer = 0): integer; + begin + /// 按指定模式处理并返回模式值。 + /// 第二行说明。 + /// @tags: 示例 枚举 + /// @param: mode {INTEGER} 处理模式，默认 0 + /// @values: mode + /// 0: 原样返回 + /// 1: 去重 + /// "auto": 自动判断 + /// @returns: INTEGER + /// @example: 使用默认模式 + /// return Normalize(); + /// @output: + /// 0 + /// @example: 指定模式 + /// return Normalize(1); + return mode; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual("", result.stdout) + self.assertIn(f"已写入 {self.output}", result.stderr) + self.assertEqual( + { + "module": "示例 / 函数", + "path": "base/example", + "declarations": [ + { + "kind": "function", + "name": "Normalize", + "signature": "Normalize(mode)", + "desc": "按指定模式处理并返回模式值。\n第二行说明。", + "tags": ["示例", "枚举"], + "params": [ + { + "name": "mode", + "type": "integer", + "optional": True, + "desc": "处理模式，默认 0", + "values": [ + {"value": 0, "desc": "原样返回"}, + {"value": 1, "desc": "去重"}, + {"value": "auto", "desc": "自动判断"}, + ], + } + ], + "returns": "integer", + "examples": [ + { + "desc": "使用默认模式", + "code": "return Normalize();", + "output": "0", + }, + { + "desc": "指定模式", + "code": "return Normalize(1);", + }, + ], + } + ], + }, + json.loads(self.output.read_text(encoding="utf-8")), + ) + + def test_documented_parameter_types_fill_untyped_signature(self): + source = self.write_tsf( + "DocumentedTypes", + """ + function DocumentedTypes(x, y): integer; + begin + /// 补充无类型签名的参数类型 + /// @param: x {integer} 第一个值 + /// @param: y {array of integer} 第二个值 + return x; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual( + [ + {"name": "x", "type": "integer", "desc": "第一个值"}, + { + "name": "y", + "type": "array of integer", + "desc": "第二个值", + }, + ], + self.read_declaration()["params"], + ) + + def test_declared_parameter_types_use_colon_and_semicolon(self): + source = self.write_tsf( + "DeclaredTypes", + """ + function DeclaredTypes(x: integer; y: array of integer): integer; + begin + /// 读取签名中的参数类型 + /// @param: x 第一个值 + /// @param: y 第二个值 + return x; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual( + [ + {"name": "x", "type": "integer", "desc": "第一个值"}, + { + "name": "y", + "type": "array of integer", + "desc": "第二个值", + }, + ], + self.read_declaration()["params"], + ) + + def test_matching_documented_parameter_type_keeps_declared_spelling(self): + source = self.write_tsf( + "MatchingParamType", + """ + function MatchingParamType(value: Array Of Integer): integer; + begin + /// 校验重复记录的参数类型 + /// @param: value {array of integer} 输入值 + return 1; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual( + [ + { + "name": "value", + "type": "Array Of Integer", + "desc": "输入值", + } + ], + self.read_declaration()["params"], + ) + + def test_parameter_type_mismatch_reports_param_line_and_preserves_output(self): + source = self.write_tsf( + "ParamTypeMismatch", + """ + function ParamTypeMismatch(value: integer): integer; + begin + /// 校验参数类型 + /// @param: value {string} 输入值 + return value; + end; + """, + ) + self.output.write_text("原内容\n", encoding="utf-8") + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:4:", result.stderr) + self.assertIn("参数 value 的类型与函数声明不一致", result.stderr) + self.assertIn("注释为 string，声明为 integer", result.stderr) + self.assertEqual("原内容\n", self.output.read_text(encoding="utf-8")) + + def test_parameter_type_braces_must_be_complete_and_nonempty(self): + cases = { + "EmptyParamType": ( + "/// @param: value {} 输入值", + "@param: 参数类型不能为空", + ), + "UnclosedParamType": ( + "/// @param: value {integer 输入值", + "@param: 参数类型缺少右花括号", + ), + "MissingParamDescription": ( + "/// @param: value {integer}", + "@param: 参数说明不能为空", + ), + } + for name, (param_line, expected) in cases.items(): + with self.subTest(name=name): + source = self.write_tsf( + name, + f""" + function {name}(value): integer; + begin + /// 校验参数类型格式 + {param_line} + return value; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:4:", result.stderr) + self.assertIn(expected, result.stderr) + self.assertFalse(self.output.exists()) + + def test_multiple_tsf_files_are_merged_in_input_order(self): + first = self.write_tsf( + "First", + """ + function First(): integer; + begin + /// 第一个函数。 + return 1; + end; + """, + ) + second = self.write_tsf( + "Second", + """ + function Second(): string; + begin + /// 第二个函数。 + return "second"; + end; + """, + ) + + result = self.run_cli(first, second) + + self.assertEqual(0, result.returncode, result.stderr) + data = json.loads(self.output.read_text(encoding="utf-8")) + self.assertEqual( + ["First()", "Second()"], + [ + function["signature"] + for function in data["declarations"] + ], + ) + + def test_mixed_function_class_unit_inputs_preserve_cli_order(self): + cls = self.write_tsf( + "Widget", + """ + type Widget = class + /// 组件。 + end; + """, + ) + function = self.write_tsf( + "OpenWidget", + """ + function OpenWidget(): Widget; + begin + /// 打开组件。 + return nil; + end; + """, + ) + unit = self.write_tsf( + "WidgetRuntime", + """ + unit WidgetRuntime; + /// 运行时接口。 + interface + implementation + end. + """, + ) + + result = self.run_cli(cls, function, unit) + + self.assertEqual(0, result.returncode, result.stderr) + declarations = json.loads( + self.output.read_text(encoding="utf-8") + )["declarations"] + self.assertEqual( + [ + ("class", "Widget"), + ("function", "OpenWidget"), + ("unit", "WidgetRuntime"), + ], + [(item["kind"], item["name"]) for item in declarations], + ) + + def test_yaml_serialization_uses_same_data_structure(self): + try: + import yaml + except ImportError: + self.skipTest("未安装 pyyaml") + source = self.write_tsf( + "Describe", + """ + function Describe(value: string); + begin + /// 返回输入内容。 + /// @param: value 输入内容 + /// @returns: string + return value; + end; + """, + ) + output = self.root / "entry.yaml" + + result = self.run_cli(source, fmt="yaml", output=output) + + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual( + { + "module": "示例 / 函数", + "path": "base/example", + "declarations": [ + { + "kind": "function", + "name": "Describe", + "signature": "Describe(value)", + "desc": "返回输入内容。", + "params": [ + { + "name": "value", + "type": "string", + "desc": "输入内容", + } + ], + "returns": "string", + } + ], + }, + yaml.safe_load(output.read_text(encoding="utf-8")), + ) + + def test_gb18030_encoded_tsf_is_supported(self): + source = self.root / "Legacy.tsf" + source.write_bytes(textwrap.dedent(""" + function Legacy(): string; + begin + /// 返回中文内容。 + return "中文"; + end; + """).lstrip().encode("gb18030")) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + function = self.read_declaration() + self.assertEqual("返回中文内容。", function["desc"]) + + def test_utf8_bom_encoded_tsf_is_supported(self): + source = self.root / "BomDemo.tsf" + source.write_text( + "function BomDemo(): integer;\n" + "begin\n" + " /// 带 BOM 的说明。\n" + " return 1;\n" + "end;\n", + encoding="utf-8-sig", + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + function = self.read_declaration() + self.assertEqual("带 BOM 的说明。", function["desc"]) + + def test_output_must_not_overwrite_an_input_tsf(self): + source = self.write_tsf( + "KeepSource", + """ + function KeepSource(): integer; + begin + /// 保留源文件。 + return 1; + end; + """, + ) + original = source.read_text(encoding="utf-8") + + result = self.run_cli(source, output=source) + + self.assertEqual(1, result.returncode) + self.assertIn("输出文件不能覆盖输入 tsf", result.stderr) + self.assertEqual(original, source.read_text(encoding="utf-8")) + + def test_atomic_write_failure_preserves_existing_output_and_cleans_temp(self): + module = load_script() + self.assertTrue( + hasattr(module, "atomic_write"), "atomic_write is missing" + ) + self.output.write_text("原内容\n", encoding="utf-8") + + with mock.patch.object( + module.os, "replace", side_effect=OSError("replace failed") + ): + with self.assertRaisesRegex(OSError, "replace failed"): + module.atomic_write(self.output, "新内容\n") + + self.assertEqual( + "原内容\n", self.output.read_text(encoding="utf-8") + ) + self.assertEqual( + [], list(self.root.glob(f".{self.output.name}.*.tmp")) + ) + + def test_missing_parameter_metadata_and_return_type_create_a_draft(self): + source = self.write_tsf( + "Draft", + """ + function Draft(value); + begin + /// 尚待完善的函数。 + return value; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + function = self.read_declaration() + self.assertEqual( + [{"name": "value", "type": "", "desc": ""}], + function["params"], + ) + self.assertEqual("", function["returns"]) + + def test_json_draft_completes_fixed_fields_for_every_declaration_kind(self): + function_source = self.write_tsf( + "DraftFunction", + """ + function DraftFunction(); + begin + /// 草稿函数 + return; + end; + """, + ) + class_source = self.write_tsf( + "DraftClass", + """ + type DraftClass = class + class function Build(); // 构建草稿 + property Title read title_; + value_; + const Limit = 1; + end; + """, + ) + unit_source = self.write_tsf( + "DraftUnit", + """ + unit DraftUnit; + interface + function Open(); + var Current; + const Limit = 1; + type Item = class + function Read(); + end; + implementation + end. + """, + ) + + result = self.run_cli(function_source, class_source, unit_source) + + self.assertEqual(0, result.returncode, result.stderr) + function, cls, unit = self.read_declarations() + self.assertEqual([], function.get("params")) + self.assertEqual("", function.get("returns")) + + method, prop, field, constant = cls["members"] + self.assertEqual([], method.get("params")) + self.assertEqual("", method.get("returns")) + self.assertEqual("", prop.get("type")) + self.assertEqual([], prop.get("params")) + self.assertEqual("", field.get("type")) + self.assertEqual("", constant.get("type")) + + unit_function, variable, unit_constant, nested_class = unit["members"] + self.assertEqual([], unit_function.get("params")) + self.assertEqual("", unit_function.get("returns")) + self.assertEqual("", variable.get("type")) + self.assertEqual("", unit_constant.get("type")) + self.assertEqual([], nested_class["members"][0].get("params")) + self.assertEqual("", nested_class["members"][0].get("returns")) + + for item in (function, cls, unit, method, prop, field, constant): + self.assertNotIn("tags", item) + + def test_documented_untyped_parameter_and_return_remain_empty_draft(self): + source = self.write_tsf( + "DocumentedDraft", + """ + function DocumentedDraft(value); + begin + /// 保留没有类型注解的函数 + /// @param: value 输入值 + return value; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + function = self.read_declaration() + self.assertEqual( + [{"name": "value", "type": "", "desc": "输入值"}], + function["params"], + ) + self.assertEqual("", function["returns"]) + + def test_yaml_draft_uses_the_same_complete_fixed_fields(self): + try: + import yaml + except ImportError: + self.skipTest("未安装 pyyaml") + source = self.write_tsf( + "YamlDraft", + """ + type YamlDraft = class + class function Build(); // 构建草稿 + property Title read title_; + const Limit = 1; + end; + """, + ) + output = self.root / "draft.yaml" + + result = self.run_cli(source, fmt="yaml", output=output) + + self.assertEqual(0, result.returncode, result.stderr) + method, prop, constant = yaml.safe_load( + output.read_text(encoding="utf-8") + )["declarations"][0]["members"] + self.assertEqual([], method.get("params")) + self.assertEqual("", method.get("returns")) + self.assertEqual("", prop.get("type")) + self.assertEqual([], prop.get("params")) + self.assertEqual("", constant.get("type")) + + def test_help_is_written_in_chinese(self): + result = subprocess.run( + [sys.executable, str(SCRIPT), "--help"], + capture_output=True, + text=True, + encoding="utf-8", + ) + + self.assertEqual(0, result.returncode, result.stderr) + self.assertIn("用法：", result.stdout) + self.assertIn("位置参数：", result.stdout) + self.assertIn("选项：", result.stdout) + self.assertIn("输出格式", result.stdout) + self.assertIn("function、class 和 unit", result.stdout) + self.assertIn("declarations", result.stdout) + self.assertNotIn("usage:", result.stdout) + self.assertNotIn("positional arguments:", result.stdout) + self.assertNotIn("options:", result.stdout) + + def test_return_type_mismatch_reports_directive_line_and_preserves_output(self): + source = self.write_tsf( + "Mismatch", + """ + function Mismatch(): integer; + begin + /// 返回一个值。 + /// @returns: string + return 1; + end; + """, + ) + self.output.write_text("原内容\n", encoding="utf-8") + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:4:", result.stderr) + self.assertIn("返回类型与函数声明不一致", result.stderr) + self.assertIn("注释为 string，声明为 integer", result.stderr) + self.assertEqual("原内容\n", self.output.read_text(encoding="utf-8")) + + def test_document_block_must_be_first_content_after_begin(self): + source = self.write_tsf( + "LateDoc", + """ + function LateDoc(): integer; + begin + value := 1; + /// 这个文档块出现得太晚。 + return value; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:3:", result.stderr) + self.assertIn("begin 后第一段内容必须是 /// 文档块", result.stderr) + self.assertFalse(self.output.exists()) + + def test_duplicate_enum_value_is_rejected(self): + source = self.write_tsf( + "DuplicateValue", + """ + function DuplicateValue(mode: integer): integer; + begin + /// 检查模式。 + /// @param: mode 模式 + /// @values: mode + /// 1: 第一项 + /// 1: 重复项 + return mode; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:7:", result.stderr) + self.assertIn("枚举值重复", result.stderr) + + def test_directives_after_examples_are_rejected(self): + source = self.write_tsf( + "BadOrder", + """ + function BadOrder(value: integer): integer; + begin + /// 返回输入值。 + /// @example: 基本用法 + /// return BadOrder(1); + /// @param: value 输入值 + return value; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:6:", result.stderr) + self.assertIn("示例组之后不能再写函数级指令", result.stderr) + + def test_class_converts_only_first_matching_public_class(self): + source = self.write_tsf( + "Widget", + """ + type Widget = class(BaseOne, BaseTwo) + /// 对外组件。 + /// @tags: 组件 示例 + public + /// 创建组件。 + /// @param: name 组件名称 + function create(name: string); overload; + /// 从配置创建。 + /// @param: path 配置路径 + class function FromConfig(path: string): Widget; + /// 组件标题。 + property Title: string read title_ write title_; + protected + /// 下一个编号。 + static next_id_: integer; + private + /// 内部状态。 + hidden_: string; + end; + + type WidgetHelper = class + public + function Hidden(): integer; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual( + { + "kind": "class", + "name": "Widget", + "desc": "对外组件。", + "tags": ["组件", "示例"], + "bases": ["BaseOne", "BaseTwo"], + "members": [ + { + "kind": "method", + "name": "create", + "visibility": "public", + "binding": "instance", + "signature": "create(name)", + "desc": "创建组件。", + "params": [ + { + "name": "name", + "type": "string", + "desc": "组件名称", + } + ], + "returns": "", + "modifiers": ["overload"], + }, + { + "kind": "method", + "name": "FromConfig", + "visibility": "public", + "binding": "class", + "signature": "FromConfig(path)", + "desc": "从配置创建。", + "params": [ + { + "name": "path", + "type": "string", + "desc": "配置路径", + } + ], + "returns": "Widget", + }, + { + "kind": "property", + "name": "Title", + "visibility": "public", + "desc": "组件标题。", + "type": "string", + "params": [], + "access": "readwrite", + }, + { + "kind": "field", + "name": "next_id_", + "visibility": "protected", + "desc": "下一个编号。", + "type": "integer", + "static": True, + }, + ], + }, + self.read_declaration(), + ) + + def test_class_out_of_class_implementation_does_not_duplicate_member(self): + source = self.write_tsf( + "Widget", + """ + type Widget = class + /// 对外组件。 + public + /// 保存组件。 + /// @param: path 保存路径 + function Save(path: string): boolean; + end; + + function Widget.Save(path: string): boolean; + begin + return true; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + members = self.read_declaration()["members"] + self.assertEqual(["Save"], [member["name"] for member in members]) + + def test_class_method_trailing_comment_fills_missing_description(self): + source = self.write_tsf( + "InlineComments", + """ + type InlineComments = class + class function Plain(value); // 普通行尾说明 + class function Overloaded(value); overload; // 重载行尾说明 + /// 正式文档说明 + class function Documented(value); // 不覆盖正式文档 + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual( + ["普通行尾说明", "重载行尾说明", "正式文档说明"], + [member["desc"] for member in self.read_declaration()["members"]], + ) + + def test_class_parameterized_property_preserves_params_and_values(self): + source = self.write_tsf( + "Indexed", + """ + type Indexed = class + /// 索引集合。 + public + /// 按索引读取项目。 + /// @param: index 项目索引 + /// @values: index + /// 0: 第一项 + property Items(index: integer): string read GetItem write SetItem; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + member = self.read_declaration()["members"][0] + self.assertEqual("property", member["kind"]) + self.assertEqual("string", member["type"]) + self.assertEqual("readwrite", member["access"]) + self.assertEqual( + [ + { + "name": "index", + "type": "integer", + "desc": "项目索引", + "values": [{"value": 0, "desc": "第一项"}], + } + ], + member["params"], + ) + + def test_class_instance_and_static_constants_are_preserved(self): + source = self.write_tsf( + "Constants", + """ + type Constants = class + /// 常量集合。 + public + /// 默认数量。 + const DefaultSize: integer = 0; + /// 是否禁用。 + static const Disabled = false; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + members = self.read_declaration()["members"] + self.assertEqual(["DefaultSize", "Disabled"], [item["name"] for item in members]) + self.assertEqual("integer", members[0]["type"]) + self.assertEqual("0", members[0]["value"]) + self.assertNotIn("static", members[0]) + self.assertEqual("false", members[1]["value"]) + self.assertIs(members[1]["static"], True) + + def test_class_virtual_and_override_modifiers_preserve_source_order(self): + source = self.write_tsf( + "Modifiers", + """ + type Modifiers = class + /// 修饰符示例。 + public + /// 检查状态。 + function Inspect(): integer; virtual; overload; + /// 重置状态。 + function Reset(); override; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + members = self.read_declaration()["members"] + self.assertEqual(["virtual", "overload"], members[0]["modifiers"]) + self.assertEqual(["override"], members[1]["modifiers"]) + + def test_class_inline_method_is_collected_once(self): + source = self.write_tsf( + "InlineClass", + """ + type InlineClass = class + /// 内联类。 + public + /// 读取当前值。 + function Value(): integer; + begin + return 1; + end; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + members = self.read_declaration()["members"] + self.assertEqual(["Value"], [member["name"] for member in members]) + + def test_class_rejects_unsupported_public_declaration_at_exact_line(self): + source = self.write_tsf( + "UnsupportedClass", + """ + type UnsupportedClass = class + public + type Alias = integer; + end; + """, + ) + self.output.write_text("原内容\n", encoding="utf-8") + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:3:", result.stderr) + self.assertIn("class 不支持成员声明：type", result.stderr) + self.assertEqual("原内容\n", self.output.read_text(encoding="utf-8")) + + def test_missing_class_and_unit_member_types_remain_explicit_drafts(self): + cls_source = self.write_tsf( + "TypeDraft", + """ + type TypeDraft = class + /// 类型草稿。 + public + /// 未知字段类型。 + value_; + /// 未知属性类型。 + property Value read value_; + end; + """, + ) + + class_result = self.run_cli(cls_source) + + self.assertEqual(0, class_result.returncode, class_result.stderr) + class_members = self.read_declaration()["members"] + self.assertEqual("", class_members[0]["type"]) + self.assertEqual("", class_members[1]["type"]) + self.assertEqual([], class_members[1]["params"]) + + unit_source = self.write_tsf( + "UnitTypeDraft", + """ + unit UnitTypeDraft; + /// 类型草稿接口。 + interface + /// 未知变量类型。 + var Current; + implementation + end. + """, + ) + + unit_result = self.run_cli(unit_source) + + self.assertEqual(0, unit_result.returncode, unit_result.stderr) + unit_member = self.read_declaration()["members"][0] + self.assertEqual("", unit_member["type"]) + + def test_unit_collects_only_interface_members_in_source_order(self): + source = self.write_tsf( + "DemoUnit", + """ + unit DemoUnit; + /// 运行时接口。 + /// @tags: 运行时 文档 + interface + uses RuntimeSupport; + /// 默认数量。 + const DefaultSize = 100; + /// 当前名称。 + var CurrentName: string; + /// 打开对象。 + /// @param: path 文件路径 + function Open(path: string): Document; + type Document = class + /// 文档对象。 + public + /// 保存文档。 + function Save(): boolean; + end; + implementation + /// 这个实现文档不属于 API。 + var hidden_: integer; + type HiddenDocument = class + public + function Hidden(): integer; + end; + const HiddenValue = 1; + function Open(path: string): Document; + begin + return nil; + end; + initialization + /// 生命周期文档也不属于 API。 + hidden_ := 1; + finalization + hidden_ := 0; + end. + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + unit = self.read_declaration() + self.assertEqual("unit", unit["kind"]) + self.assertEqual("DemoUnit", unit["name"]) + self.assertEqual("运行时接口。", unit["desc"]) + self.assertEqual(["运行时", "文档"], unit["tags"]) + self.assertEqual( + ["DefaultSize", "CurrentName", "Open", "Document"], + [member["name"] for member in unit["members"]], + ) + self.assertEqual( + { + "kind": "constant", + "name": "DefaultSize", + "desc": "默认数量。", + "type": "", + "value": "100", + }, + unit["members"][0], + ) + self.assertEqual( + { + "kind": "variable", + "name": "CurrentName", + "desc": "当前名称。", + "type": "string", + }, + unit["members"][1], + ) + self.assertEqual("Document", unit["members"][2]["returns"]) + self.assertEqual("class", unit["members"][3]["kind"]) + self.assertEqual("Save", unit["members"][3]["members"][0]["name"]) + serialized = json.dumps(unit, ensure_ascii=False) + self.assertNotIn("hidden_", serialized) + self.assertNotIn("HiddenDocument", serialized) + self.assertNotIn("HiddenValue", serialized) + self.assertNotIn("生命周期文档", serialized) + + def test_unit_filename_must_match_name_at_declaration_line(self): + source = self.write_tsf( + "Wrong", + "unit Right;\ninterface\nimplementation\nend.\n", + ) + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:1:", result.stderr) + self.assertIn("unit 名称必须与文件名一致", result.stderr) + + def test_unit_preserves_multiple_interface_classes_in_source_order(self): + source = self.write_tsf( + "ClassUnit", + """ + unit ClassUnit; + /// 类接口。 + interface + type First = class + /// 第一个类。 + end; + type Second = class + /// 第二个类。 + end; + implementation + end. + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + members = self.read_declaration()["members"] + self.assertEqual(["First", "Second"], [member["name"] for member in members]) + + def test_unit_unbound_document_reports_exact_line(self): + source = self.write_tsf( + "UnboundUnit", + """ + unit UnboundUnit; + interface + /// 无法绑定的接口文档。 + // 普通注释切断绑定。 + var Current: integer; + implementation + end. + """, + ) + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:3:", result.stderr) + self.assertIn("unit interface 文档块无法绑定", result.stderr) + + def test_unit_unbound_header_document_reports_exact_line(self): + source = self.write_tsf( + "UnboundHeaderUnit", + """ + unit UnboundHeaderUnit; + /// 无法绑定的 unit 文档。 + // 普通注释切断绑定。 + interface + implementation + end. + """, + ) + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:2:", result.stderr) + self.assertIn("unit interface 文档块无法绑定", result.stderr) + + def test_unit_end_dot_must_be_the_terminal_tokens(self): + source = self.write_tsf( + "TrailingUnit", + """ + unit TrailingUnit; + interface + implementation + end. + function Trailing(): integer; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:5:", result.stderr) + self.assertIn("end. 必须结束整个文件", result.stderr) + + def test_class_missing_docs_default_visibility_and_private_filter_create_draft(self): + source = self.write_tsf( + "DraftClass", + """ + type DraftClass = class + value_: integer; + private + /// 不应进入草稿。 + hidden_: string; + protected + /// 检查状态。 + function Inspect(); + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + cls = self.read_declaration() + self.assertEqual("", cls["desc"]) + self.assertEqual(["value_", "Inspect"], [item["name"] for item in cls["members"]]) + self.assertEqual("public", cls["members"][0]["visibility"]) + self.assertEqual("", cls["members"][0]["desc"]) + self.assertEqual("protected", cls["members"][1]["visibility"]) + + def test_class_private_multi_name_fields_and_constants_are_ignored(self): + source = self.write_tsf( + "PrivateMulti", + """ + type PrivateMulti = class + /// 私有声明示例。 + private + /// 多名称私有字段。 + left_, right_: integer; + /// 多名称私有常量。 + const First, Second = 1; + public + /// 对外值。 + value_: integer; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + members = self.read_declaration()["members"] + self.assertEqual(["value_"], [member["name"] for member in members]) + + def test_class_structure_errors_preserve_existing_output(self): + cases = { + "WrongName": ( + "type Actual = class\nend;\n", + 1, + "对外 class 名称必须与文件名一致", + ), + "StaticMethod": ( + "type StaticMethod = class\npublic\nstatic function Bad();\nend;\n", + 3, + "不存在 static function", + ), + "ManyFields": ( + "type ManyFields = class\npublic\nleft_, right_: integer;\nend;\n", + 3, + "一项一条声明", + ), + } + for name, (text, line, expected) in cases.items(): + with self.subTest(name=name): + source = self.write_tsf(name, text) + self.output.write_text("原内容\n", encoding="utf-8") + result = self.run_cli(source) + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:{line}:", result.stderr) + self.assertIn(expected, result.stderr) + self.assertEqual("原内容\n", self.output.read_text(encoding="utf-8")) + + def test_class_and_unit_constants_reject_multiple_names_at_declaration_line(self): + cases = { + "ManyConstants": ( + "type ManyConstants = class\npublic\nconst Left, Right = 1;\nend;\n", + 3, + ), + "ManyUnitConstants": ( + "unit ManyUnitConstants;\n" + "interface\n" + "const Left, Right = 1;\n" + "implementation\n" + "end.\n", + 3, + ), + } + for name, (text, line) in cases.items(): + with self.subTest(name=name): + source = self.write_tsf(name, text) + self.output.write_text("原内容\n", encoding="utf-8") + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:{line}:", result.stderr) + self.assertIn("对外常量必须一项一条声明", result.stderr) + self.assertEqual("原内容\n", self.output.read_text(encoding="utf-8")) + + def test_class_rejects_procedure_at_declaration_line(self): + source = self.write_tsf( + "ProcedureClass", + """ + type ProcedureClass = class + public + procedure Open(); + end; + """, + ) + self.output.write_text("原内容\n", encoding="utf-8") + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:3:", result.stderr) + self.assertIn("class 暂不支持 procedure", result.stderr) + self.assertEqual("原内容\n", self.output.read_text(encoding="utf-8")) + + def test_class_unbound_document_reports_its_line(self): + source = self.write_tsf( + "UnboundDoc", + """ + type UnboundDoc = class + /// 类描述。 + public + /// 无法绑定的成员描述。 + // 普通注释切断绑定。 + value_: integer; + end; + """, + ) + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:4:", result.stderr) + self.assertIn("文档块无法绑定", result.stderr) + + def test_unit_rejects_shorthand_procedure_and_non_class_type(self): + cases = { + "ShortUnit": ( + "unit ShortUnit;\nfunction Open(): integer;\nbegin return 1; end;\nend.\n", + 1, + "显式 interface", + ), + "ProcedureUnit": ( + "unit ProcedureUnit;\ninterface\nprocedure Open();\nimplementation\nend.\n", + 3, + "暂不支持 procedure", + ), + "AliasUnit": ( + "unit AliasUnit;\ninterface\ntype Size = integer;\nimplementation\nend.\n", + 3, + "只支持 class type", + ), + } + for name, (text, line, expected) in cases.items(): + with self.subTest(name=name): + source = self.write_tsf(name, text) + result = self.run_cli(source) + self.assertEqual(1, result.returncode) + self.assertIn(f"{source}:{line}:", result.stderr) + self.assertIn(expected, result.stderr) + + def test_unit_var_and_const_sections_keep_each_declaration(self): + source = self.write_tsf( + "StateUnit", + """ + unit StateUnit; + /// 状态接口。 + interface + var + /// 当前名称。 + CurrentName: string; + /// 当前编号。 + CurrentId: integer; + const + /// 默认名称。 + DefaultName = "demo"; + /// 默认编号。 + DefaultId = 1; + implementation + end. + """, + ) + + result = self.run_cli(source) + + self.assertEqual(0, result.returncode, result.stderr) + members = self.read_declaration()["members"] + self.assertEqual( + ["CurrentName", "CurrentId", "DefaultName", "DefaultId"], + [member["name"] for member in members], + ) + self.assertEqual( + ["当前名称。", "当前编号。", "默认名称。", "默认编号。"], + [member["desc"] for member in members], + ) + + def test_function_overloads_are_preserved_in_input_order(self): + first = self.write_nested_tsf( + "first", + "Parse", + """ + function Parse(path: string): integer; + begin + /// 按路径解析。 + /// @param: path 文件路径 + return 1; + end; + """, + ) + second = self.write_nested_tsf( + "second", + "Parse", + """ + function Parse(mode: integer): integer; + begin + /// 按模式解析。 + /// @param: mode 解析模式 + return 1; + end; + """, + ) + + result = self.run_cli(first, second) + + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual( + ["Parse(path)", "Parse(mode)"], + [item["signature"] for item in self.read_declarations()], + ) + + def test_duplicate_function_signature_preserves_existing_output(self): + first = self.write_nested_tsf( + "first", + "Parse", + """ + function Parse(value: string): integer; + begin + /// 解析值。 + /// @param: value 输入值 + return 1; + end; + """, + ) + second = self.write_nested_tsf( + "second", + "parse", + """ + function parse(value: string): integer; + begin + /// 再次解析值。 + /// @param: value 输入值 + return 1; + end; + """, + ) + self.output.write_text("原内容\n", encoding="utf-8") + + result = self.run_cli(first, second) + + self.assertEqual(1, result.returncode) + self.assertIn("重复 function signature", result.stderr) + self.assertEqual("原内容\n", self.output.read_text(encoding="utf-8")) + + def test_duplicate_class_and_unit_names_are_rejected_case_insensitively(self): + cases = ( + ( + self.write_nested_tsf( + "class-a", "Widget", "type Widget = class\nend;\n" + ), + self.write_nested_tsf( + "class-b", "widget", "type widget = class\nend;\n" + ), + "重复 class", + ), + ( + self.write_nested_tsf( + "unit-a", + "Runtime", + "unit Runtime;\ninterface\nimplementation\nend.\n", + ), + self.write_nested_tsf( + "unit-b", + "runtime", + "unit runtime;\ninterface\nimplementation\nend.\n", + ), + "重复 unit", + ), + ) + for first, second, expected in cases: + with self.subTest(expected=expected): + self.output.write_text("原内容\n", encoding="utf-8") + + result = self.run_cli(first, second) + + self.assertEqual(1, result.returncode) + self.assertIn(expected, result.stderr) + self.assertEqual( + "原内容\n", self.output.read_text(encoding="utf-8") + ) + + def test_same_name_across_kinds_is_allowed(self): + function = self.write_nested_tsf( + "function", + "Widget", + """ + function Widget(): integer; + begin + /// 同名函数。 + return 1; + end; + """, + ) + cls = self.write_nested_tsf( + "class", "Widget", "type Widget = class\nend;\n" + ) + + result = self.run_cli(function, cls) + + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual( + ["function", "class"], + [item["kind"] for item in self.read_declarations()], + ) + + def test_multiple_classes_and_units_are_allowed(self): + inputs = ( + self.write_tsf("FirstClass", "type FirstClass = class\nend;\n"), + self.write_tsf("SecondClass", "type SecondClass = class\nend;\n"), + self.write_tsf( + "FirstUnit", + "unit FirstUnit;\ninterface\nimplementation\nend.\n", + ), + self.write_tsf( + "SecondUnit", + "unit SecondUnit;\ninterface\nimplementation\nend.\n", + ), + ) + + result = self.run_cli(*inputs) + + self.assertEqual(0, result.returncode, result.stderr) + self.assertEqual( + ["FirstClass", "SecondClass", "FirstUnit", "SecondUnit"], + [item["name"] for item in self.read_declarations()], + ) + + def test_top_level_procedure_remains_unsupported(self): + source = self.write_tsf( + "ProcedureDemo", + "procedure ProcedureDemo();\nbegin\nend;\n", + ) + + result = self.run_cli(source) + + self.assertEqual(1, result.returncode) + self.assertIn("目前支持独立顶层 function、class 和完整 unit", result.stderr) + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/tsl-codegen/tests/test_generate.py b/tools/tsl-codegen/tests/test_generate.py index aa80db45..7c76daed 100644 --- a/tools/tsl-codegen/tests/test_generate.py +++ b/tools/tsl-codegen/tests/test_generate.py @@ -1,12 +1,23 @@ +import importlib.util import json +import os +import shutil import subprocess import sys import tempfile import unittest from pathlib import Path - +from unittest import mock SCRIPT = Path(__file__).parents[1] / "scripts" / "generate.py" +REPO_ROOT = Path(__file__).resolve().parents[3] + + +def load_script(): + spec = importlib.util.spec_from_file_location("tsl_codegen_generate", SCRIPT) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + return module class DocGenCliTest(unittest.TestCase): @@ -19,8 +30,10 @@ class DocGenCliTest(unittest.TestCase): { "module": "项目 / 示例", "path": "base/my_functions", - "functions": [ + "declarations": [ { + "kind": "function", + "name": "demo", "signature": "demo()", "desc": "示例函数。", "returns": "nil", @@ -35,15 +48,60 @@ class DocGenCliTest(unittest.TestCase): def tearDown(self): self.temp_dir.cleanup() - def run_cli(self, *args): + def run_cli(self, *args, env=None): return subprocess.run( [sys.executable, str(SCRIPT), str(self.input), *args], capture_output=True, text=True, encoding="utf-8", cwd=self.root, + env=env, ) + def write_input(self, data): + self.input.write_text( + json.dumps(data, ensure_ascii=False), + encoding="utf-8", + ) + + def generated(self, relative, scope="project"): + return ( + self.root + / "skills" + / "tsl-api-reference" + / "references" + / "codegen" + / scope + / f"{relative}.md" + ) + + def write_class_member_input(self, member, relative): + self.write_input( + { + "module": "示例 / 类", + "path": relative, + "declarations": [ + { + "kind": "class", + "name": "StrictClass", + "desc": "严格类。", + "members": [member], + } + ], + } + ) + + def assert_rejected_without_overwrite(self, relative, expected): + output = self.generated(relative) + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text("原内容\n", encoding="utf-8") + + result = self.run_cli() + + self.assertEqual(1, result.returncode) + self.assertIn(expected, result.stderr) + self.assertEqual("原内容\n", output.read_text(encoding="utf-8")) + def test_default_scope_writes_configured_path_under_project(self): result = self.run_cli() output = ( @@ -58,7 +116,9 @@ class DocGenCliTest(unittest.TestCase): ) self.assertEqual(result.returncode, 0, result.stderr) self.assertTrue(output.is_file()) - self.assertTrue(output.read_text(encoding="utf-8").startswith("# 项目 / 示例\n")) + self.assertTrue( + output.read_text(encoding="utf-8").startswith("# 项目 / 示例\n") + ) def test_custom_scope_changes_first_destination_directory(self): result = self.run_cli("--scope", "my-project") @@ -80,6 +140,1094 @@ class DocGenCliTest(unittest.TestCase): self.assertNotEqual(result.returncode, 0) self.assertIn("unrecognized arguments: --output", result.stderr) + def test_generated_markdown_is_formatted_by_repo_prettier(self): + self.input.write_text( + json.dumps( + { + "module": "项目 / 示例", + "path": "base/formatted", + "declarations": [ + { + "kind": "function", + "name": "demo", + "signature": "demo(short, long_name)", + "desc": "示例函数。", + "params": [ + { + "name": "short", + "type": "integer", + "desc": "短说明", + }, + { + "name": "long_name", + "type": "very_long_type", + "desc": "这是较长的说明", + }, + ], + "returns": "integer", + } + ], + }, + ensure_ascii=False, + ), + encoding="utf-8", + ) + output = ( + self.root + / "skills" + / "tsl-api-reference" + / "references" + / "codegen" + / "project" + / "base" + / "formatted.md" + ) + + result = self.run_cli() + self.assertEqual(0, result.returncode, result.stderr) + generated = output.read_text(encoding="utf-8") + prettier = subprocess.run( + [ + shutil.which("npx"), + "--no-install", + "prettier", + "--config", + str(REPO_ROOT / ".prettierrc.json"), + "--parser", + "markdown", + ], + input=generated, + capture_output=True, + text=True, + encoding="utf-8", + cwd=REPO_ROOT, + ) + + self.assertEqual(0, prettier.returncode, prettier.stderr) + self.assertEqual(prettier.stdout, generated) + + def test_missing_prettier_fails_without_writing_markdown(self): + output = ( + self.root + / "skills" + / "tsl-api-reference" + / "references" + / "codegen" + / "project" + / "base" + / "my_functions.md" + ) + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text("原内容\n", encoding="utf-8") + env = os.environ.copy() + env["PATH"] = "" + + result = self.run_cli(env=env) + + self.assertEqual(1, result.returncode) + self.assertIn("未找到 Prettier", result.stderr) + self.assertEqual("原内容\n", output.read_text(encoding="utf-8")) + + def test_prettier_process_failure_preserves_existing_markdown(self): + output = self.generated("base/my_functions") + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text("原内容\n", encoding="utf-8") + bin_dir = self.root / "bin" + bin_dir.mkdir() + fake_npx = bin_dir / "npx" + fake_npx.write_text( + "#!/bin/sh\necho formatter-failed >&2\nexit 9\n", + encoding="utf-8", + ) + fake_npx.chmod(0o755) + env = os.environ.copy() + env["PATH"] = str(bin_dir) + + result = self.run_cli(env=env) + + self.assertEqual(1, result.returncode) + self.assertIn("Prettier 格式化失败", result.stderr) + self.assertIn("formatter-failed", result.stderr) + self.assertEqual("原内容\n", output.read_text(encoding="utf-8")) + + def test_mixed_declarations_render_typed_h2_in_input_order(self): + self.write_input( + { + "module": "示例 / 混合", + "path": "base/mixed", + "declarations": [ + { + "kind": "class", + "name": "Widget", + "desc": "组件。", + "members": [], + }, + { + "kind": "function", + "name": "OpenWidget", + "signature": "OpenWidget()", + "desc": "打开组件。", + "returns": "Widget", + }, + { + "kind": "unit", + "name": "WidgetRuntime", + "desc": "运行时接口。", + "members": [], + }, + ], + } + ) + + result = self.run_cli() + + self.assertEqual(0, result.returncode, result.stderr) + text = self.generated("base/mixed").read_text(encoding="utf-8") + markers = [ + "## `Widget`\n\n声明：class\n\n组件。", + "## `OpenWidget()`\n\n声明：function\n\n打开组件。", + "## `WidgetRuntime`\n\n声明：unit\n\n运行时接口。", + ] + positions = [text.index(marker) for marker in markers] + self.assertEqual(sorted(positions), positions) + + def test_class_page_renders_binding_visibility_and_member_order(self): + self.write_input( + { + "module": "示例 / 类", + "path": "base/widget", + "declarations": [ + { + "kind": "class", + "name": "Widget", + "desc": "表示组件。", + "tags": ["组件", "示例"], + "bases": ["BaseWidget"], + "members": [ + { + "kind": "method", + "name": "Close", + "visibility": "public", + "binding": "instance", + "signature": "Close()", + "desc": "关闭组件。", + }, + { + "kind": "method", + "name": "Create", + "visibility": "protected", + "binding": "class", + "signature": "Create(name)", + "desc": "创建组件。", + "params": [ + { + "name": "name", + "type": "string", + "desc": "组件名称", + } + ], + "returns": "Widget", + "modifiers": ["overload"], + }, + { + "kind": "property", + "name": "Title", + "visibility": "public", + "desc": "组件标题。", + "type": "string", + "access": "readwrite", + }, + { + "kind": "field", + "name": "Count", + "visibility": "protected", + "desc": "组件数量。", + "type": "integer", + "static": True, + }, + { + "kind": "constant", + "name": "DefaultName", + "visibility": "public", + "desc": "默认名称。", + "value": "'widget'", + }, + ], + } + ], + } + ) + + result = self.run_cli() + + self.assertEqual(0, result.returncode, result.stderr) + text = self.generated("base/widget").read_text(encoding="utf-8") + expected = [ + "## `Widget`", + "声明：class", + "父类：`BaseWidget`", + "### `Close()`\n\n声明：function", + "### `Create(name)`\n\n声明：class function", + "### `Title`\n\n声明：property", + "### `Count`\n\n声明：static field", + "### `DefaultName`\n\n声明：const", + ] + positions = [text.index(fragment) for fragment in expected] + self.assertEqual(sorted(positions), positions) + self.assertIn("可见性：`protected`", text) + self.assertIn("修饰符：`overload`", text) + self.assertIn("访问：read / write", text) + self.assertNotIn("static function", text) + self.assertEqual( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "\n\n" + "父类：`BaseWidget`\n\n" + "### `Close()`\n\n" + "声明：function\n\n" + "关闭组件。\n\n" + "可见性：`public`\n\n" + "### `Create(name)`\n\n" + "声明：class function\n\n" + "创建组件。\n\n" + "可见性：`protected`\n\n" + "修饰符：`overload`\n\n" + "| 参数 | 类型 | 说明 |\n" + "| ------ | ------ | -------- |\n" + "| `name` | string | 组件名称 |\n\n" + "返回：Widget\n\n" + "### `Title`\n\n" + "声明：property\n\n" + "组件标题。\n\n" + "可见性：`public`\n\n" + "类型：string\n\n" + "访问：read / write\n\n" + "### `Count`\n\n" + "声明：static field\n\n" + "组件数量。\n\n" + "可见性：`protected`\n\n" + "类型：integer\n\n" + "### `DefaultName`\n\n" + "声明：const\n\n" + "默认名称。\n\n" + "可见性：`public`\n\n" + "值：`'widget'`\n", + text, + ) + + def test_unit_page_renders_interface_class_members_at_h4(self): + self.write_input( + { + "module": "示例 / Unit", + "path": "base/demo_unit", + "declarations": [ + { + "kind": "unit", + "name": "DemoUnit", + "desc": "提供文档能力。", + "members": [ + { + "kind": "constant", + "name": "DefaultSize", + "desc": "默认大小。", + "value": 100, + }, + { + "kind": "variable", + "name": "CurrentDocument", + "desc": "当前文档。", + "type": "Document", + }, + { + "kind": "function", + "name": "OpenDocument", + "signature": "OpenDocument(path)", + "desc": "打开文档。", + "params": [ + { + "name": "path", + "type": "string", + "desc": "文档路径", + } + ], + "returns": "Document", + }, + { + "kind": "class", + "name": "Document", + "desc": "文档对象。", + "bases": ["BaseDocument"], + "members": [ + { + "kind": "method", + "name": "Save", + "visibility": "public", + "binding": "instance", + "signature": "Save()", + "desc": "保存文档。", + "returns": "boolean", + } + ], + }, + ], + } + ], + } + ) + + result = self.run_cli() + + self.assertEqual(0, result.returncode, result.stderr) + text = self.generated("base/demo_unit").read_text(encoding="utf-8") + self.assertIn("## `DemoUnit`", text) + self.assertIn("声明：unit", text) + self.assertIn("### `DefaultSize`\n\n声明：const", text) + self.assertIn("值：`100`", text) + self.assertIn("### `CurrentDocument`\n\n声明：var", text) + self.assertIn("### `OpenDocument(path)`\n\n声明：function", text) + self.assertIn("### `Document`\n\n声明：class", text) + self.assertIn("父类：`BaseDocument`", text) + self.assertIn("#### `Save()`\n\n声明：function", text) + self.assertNotIn("可见性：`public`\n\n### `Document`", text) + self.assertEqual( + "# 示例 / Unit\n\n" + "## `DemoUnit`\n\n" + "声明：unit\n\n" + "提供文档能力。\n\n" + "### `DefaultSize`\n\n" + "声明：const\n\n" + "默认大小。\n\n" + "值：`100`\n\n" + "### `CurrentDocument`\n\n" + "声明：var\n\n" + "当前文档。\n\n" + "类型：Document\n\n" + "### `OpenDocument(path)`\n\n" + "声明：function\n\n" + "打开文档。\n\n" + "| 参数 | 类型 | 说明 |\n" + "| ------ | ------ | -------- |\n" + "| `path` | string | 文档路径 |\n\n" + "返回：Document\n\n" + "### `Document`\n\n" + "声明：class\n\n" + "文档对象。\n\n" + "父类：`BaseDocument`\n\n" + "#### `Save()`\n\n" + "声明：function\n\n" + "保存文档。\n\n" + "可见性：`public`\n\n" + "返回：boolean\n", + text, + ) + + def test_class_method_empty_returns_is_treated_as_omitted(self): + self.write_input( + { + "module": "示例 / 类", + "path": "base/no_return", + "declarations": [ + { + "kind": "class", + "name": "NoReturn", + "desc": "无返回类。", + "members": [ + { + "kind": "method", + "name": "Close", + "visibility": "public", + "binding": "instance", + "signature": "Close()", + "desc": "关闭。", + "returns": "", + } + ], + } + ], + } + ) + class_result = self.run_cli() + self.assertEqual(0, class_result.returncode, class_result.stderr) + class_text = self.generated("base/no_return").read_text(encoding="utf-8") + self.assertNotIn("返回：", class_text) + + def test_unit_function_requires_returns(self): + self.write_input( + { + "module": "示例 / Unit", + "path": "base/missing_return", + "declarations": [ + { + "kind": "unit", + "name": "MissingReturn", + "desc": "缺少返回。", + "members": [ + { + "kind": "function", + "name": "Open", + "signature": "Open()", + "desc": "打开。", + } + ], + } + ], + } + ) + + result = self.run_cli() + + self.assertEqual(1, result.returncode) + self.assertIn("missing 'returns'", result.stderr) + + def test_examples_list_renders_independent_fences_and_output_comments(self): + self.write_input( + { + "module": "项目 / 示例", + "path": "base/examples", + "declarations": [ + { + "kind": "function", + "name": "demo", + "signature": "demo()", + "desc": "示例函数。", + "returns": "string", + "examples": [ + { + "desc": "单行输出", + "code": "return demo();", + "output": "ok", + }, + { + "desc": "多行输出", + "code": "return demo();", + "output": "first\nsecond", + }, + ], + } + ], + } + ) + + result = self.run_cli() + + self.assertEqual(0, result.returncode, result.stderr) + text = self.generated("base/examples").read_text(encoding="utf-8") + self.assertEqual(2, text.count("```tsl")) + self.assertIn("范例01：单行输出", text) + self.assertIn("// 输出：ok", text) + self.assertIn("范例02：多行输出", text) + self.assertIn("// 输出：\n// first\n// second", text) + + def test_atomic_write_failure_preserves_existing_output_and_cleans_temp(self): + module = load_script() + output = self.root / "existing.md" + output.write_text("原内容\n", encoding="utf-8") + + with mock.patch.object(module.os, "replace", side_effect=OSError("replace failed")): + with self.assertRaisesRegex(OSError, "replace failed"): + module.atomic_write(output, "新内容\n") + + self.assertEqual("原内容\n", output.read_text(encoding="utf-8")) + self.assertFalse(any(path.suffix == ".tmp" for path in self.root.iterdir())) + + def test_legacy_root_keys_are_rejected_without_overwrite(self): + legacy_values = { + "functions": [ + {"signature": "Old()", "desc": "旧函数。", "returns": "nil"} + ], + "class": {"name": "Old", "desc": "旧类。", "members": []}, + "unit": {"name": "Old", "desc": "旧接口。", "members": []}, + } + for key, value in legacy_values.items(): + with self.subTest(key=key): + relative = f"base/legacy_{key}" + self.write_input( + { + "module": "示例 / 旧录入", + "path": relative, + key: value, + } + ) + + self.assert_rejected_without_overwrite( + relative, f"unknown field(s): {key}" + ) + + def test_unknown_class_member_kind_is_rejected_without_overwrite(self): + self.write_class_member_input( + { + "kind": "event", + "name": "Changed", + "visibility": "public", + "desc": "发生变化。", + }, + "base/unknown_kind", + ) + + self.assert_rejected_without_overwrite( + "base/unknown_kind", "unknown kind 'event'" + ) + + def test_method_static_field_is_rejected_without_overwrite(self): + self.write_class_member_input( + { + "kind": "method", + "name": "Bad", + "visibility": "public", + "binding": "instance", + "signature": "Bad()", + "desc": "非法方法。", + "static": True, + }, + "base/method_static", + ) + + self.assert_rejected_without_overwrite( + "base/method_static", "unknown field(s): static" + ) + + def test_private_class_member_is_rejected_without_overwrite(self): + self.write_class_member_input( + { + "kind": "field", + "name": "hidden_", + "visibility": "private", + "desc": "私有字段。", + "type": "string", + }, + "base/private_member", + ) + + self.assert_rejected_without_overwrite( + "base/private_member", "visibility must be public or protected" + ) + + def test_property_empty_type_is_treated_as_omitted(self): + self.write_class_member_input( + { + "kind": "property", + "name": "Title", + "visibility": "public", + "desc": "标题。", + "type": "", + "access": "read", + }, + "base/property_type", + ) + + result = self.run_cli() + + self.assertEqual(0, result.returncode, result.stderr) + text = self.generated("base/property_type").read_text(encoding="utf-8") + self.assertIn("### `Title`\n\n声明：property", text) + self.assertIn("访问：read", text) + self.assertNotIn("类型：", text) + + def test_constant_empty_type_is_treated_as_omitted(self): + self.write_class_member_input( + { + "kind": "constant", + "name": "DefaultSize", + "visibility": "public", + "desc": "默认大小。", + "type": "", + "value": 100, + }, + "base/constant_type", + ) + + result = self.run_cli() + + self.assertEqual(0, result.returncode, result.stderr) + text = self.generated("base/constant_type").read_text(encoding="utf-8") + self.assertIn("值：`100`", text) + self.assertNotIn("类型：", text) + + def test_field_missing_type_is_rejected_without_overwrite(self): + self.write_class_member_input( + { + "kind": "field", + "name": "Count", + "visibility": "public", + "desc": "数量。", + }, + "base/field_type", + ) + + self.assert_rejected_without_overwrite( + "base/field_type", "members[0]: type: must be non-empty" + ) + + def test_unit_variable_missing_type_is_rejected_without_overwrite(self): + self.write_input( + { + "module": "示例 / Unit", + "path": "base/variable_type", + "declarations": [ + { + "kind": "unit", + "name": "StrictUnit", + "desc": "严格接口。", + "members": [ + { + "kind": "variable", + "name": "Current", + "desc": "当前值。", + } + ], + } + ], + } + ) + + self.assert_rejected_without_overwrite( + "base/variable_type", "members[0]: type: must be non-empty" + ) + + def test_invalid_method_binding_is_rejected_without_overwrite(self): + self.write_class_member_input( + { + "kind": "method", + "name": "Bad", + "visibility": "public", + "binding": "static", + "signature": "Bad()", + "desc": "非法绑定。", + }, + "base/invalid_binding", + ) + + self.assert_rejected_without_overwrite( + "base/invalid_binding", "invalid binding" + ) + + def test_invalid_property_access_is_rejected_without_overwrite(self): + self.write_class_member_input( + { + "kind": "property", + "name": "Title", + "visibility": "public", + "desc": "标题。", + "type": "string", + "access": "readonly", + }, + "base/invalid_access", + ) + + self.assert_rejected_without_overwrite( + "base/invalid_access", "invalid access" + ) + + def test_invalid_method_modifier_is_rejected_without_overwrite(self): + self.write_class_member_input( + { + "kind": "method", + "name": "Bad", + "visibility": "public", + "binding": "instance", + "signature": "Bad()", + "desc": "非法修饰符。", + "modifiers": ["final"], + }, + "base/invalid_modifier", + ) + + self.assert_rejected_without_overwrite( + "base/invalid_modifier", "invalid modifier" + ) + + def test_property_examples_are_rejected_without_overwrite(self): + self.write_class_member_input( + { + "kind": "property", + "name": "Title", + "visibility": "public", + "desc": "标题。", + "type": "string", + "access": "read", + "examples": [ + { + "desc": "读取标题", + "code": "return Widget.Title;", + } + ], + }, + "base/property_examples", + ) + + self.assert_rejected_without_overwrite( + "base/property_examples", "unknown field(s): examples" + ) + + def test_unit_rejects_implementation_data_without_overwrite(self): + self.write_input( + { + "module": "示例 / Unit", + "path": "base/unit_implementation", + "declarations": [ + { + "kind": "unit", + "name": "StrictUnit", + "desc": "严格接口。", + "members": [], + "implementation": ["hidden_"], + } + ], + } + ) + + self.assert_rejected_without_overwrite( + "base/unit_implementation", "unknown field(s): implementation" + ) + + def test_empty_declarations_are_rejected_without_overwrite(self): + self.write_input( + { + "module": "示例 / 空页", + "path": "base/missing_branch", + "declarations": [], + } + ) + + self.assert_rejected_without_overwrite( + "base/missing_branch", "declarations must be a non-empty list" + ) + + def test_unknown_declaration_kind_is_rejected_without_overwrite(self): + self.write_input( + { + "module": "示例 / 未知声明", + "path": "base/unknown_declaration", + "declarations": [{"kind": "procedure"}], + } + ) + + self.assert_rejected_without_overwrite( + "base/unknown_declaration", "unknown kind 'procedure'" + ) + + def test_top_level_function_requires_name(self): + self.write_input( + { + "module": "示例 / 函数", + "path": "base/missing_function_name", + "declarations": [ + { + "kind": "function", + "signature": "Open()", + "desc": "打开。", + "returns": "nil", + } + ], + } + ) + + self.assert_rejected_without_overwrite( + "base/missing_function_name", "declarations[0]: name" + ) + + def test_top_level_function_name_must_match_signature(self): + self.write_input( + { + "module": "示例 / 函数", + "path": "base/function_name_mismatch", + "declarations": [ + { + "kind": "function", + "name": "Open", + "signature": "Close()", + "desc": "关闭。", + "returns": "nil", + } + ], + } + ) + + self.assert_rejected_without_overwrite( + "base/function_name_mismatch", "name and signature differ" + ) + + def test_duplicate_function_signature_is_rejected(self): + self.write_input( + { + "module": "示例 / 重复函数", + "path": "base/duplicate_function", + "declarations": [ + { + "kind": "function", + "name": "Open", + "signature": "Open()", + "desc": "打开。", + "returns": "nil", + }, + { + "kind": "function", + "name": "open", + "signature": "open()", + "desc": "再次打开。", + "returns": "nil", + }, + ], + } + ) + + self.assert_rejected_without_overwrite( + "base/duplicate_function", "duplicate function signature" + ) + + def test_duplicate_class_and_unit_names_are_rejected(self): + cases = { + "class": [ + { + "kind": "class", + "name": "Widget", + "desc": "组件。", + "members": [], + }, + { + "kind": "class", + "name": "widget", + "desc": "另一组件。", + "members": [], + }, + ], + "unit": [ + { + "kind": "unit", + "name": "Runtime", + "desc": "运行时。", + "members": [], + }, + { + "kind": "unit", + "name": "runtime", + "desc": "另一运行时。", + "members": [], + }, + ], + } + for kind, declarations in cases.items(): + with self.subTest(kind=kind): + relative = f"base/duplicate_{kind}" + self.write_input( + { + "module": "示例 / 重复声明", + "path": relative, + "declarations": declarations, + } + ) + + self.assert_rejected_without_overwrite( + relative, f"duplicate {kind} name" + ) + + def test_function_overloads_and_cross_kind_same_name_are_allowed(self): + self.write_input( + { + "module": "示例 / 重载", + "path": "base/overloads", + "declarations": [ + { + "kind": "function", + "name": "Open", + "signature": "Open(path)", + "desc": "按路径打开。", + "params": [ + {"name": "path", "type": "string", "desc": "路径"} + ], + "returns": "nil", + }, + { + "kind": "function", + "name": "Open", + "signature": "Open(mode)", + "desc": "按模式打开。", + "params": [ + {"name": "mode", "type": "integer", "desc": "模式"} + ], + "returns": "nil", + }, + { + "kind": "class", + "name": "Open", + "desc": "打开器。", + "members": [], + }, + { + "kind": "unit", + "name": "Open", + "desc": "打开接口。", + "members": [], + }, + ], + } + ) + + result = self.run_cli() + + self.assertEqual(0, result.returncode, result.stderr) + text = self.generated("base/overloads").read_text(encoding="utf-8") + markers = [ + "## `Open(path)`", + "## `Open(mode)`", + "## `Open`", + "## `Open`", + ] + positions = [] + start = 0 + for marker in markers: + position = text.index(marker, start) + positions.append(position) + start = position + len(marker) + self.assertEqual(sorted(positions), positions) + self.assertEqual(2, text.count("声明：function")) + self.assertEqual(1, text.count("声明：class")) + self.assertEqual(1, text.count("声明：unit")) + + def test_constant_missing_value_is_rejected_without_overwrite(self): + self.write_class_member_input( + { + "kind": "constant", + "name": "Missing", + "visibility": "public", + "desc": "缺少值。", + }, + "base/missing_value", + ) + + self.assert_rejected_without_overwrite( + "base/missing_value", "missing 'value'" + ) + + def test_class_missing_description_is_rejected_without_overwrite(self): + self.write_input( + { + "module": "示例 / 类", + "path": "base/missing_class_desc", + "declarations": [ + { + "kind": "class", + "name": "MissingDescription", + "desc": "", + "members": [], + } + ], + } + ) + + self.assert_rejected_without_overwrite( + "base/missing_class_desc", + "declarations[0]: desc: must be non-empty", + ) + + def test_method_name_and_parameter_order_must_match_signature(self): + cases = { + "name": { + "kind": "method", + "name": "Expected", + "visibility": "public", + "binding": "instance", + "signature": "Actual()", + "desc": "名称不一致。", + }, + "parameter_order": { + "kind": "method", + "name": "Open", + "visibility": "public", + "binding": "instance", + "signature": "Open(first, second)", + "desc": "参数顺序不一致。", + "params": [ + {"name": "second", "type": "integer", "desc": "第二项"}, + {"name": "first", "type": "integer", "desc": "第一项"}, + ], + }, + } + for name, member in cases.items(): + with self.subTest(name=name): + relative = f"base/mismatch_{name}" + self.write_class_member_input(member, relative) + + expected = "differ" if name == "name" else "signature order" + self.assert_rejected_without_overwrite(relative, expected) + + def test_instance_field_and_static_constant_render_distinct_headings(self): + self.write_input( + { + "module": "示例 / 类", + "path": "base/bindings", + "declarations": [ + { + "kind": "class", + "name": "Bindings", + "desc": "绑定示例。", + "members": [ + { + "kind": "field", + "name": "Name", + "visibility": "public", + "desc": "名称。", + "type": "string", + }, + { + "kind": "constant", + "name": "Maximum", + "visibility": "protected", + "desc": "最大值。", + "value": 10, + "static": True, + }, + ], + } + ], + } + ) + + result = self.run_cli() + + self.assertEqual(0, result.returncode, result.stderr) + text = self.generated("base/bindings").read_text(encoding="utf-8") + self.assertIn("### `Name`\n\n声明：field", text) + self.assertIn("### `Maximum`\n\n声明：static const", text) + + def test_constants_accept_zero_and_false_values(self): + self.write_input( + { + "module": "示例 / Unit", + "path": "base/constants", + "declarations": [ + { + "kind": "unit", + "name": "Constants", + "desc": "常量接口。", + "members": [ + { + "kind": "constant", + "name": "Zero", + "desc": "零。", + "value": 0, + }, + { + "kind": "constant", + "name": "Disabled", + "desc": "关闭。", + "value": False, + }, + ], + } + ], + } + ) + + result = self.run_cli() + + self.assertEqual(0, result.returncode, result.stderr) + text = self.generated("base/constants").read_text(encoding="utf-8") + self.assertIn("值：`0`", text) + self.assertIn("值：`false`", text) + if __name__ == "__main__": unittest.main() diff --git a/tools/tsl-codegen/tests/test_lint.py b/tools/tsl-codegen/tests/test_lint.py index 84d56cdf..896d0b12 100644 --- a/tools/tsl-codegen/tests/test_lint.py +++ b/tools/tsl-codegen/tests/test_lint.py @@ -1,3 +1,4 @@ +import importlib.util import subprocess import sys import tempfile @@ -6,7 +7,22 @@ from pathlib import Path SCRIPT = Path(__file__).parents[1] / "scripts" / "lint.py" -VALID_PAGE = "# 项目 / 示例\n\n## `demo()`\n\n示例函数\n\n返回：nil\n" +API_MARKDOWN = Path(__file__).parents[1] / "scripts" / "api_markdown.py" +VALID_PAGE = ( + "# 项目 / 示例\n\n" + "## `demo()`\n\n" + "声明：function\n\n" + "示例函数\n\n" + "返回：nil\n" +) + + +def load_api_markdown(): + spec = importlib.util.spec_from_file_location("tsl_api_markdown", API_MARKDOWN) + module = importlib.util.module_from_spec(spec) + sys.modules[spec.name] = module + spec.loader.exec_module(module) + return module class DocLintCliTest(unittest.TestCase): @@ -38,6 +54,781 @@ class DocLintCliTest(unittest.TestCase): self.assertEqual(result.returncode, 0, result.stderr) self.assertIn("1 files", result.stderr) + def test_accepts_mixed_typed_h2_declarations(self): + self.page.write_text( + "# 示例 / 混合\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "组件。\n\n" + "### `Open()`\n\n" + "声明：function\n\n" + "打开。\n\n" + "可见性：`public`\n\n" + "## `Parse()`\n\n" + "声明：function\n\n" + "解析。\n\n" + "返回：Widget\n\n" + "### 示例\n\n" + "```tsl\nreturn Parse();\n```\n\n" + "## `Runtime`\n\n" + "声明：unit\n\n" + "运行时接口。\n\n" + "### `Document`\n\n" + "声明：class\n\n" + "文档。\n\n" + "#### `Save()`\n\n" + "声明：function\n\n" + "保存。\n\n" + "可见性：`public`\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page, "--strict") + + self.assertEqual(0, result.returncode, result.stdout + result.stderr) + + def test_declaration_must_use_exact_canonical_line(self): + self.page.write_text( + "# 示例 / 函数\n\n" + "## `Open()`\n\n" + "声明:Function\n\n" + "打开。\n\n" + "返回：nil\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[heading]", result.stdout) + self.assertIn("声明：function", result.stdout) + + def test_business_type_metadata_is_not_a_declaration_marker(self): + module = load_api_markdown() + lines = [ + "## `Account`", + "", + "声明：function", + "", + "远程服务器账户。", + "", + "类型：读写", + "", + "返回：nil", + ] + + entries = list(module.iter_api_entries(lines)) + + self.assertEqual(1, len(entries)) + self.assertTrue(entries[0].heading.valid) + self.assertEqual("function", entries[0].heading.root_kind) + + self.page.write_text( + "# 示例 / 遗留属性\n\n" + "\n".join(lines) + "\n", + encoding="utf-8", + ) + result = self.run_cli("--file", self.page, "--strict") + self.assertEqual(0, result.returncode, result.stdout + result.stderr) + + def test_declaration_must_be_first_non_empty_line_after_heading(self): + self.page.write_text( + "# 示例 / 函数\n\n" + "## `Open()`\n\n" + "打开。\n\n" + "声明：function\n\n" + "返回：nil\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[heading]", result.stdout) + self.assertIn("第一条非空正文", result.stdout) + + def test_tags_must_follow_description(self): + self.page.write_text( + "# 示例 / 函数\n\n" + "## `Open()`\n\n" + "声明：function\n\n" + "\n\n" + "打开。\n\n" + "返回：nil\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[description]", result.stdout) + + def test_property_type_is_optional_but_access_is_required(self): + page_without_type = ( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "### `Title`\n\n" + "声明：property\n\n" + "组件标题。\n\n" + "可见性：`public`\n\n" + "访问：read\n" + ) + self.page.write_text(page_without_type, encoding="utf-8") + + valid = self.run_cli("--file", self.page, "--strict") + + self.assertEqual(0, valid.returncode, valid.stdout + valid.stderr) + + self.page.write_text( + page_without_type.replace("\n访问：read\n", "\n"), + encoding="utf-8", + ) + invalid = self.run_cli("--file", self.page) + + self.assertEqual(1, invalid.returncode) + self.assertIn("[access]", invalid.stdout) + + def test_accepts_class_page_with_optional_method_return(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "### `Close()`\n\n" + "声明：function\n\n" + "关闭组件。\n\n" + "可见性：`public`\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page, "--strict") + + self.assertEqual(0, result.returncode, result.stdout + result.stderr) + + def test_class_member_requires_visibility(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "### `Count`\n\n" + "声明：field\n\n" + "组件数量。\n\n" + "类型：integer\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[visibility]", result.stdout) + + def test_parameterized_property_requires_the_fixed_parameter_table(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "### `Items(index)`\n\n" + "声明：property\n\n" + "按索引读取组件。\n\n" + "可见性：`public`\n\n" + "类型：Widget\n\n" + "访问：read\n\n" + "| 名称 | 类型 | 说明 |\n" + "| --- | --- | --- |\n" + "| `index` | integer | 索引 |\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[param-header]", result.stdout) + + def test_nullary_property_rejects_a_parameter_table(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "### `Title`\n\n" + "声明：property\n\n" + "组件标题。\n\n" + "可见性：`public`\n\n" + "类型：string\n\n" + "访问：read\n\n" + "| 参数 | 类型 | 说明 |\n" + "| --- | --- | --- |\n" + "| `unused` | integer | 不应存在 |\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[param-table]", result.stdout) + + def test_class_method_allows_h4_value_and_example_subheadings(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "### `Open(mode)`\n\n" + "声明：function\n\n" + "打开组件。\n\n" + "可见性：`public`\n\n" + "| 参数 | 类型 | 说明 |\n" + "| --- | --- | --- |\n" + "| `mode` | integer | 打开模式 |\n\n" + "#### `mode` 取值\n\n" + "- `0` — 默认模式\n\n" + "#### 示例\n\n" + "```tsl\nreturn Open(0);\n```\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page, "--strict") + + self.assertEqual(0, result.returncode, result.stdout + result.stderr) + + def test_property_rejects_example_subheading(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "### `Title`\n\n" + "声明：property\n\n" + "组件标题。\n\n" + "可见性：`public`\n\n" + "类型：string\n\n" + "访问：read\n\n" + "#### 示例\n\n" + "```tsl\nreturn Widget.Title;\n```\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[subheading]", result.stdout) + self.assertIn("property", result.stdout) + + def test_accepts_unit_page_with_nested_class_member(self): + self.page.write_text( + "# 示例 / Unit\n\n" + "## `DemoUnit`\n\n" + "声明：unit\n\n" + "提供接口。\n\n" + "### `DefaultSize`\n\n" + "声明：const\n\n" + "默认大小。\n\n" + "值：`100`\n\n" + "### `Open()`\n\n" + "声明：function\n\n" + "打开。\n\n" + "返回：Document\n\n" + "### `Document`\n\n" + "声明：class\n\n" + "文档对象。\n\n" + "#### `Save()`\n\n" + "声明：function\n\n" + "保存。\n\n" + "可见性：`protected`\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page, "--strict") + + self.assertEqual(0, result.returncode, result.stdout + result.stderr) + + def test_unit_class_method_allows_h5_value_and_example_subheadings(self): + self.page.write_text( + "# 示例 / Unit\n\n" + "## `DemoUnit`\n\n" + "声明：unit\n\n" + "提供接口。\n\n" + "### `Document`\n\n" + "声明：class\n\n" + "文档对象。\n\n" + "#### `Save(mode)`\n\n" + "声明：function\n\n" + "保存。\n\n" + "可见性：`public`\n\n" + "| 参数 | 类型 | 说明 |\n" + "| --- | --- | --- |\n" + "| `mode` | integer | 保存模式 |\n\n" + "##### `mode` 取值\n\n" + "- `0` — 默认模式\n\n" + "##### 示例\n\n" + "```tsl\nreturn Save(0);\n```\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page, "--strict") + + self.assertEqual(0, result.returncode, result.stdout + result.stderr) + + def test_unit_function_requires_return(self): + self.page.write_text( + "# 示例 / Unit\n\n" + "## `DemoUnit`\n\n" + "声明：unit\n\n" + "提供接口。\n\n" + "### `Open()`\n\n" + "声明：function\n\n" + "打开。\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[return]", result.stdout) + + def test_unit_function_rejects_an_empty_return_type(self): + self.page.write_text( + "# 示例 / Unit\n\n" + "## `DemoUnit`\n\n" + "声明：unit\n\n" + "提供接口。\n\n" + "### `Open()`\n\n" + "声明：function\n\n" + "打开。\n\n" + "返回：\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[return]", result.stdout) + + def test_empty_type_and_value_lines_are_rejected(self): + cases = { + "property": ( + "# 示例 / 类\n\n## `Widget`\n\n声明：class\n\n组件。\n\n" + "### `Title`\n\n声明：property\n\n标题。\n\n可见性：`public`\n\n" + "类型：\n\n访问：read\n", + "[type]", + ), + "field": ( + "# 示例 / 类\n\n## `Widget`\n\n声明：class\n\n组件。\n\n" + "### `Count`\n\n声明：field\n\n数量。\n\n可见性：`public`\n\n类型：\n", + "[type]", + ), + "variable": ( + "# 示例 / Unit\n\n## `DemoUnit`\n\n声明：unit\n\n接口。\n\n" + "### `Current`\n\n声明：var\n\n当前值。\n\n类型：\n", + "[type]", + ), + "constant": ( + "# 示例 / Unit\n\n## `DemoUnit`\n\n声明：unit\n\n接口。\n\n" + "### `Default`\n\n声明：const\n\n默认值。\n\n值：\n", + "[value]", + ), + } + for name, (text, expected) in cases.items(): + with self.subTest(name=name): + self.page.write_text(text, encoding="utf-8") + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn(expected, result.stdout) + + def test_class_parent_metadata_does_not_count_as_description(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "父类：`BaseWidget`\n\n" + "", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[description]", result.stdout) + + def test_class_parameter_rows_must_match_signature(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "### `Open(mode)`\n\n" + "声明：function\n\n" + "打开组件。\n\n" + "可见性：`public`\n\n" + "| 参数 | 类型 | 说明 |\n" + "| --- | --- | --- |\n" + "| `other` | integer | 错误参数 |\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[param-names]", result.stdout) + + def test_class_parameter_rows_require_type_and_description(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "### `Open(mode)`\n\n" + "声明：function\n\n" + "打开组件。\n\n" + "可见性：`public`\n\n" + "| 参数 | 类型 | 说明 |\n" + "| --- | --- | --- |\n" + "| `mode` | | |\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[param-row]", result.stdout) + + def test_unit_class_member_requires_visibility(self): + self.page.write_text( + "# 示例 / Unit\n\n" + "## `DemoUnit`\n\n" + "声明：unit\n\n" + "提供接口。\n\n" + "### `Document`\n\n" + "声明：class\n\n" + "文档对象。\n\n" + "#### `Name`\n\n" + "声明：field\n\n" + "文档名称。\n\n" + "类型：string\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[visibility]", result.stdout) + + def test_class_and_unit_member_data_lines_are_required(self): + cases = { + "property": ( + "# 示例 / 类\n\n## `Widget`\n\n声明：class\n\n组件。\n\n" + "### `Title`\n\n声明：property\n\n标题。\n\n可见性：`public`\n", + "[access]", + ), + "field": ( + "# 示例 / 类\n\n## `Widget`\n\n声明：class\n\n组件。\n\n" + "### `Count`\n\n声明：field\n\n数量。\n\n可见性：`public`\n", + "[type]", + ), + "variable": ( + "# 示例 / Unit\n\n## `DemoUnit`\n\n声明：unit\n\n接口。\n\n" + "### `Current`\n\n声明：var\n\n当前值。\n", + "[type]", + ), + "constant": ( + "# 示例 / Unit\n\n## `DemoUnit`\n\n声明：unit\n\n接口。\n\n" + "### `Default`\n\n声明：const\n\n默认值。\n", + "[value]", + ), + } + for name, (text, expected) in cases.items(): + with self.subTest(name=name): + self.page.write_text(text, encoding="utf-8") + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn(expected, result.stdout) + + def test_class_legacy_typed_api_heading_is_rejected(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "### function `Open()`\n\n" + "打开组件。\n\n" + "可见性：`public`\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[heading]", result.stdout) + self.assertIn("标题只写名称或调用签名", result.stdout) + + def test_unit_legacy_typed_api_heading_is_rejected(self): + self.page.write_text( + "# 示例 / Unit\n\n" + "## `DemoUnit`\n\n" + "声明：unit\n\n" + "提供接口。\n\n" + "### function `Open()`\n\n" + "打开。\n\n" + "返回：Document\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[heading]", result.stdout) + self.assertIn("标题只写名称或调用签名", result.stdout) + + def test_class_member_at_h4_is_rejected(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "#### `Open()`\n\n" + "声明：function\n\n" + "打开组件。\n\n" + "可见性：`public`\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[heading]", result.stdout) + self.assertIn("H3", result.stdout) + + def test_unit_class_member_at_h5_is_rejected(self): + self.page.write_text( + "# 示例 / Unit\n\n" + "## `DemoUnit`\n\n" + "声明：unit\n\n" + "提供接口。\n\n" + "### `Document`\n\n" + "声明：class\n\n" + "文档对象。\n\n" + "##### `Save()`\n\n" + "声明：function\n\n" + "保存。\n\n" + "可见性：`public`\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("[heading]", result.stdout) + self.assertIn("H4", result.stdout) + + def test_rejects_static_function_and_private_api(self): + self.page.write_text( + "# 示例 / 类\n\n" + "## `Widget`\n\n" + "声明：class\n\n" + "表示组件。\n\n" + "### `Create()`\n\n" + "声明：static function\n\n" + "创建。\n\n" + "可见性：`private`\n", + encoding="utf-8", + ) + + result = self.run_cli("--file", self.page) + + self.assertEqual(1, result.returncode) + self.assertIn("static function", result.stdout) + self.assertIn("private", result.stdout) + + def test_mixed_h2_declarations_reset_nested_context(self): + module = load_api_markdown() + lines = [ + "# Demo", + "", + "## `Widget`", + "", + "声明：class", + "", + "组件。", + "", + "### `Open()`", + "", + "声明：function", + "", + "打开。", + "", + "可见性：public", + "", + "## `Parse(text)`", + "", + "声明：function", + "", + "解析。", + "", + "返回：Widget", + "", + "## `DemoUnit`", + "", + "声明：unit", + "", + "接口。", + "", + "### `Document`", + "", + "声明：class", + "", + "文档。", + "", + "#### `Save()`", + "", + "声明：function", + "", + "保存。", + "", + "可见性：public", + ] + + headings = [entry.heading for entry in module.iter_api_entries(lines)] + + self.assertEqual( + [ + (2, "class", "class", "Widget"), + (3, "method", "class", "Open"), + (2, "function", "function", "Parse"), + (2, "unit", "unit", "DemoUnit"), + (3, "class", "unit", "Document"), + (4, "method", "unit", "Save"), + ], + [ + (item.level, item.kind, item.root_kind, item.name) + for item in headings + ], + ) + + def test_missing_duplicate_and_unknown_declarations_are_errors(self): + module = load_api_markdown() + cases = { + "missing": ( + ["## `Open()`", "", "打开。", "", "返回：nil"], + "第一条非空正文", + ), + "duplicate": ( + [ + "## `Widget`", + "", + "声明：class", + "", + "组件。", + "", + "声明：unit", + ], + "必须且只能包含一行", + ), + "unknown": ( + ["## `Runtime`", "", "声明：module", "", "接口。"], + "必须是 function、class 或 unit", + ), + } + for name, (lines, expected) in cases.items(): + with self.subTest(name=name): + headings = [ + entry.heading for entry in module.iter_api_entries(lines) + ] + + self.assertEqual(1, len(headings)) + self.assertFalse(headings[0].valid) + self.assertIn(expected, headings[0].error) + + def test_plain_h2_ends_api_body_and_resets_nested_context(self): + module = load_api_markdown() + lines = [ + "# Demo", + "", + "## `Parse()`", + "", + "声明：function", + "", + "解析。", + "", + "返回：nil", + "", + "## 其他说明", + "", + "### function `NotAMember()`", + ] + + entries = list(module.iter_api_entries(lines)) + + self.assertEqual(["Parse"], [entry.heading.name for entry in entries]) + self.assertEqual(10, entries[0].end) + + def test_parameter_value_and_example_headings_are_not_api_entries(self): + module = load_api_markdown() + lines = [ + "# Demo", + "", + "## `Widget`", + "", + "声明：class", + "", + "表示组件。", + "", + "### `Open(mode)`", + "", + "声明：function", + "", + "打开。", + "", + "可见性：`public`", + "", + "#### `mode` 取值", + "", + "#### 示例", + "", + "## `Parse(mode)`", + "", + "声明：function", + "", + "解析。", + "", + "| 参数 | 类型 | 说明 |", + "| --- | --- | --- |", + "| `mode` | integer | 模式 |", + "", + "**mode 取值**", + "", + "### 示例", + "", + "## `DemoUnit`", + "", + "声明：unit", + "", + "接口。", + "", + "### `Load(mode)`", + "", + "声明：function", + "", + "加载。", + "", + "#### `mode` 取值", + "", + "#### 示例", + ] + + entries = list(module.iter_api_entries(lines)) + + self.assertEqual( + ["Widget", "Open", "Parse", "DemoUnit", "Load"], + [entry.heading.name for entry in entries], + ) + if __name__ == "__main__": unittest.main()