📦 deps(thirdparty): update snapshots
This commit is contained in:
ci[bot]
@@ -0,0 +1,101 @@
|
||||
# Meeting Intelligence Skill Evaluations
|
||||
|
||||
Evaluation scenarios for testing the Meeting Intelligence skill across different Codex models.
|
||||
|
||||
## Purpose
|
||||
|
||||
These evaluations ensure the Meeting Intelligence skill:
|
||||
- Gathers context from Notion workspace
|
||||
- Enriches with Codex research appropriately
|
||||
- Creates both internal pre-reads and external agendas
|
||||
- Distinguishes between Notion facts and Codex insights
|
||||
- Works consistently across Haiku, Sonnet, and Opus
|
||||
|
||||
## Evaluation Files
|
||||
|
||||
### decision-meeting-prep.json
|
||||
Tests preparation for a decision-making meeting.
|
||||
|
||||
**Scenario**: Prep for database migration decision meeting
|
||||
**Key Behaviors**:
|
||||
- Searches Notion for migration context (specs, discussions, options)
|
||||
- Fetches 2-3 relevant pages
|
||||
- Enriches with Codex research (decision frameworks, migration best practices)
|
||||
- Creates comprehensive internal pre-read with recommendation
|
||||
- Creates clean, professional external agenda
|
||||
- Clearly distinguishes Notion facts from Codex insights
|
||||
- Cross-links both documents
|
||||
|
||||
### status-meeting-prep.json
|
||||
Tests preparation for a status update or review meeting.
|
||||
|
||||
**Scenario**: Prep for project status review
|
||||
**Key Behaviors**:
|
||||
- Gathers project metrics and progress from Notion
|
||||
- Fetches relevant pages (roadmap, tasks, milestones)
|
||||
- Adds Codex context (industry benchmarks, best practices)
|
||||
- Creates internal pre-read with honest assessment
|
||||
- Creates external agenda with structured flow
|
||||
- Includes source citations using mention-page tags
|
||||
- Time-boxes agenda items
|
||||
|
||||
## Running Evaluations
|
||||
|
||||
1. Enable the `meeting-intelligence` skill
|
||||
2. Submit the query from the evaluation file
|
||||
3. Verify the skill searches Notion first (not Codex research)
|
||||
4. Check that TWO documents are created (internal + external)
|
||||
5. Verify Codex enrichment adds value without replacing Notion content
|
||||
6. Test with Haiku, Sonnet, and Opus
|
||||
|
||||
## Expected Skill Behaviors
|
||||
|
||||
Meeting Intelligence evaluations should verify:
|
||||
|
||||
### Notion Context Gathering
|
||||
- Searches workspace for relevant context first
|
||||
- Fetches specific pages (not generic)
|
||||
- Extracts key information from Notion content
|
||||
- Cites sources using mention-page tags
|
||||
|
||||
### Codex Research Integration
|
||||
- Adds industry context, frameworks, or best practices
|
||||
- Enrichment is relevant and valuable (not filler)
|
||||
- Clearly distinguishes Notion facts from Codex insights
|
||||
- Research complements (doesn't replace) Notion content
|
||||
|
||||
### Two-Document Creation
|
||||
- **Internal Pre-Read**: Comprehensive, includes strategy, recommendations, detailed pros/cons
|
||||
- **External Agenda**: Professional, focused on meeting flow, no internal strategy
|
||||
- Both documents are clearly labeled
|
||||
- Documents are cross-linked
|
||||
|
||||
### Document Quality
|
||||
- Pre-read follows structure: Overview → Background → Current Status → Context & Insights → Discussion Points
|
||||
- Agenda follows structure: Details → Objective → Agenda Items (with times) → Decisions → Actions → Resources
|
||||
- Titles include date or meeting context
|
||||
- Content is actionable and meeting-ready
|
||||
|
||||
## Creating New Evaluations
|
||||
|
||||
When adding Meeting Intelligence evaluations:
|
||||
|
||||
1. **Test different meeting types** - Decision, status, brainstorm, 1:1, sprint planning, retrospective
|
||||
2. **Vary complexity** - Simple updates vs. complex strategic decisions
|
||||
3. **Test with/without Notion content** - Rich workspace vs. minimal existing pages
|
||||
4. **Verify enrichment value** - Is Codex research genuinely helpful?
|
||||
5. **Check internal/external distinction** - Is sensitive info kept in pre-read only?
|
||||
|
||||
## Example Success Criteria
|
||||
|
||||
**Good** (specific, testable):
|
||||
- "Creates TWO documents (internal pre-read + external agenda)"
|
||||
- "Internal pre-read marked 'INTERNAL ONLY' or 'For team only'"
|
||||
- "Cites at least 2-3 Notion pages using mention-page tags"
|
||||
- "Agenda includes time allocations for each section"
|
||||
- "Codex enrichment includes decision frameworks or best practices"
|
||||
|
||||
**Bad** (vague, untestable):
|
||||
- "Creates meeting materials"
|
||||
- "Gathers context effectively"
|
||||
- "Prepares well"
|
||||
+35
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"name": "Decision Meeting Preparation",
|
||||
"skills": ["meeting-intelligence"],
|
||||
"query": "Prep for tomorrow's meeting where we need to decide on our database migration approach. Create both an internal pre-read for the team and an agenda for the meeting.",
|
||||
"expected_behavior": [
|
||||
"Step 1: Uses Notion:notion-search to find context about database migration (project pages, technical specs, previous discussions, options analysis)",
|
||||
"Step 2: Fetches at least 2-3 relevant pages using Notion:notion-fetch to gather information from Notion",
|
||||
"Step 3: Identifies the decision to be made and available options from fetched Notion content",
|
||||
"Step 4: Enriches with Codex research - adds decision-making frameworks (e.g., cost-benefit analysis, risk assessment), technical context for migration approaches, best practices for database migrations",
|
||||
"Step 5: Distinguishes Notion facts from Codex insights in synthesis",
|
||||
"Step 6: Creates INTERNAL PRE-READ using Notion:notion-create-pages with title like 'INTERNAL: Database Migration Decision - Pre-Read - [Date]'",
|
||||
"Step 6a: Internal pre-read includes: Meeting overview, background context (from Notion), current status and technical details, context & insights (from Codex research on migration best practices), decision options with detailed pros/cons, recommendation with rationale, what we need from meeting",
|
||||
"Step 6b: Internal pre-read marked clearly as 'INTERNAL ONLY' or 'For team only'",
|
||||
"Step 7: Creates EXTERNAL AGENDA using Notion:notion-create-pages with title like 'Meeting Agenda: Database Migration Decision - [Date]'",
|
||||
"Step 7a: External agenda includes: Meeting details, objective (clear decision to make), agenda items with time allocations, discussion topics, decisions needed, action items section (empty), related resources with link to pre-read",
|
||||
"Step 7b: External agenda is clean, professional, focused (no internal strategy or detailed pros/cons)",
|
||||
"Step 8: Links both documents together (agenda mentions pre-read, pre-read mentions agenda)",
|
||||
"Both documents link to source pages using <mention-page url='...'>"
|
||||
],
|
||||
"success_criteria": [
|
||||
"TWO documents are created (internal pre-read + external agenda), not just one",
|
||||
"Internal pre-read is comprehensive with: Notion context + Codex insights + detailed pros/cons + recommendation",
|
||||
"Internal pre-read is clearly marked 'INTERNAL' or 'For team only'",
|
||||
"External agenda is professional, structured, focused on meeting flow (not internal strategy)",
|
||||
"Codex enrichment is present and adds value (decision frameworks, migration best practices, risk patterns)",
|
||||
"Notion facts are clearly sourced, Codex insights are distinguished",
|
||||
"At least 2-3 Notion source pages are cited using mention-page tags",
|
||||
"Internal pre-read follows structure from SKILL.md Step 5 (Meeting Overview → Background → Current Status → Context & Insights → Key Discussion Points → What We Need)",
|
||||
"External agenda follows structure from SKILL.md Step 6 (Meeting Details → Objective → Agenda Items → Discussion Topics → Decisions Needed → Action Items → Related Resources)",
|
||||
"Documents are cross-linked (pre-read mentions agenda, agenda mentions pre-read)",
|
||||
"Meeting date is included in both titles",
|
||||
"Uses correct tool names (Notion:notion-search, Notion:notion-fetch, Notion:notion-create-pages for BOTH documents)"
|
||||
]
|
||||
}
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
{
|
||||
"name": "Status Update Meeting Preparation",
|
||||
"skills": ["meeting-intelligence", "task-manager"],
|
||||
"query": "Prep for Friday's project status meeting on the Mobile App Redesign project. Create both an internal pre-read and an external agenda.",
|
||||
"expected_behavior": [
|
||||
"Step 1: Uses Notion:notion-search to find Mobile App Redesign project page",
|
||||
"Step 2: Fetches project page using Notion:notion-fetch to get current status and context",
|
||||
"Step 3: Uses Notion:notion-search to find tasks database",
|
||||
"Step 4: Queries task database using Notion:notion-query-data-sources for project tasks (WHERE Project = 'Mobile App Redesign')",
|
||||
"Step 5: Analyzes task data: calculates completion %, identifies completed work, in-progress items, and blockers",
|
||||
"Step 6: Enriches with Codex research - adds project management insights (velocity trends, risk patterns, common project pitfalls), suggests discussion frameworks if risks identified, provides context on timeline implications",
|
||||
"Step 7: Creates INTERNAL PRE-READ using Notion:notion-create-pages with title 'INTERNAL: Mobile App Redesign Status - Pre-Read - [Date]'",
|
||||
"Step 7a: Internal pre-read includes: Project overview, current status with metrics (from Notion/tasks), progress summary with specifics, context & insights (Codex research on project health patterns), honest assessment of challenges/risks, what we need from meeting",
|
||||
"Step 7b: Internal pre-read contains detailed metrics, blockers, and strategic considerations",
|
||||
"Step 8: Creates EXTERNAL AGENDA using Notion:notion-create-pages with title 'Meeting Agenda: Mobile App Redesign Status Update - [Date]'",
|
||||
"Step 8a: External agenda uses Status Update structure: Meeting Details → Objective → Agenda Items (timed) → Discussion Topics → Action Items",
|
||||
"Step 8b: External agenda is concise, professional, focuses on meeting flow (summary-level metrics only)",
|
||||
"Step 9: Links both documents together",
|
||||
"Both documents link to project page and task database using <mention-page> and <mention-database>"
|
||||
],
|
||||
"success_criteria": [
|
||||
"TWO documents are created (internal pre-read + external agenda)",
|
||||
"Internal pre-read contains: Detailed metrics from task query, honest assessment of blockers/risks, Codex insights on project health patterns, strategic considerations",
|
||||
"Internal pre-read is marked 'INTERNAL' or 'For team only'",
|
||||
"External agenda is professional status update format with: Overall status, progress summary (high-level), upcoming work, discussion topics, action items section",
|
||||
"External agenda does NOT contain detailed risk analysis or internal strategic discussion (that's in pre-read)",
|
||||
"Progress metrics are accurate based on Notion:notion-query-data-sources results (e.g., X complete, Y in progress, Z blocked)",
|
||||
"Blockers identified from task query are included in internal pre-read with full context",
|
||||
"Codex enrichment adds value (project management insights, risk patterns, velocity analysis)",
|
||||
"Links to project page and task database using mention tags",
|
||||
"Documents are cross-linked for easy navigation",
|
||||
"Uses correct tool names (Notion:notion-search, Notion:notion-fetch, Notion:notion-query-data-sources, Notion:notion-create-pages for BOTH documents)"
|
||||
]
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user