Tier-2 Review: notion-knowledge-capture

Cluster: content-writing Source: https://github.com/openai/skills/blob/main/skills/.curated/notion-knowledge-capture/SKILL.md Composite score: 4.1 / 5.0 (D1 4.5 · D2 4.0 · D3 4.0 · D4 4.0 · D5 3.5) Harness result: 0 passed, 0 partial, 2 failed — rating is theoretical pending a physical re-run.

What we attempted

We ran the standard two-stage harness: an install step (resolve and provision whatever the skill declares it needs) and a smoke-invocation step (drive the skill end-to-end with a minimal input to confirm it produces its documented artifact). The intent was to validate that the SKILL.md is executable as written — not just well-written.

What failed

Both stages failed, and for the same underlying reason: the skill has no installable artifact and no runnable entrypoint.

• install (fail). SKILL.md documents no install command for the skill itself. There is no package manifest, no pip/npm install, no setup script, no Makefile target. The only install-like line in the document is codex mcp add notion --url https://mcp.notion.com/mcp — and that installs the Notion MCP server, not the skill. The harness cannot provision a skill that ships as prose alone.
• smoke-invocation (fail). There is no README, no CLI, no function signature — nothing to invoke. The workflow is entirely mediated by Notion MCP tools (Notion:notion-search, notion-fetch, notion-create-pages, notion-update-page), each of which requires an OAuth-authenticated Notion workspace and [features].rmcp_client = true in config.toml. In a clean container, Step 0 of the skill explicitly instructs the agent to pause and set up MCP, which means a minimal invocation cannot complete by design.

This is not a flaky failure. It is deterministic and structural.

What we observed

The failure mode is environment coupling, not authoring quality. The skill is written as an operating procedure for an agent that already has Notion MCP connected — and read that way, it is coherent and well-structured. The problem is that the harness (and any clean container) does not meet that precondition, and the skill provides no way to reach it non-interactively. OAuth login is a human-in-the-loop gate; codex mcp login notion cannot be satisfied by a script. Step 0 even acknowledges this ("the user will have to restart codex"), which is honest but also confirms the skill is not headless-executable.

Observed dependencies the harness could not satisfy: codex CLI, Notion MCP server, config.toml with [features].rmcp_client = true, OAuth login, a live Notion workspace with the target databases, plus the reference/*-database.md and examples/*.md files the workflow leans on.

Why the score is theoretical

The 4.1 composite was assigned from static reading of SKILL.md — trigger phrasing, output specificity, scope boundaries, self-containment, reusability. None of those dimensions can be confirmed or refuted by a harness that never got the skill to run. Until a physical re-run with a connected Notion workspace resolves both failures, treat the score as a documentation-quality estimate, not a validated one. It is entirely possible the skill performs well once MCP is live — and equally possible that the reference/ and examples/ files it depends on are missing or thin, which the harness could not check.

Is it still valuable in principle?

Yes — with a caveat. The skill solves a real problem (turning ad-hoc conversations into structured, linkable Notion pages) and its workflow is genuinely good: it forces a destination decision before drafting, distinguishes content types (decision vs. how-to vs. FAQ), and insists on backlinks and changelogs. That is the kind of discipline most "dump it in Notion" skills lack.

The caveat is that its value is conditional on a pre-provisioned environment. As shipped, it is a playbook, not a skill artifact. To be harness-verifiable it would need at minimum: (a) a declared install/setup step distinct from the MCP server install, (b) a non-interactive path or a documented fixture/mock for the Notion MCP tools, and (c) the reference/ and examples/ files bundled and validated. Fix those three and the theoretical 4.1 becomes a measurable number. Leave them as-is and the skill remains useful to humans reading it, but unverifiable by automation.

Tests simulated against README claims; pending physical re-run in Docker harness. Ran 2026-09-20.

Overall: broken. 0 tests passed, 0 partial, 2 failed; key blocker: SKILL.md defines no installable artifact or runnable entrypoint and requires an OAuth-authenticated Notion MCP connection that is unavailable in a clean container.

Inferred dependencies: codex CLI, Notion MCP server (https://mcp.notion.com/mcp), config.toml with [features].rmcp_client = true (or --enable rmcp_client), OAuth login via codex mcp login notion, Notion workspace access with target databases, reference/-database.md schema guides, examples/.md capture patterns.

Test	Status	Notes
install	fail	SKILL.md documents no install command for the skill itself; the only install-like step is codex mcp add notion --url https://mcp.notion.com/mcp, which installs the Notion MCP server, not the skill. No package manifest, pip/npm install, or setup script is described, so the documented install cannot be executed as written.
smoke-invocation	fail	SKILL.md has no README or runnable CLI entrypoint; the workflow depends on Notion MCP tools (Notion:notion-search, notion-fetch, notion-create-pages, notion-update-page) that require OAuth login and [features].rmcp_client = true. In a clean container without Notion MCP connected, Step 0 explicitly says to pause and set it up, so a minimal invocation cannot complete.