openai-docs

Tier-2 Review: openai-docs (Slug: openai-docs)

What We Attempted

This review evaluates the openai-docs skill, which is designed to provide authoritative, current guidance from OpenAI developer documentation using a dedicated MCP server. The skill claims to handle general docs lookup, model selection, API model migration, and prompt-upgrade guidance. Our test harness attempted two core validations: (1) installing the required MCP server via the codex mcp add command, and (2) performing a smoke-invocation of the primary MCP tool mcp__openaiDeveloperDocs__search_openai_docs to verify the skill's core functionality works end-to-end.

What Failed

Both tests failed completely:

1. Install (fail): The install command codex mcp add openaiDeveloperDocs --url https://developers.openai.com/mcp failed because the codex CLI is not installed in the test environment. The SKILL.md does not list codex as a dependency, and the command predictably returned "command not found." This is the fundamental blocker—without the Codex CLI, the MCP server cannot be installed, and the entire skill's primary workflow is unreachable.

2. Smoke-invocation (fail): Because the install step failed, the MCP tool mcp__openaiDeveloperDocs__search_openai_docs is not available. The SKILL.md describes a fallback to web search only if the MCP server is installed but returns no results; our test does not exercise this fallback because the server itself was never installed. The skill's core functionality is therefore untestable in its current form without the Codex CLI.

What We Observed

Beyond the immediate failures, several structural observations emerged:

• Missing dependency declaration: The skill's explicit instructions require codex CLI to install the MCP server, yet this dependency is not listed in the skill's metadata or quick-start section. Any environment without Codex will hit this blocker immediately.
• Chain of dependencies: The skill also references node (for scripts/resolve-latest-model-info.js) and implicitly requires python>=3.10 (as the Codex environment implies). These are not declared, creating a fragile dependency chain.
• Fallback ambiguity: The SKILL.md describes a fallback to web search only if the MCP server "returns no meaningful results," but the install step itself has no fallback. If Codex is missing, the entire skill is non-functional.
• Theoretical score vs. practical reality: The skill scores an impressive 4.6/5.0 across all dimensions on paper—trigger clarity (5.0), output specificity (4.5), scope precision (4.5), self-containment (5.0), and reusability (4.0). However, these scores are purely theoretical until the dependency issue is resolved and the skill can be physically re-run.

Rating Acknowledgment

The composite score of 4.6 / 5.0 should be considered provisional until the test harness can actually execute the skill. The failures here are environmental, not conceptual—the skill's design is sound, its triggers are clear, and its scope is well-defined. But a skill that cannot be installed or invoked cannot be meaningfully verified.

Value Assessment

Despite the test failures, the skill retains significant value in principle. The idea of an MCP-server-backed documentation skill that provides authoritative, citation-linked answers from official OpenAI docs is excellent. The workflow for model selection, migration, and prompt-upgrade guidance is well-thought-out and addresses a genuine developer need. The skill's structure—preferring remote docs over bundled references, preserving explicit user targets, and providing a clear "if MCP server is missing" fallback chain—demonstrates careful design.

The primary weakness is not the skill's logic but its tight coupling to the Codex CLI, which is not a universal dependency. If the skill were refactored to either (a) declare the Codex dependency explicitly, (b) provide an alternative install path (e.g., direct npm/pip install), or (c) document the prerequisite clearly, it would be a genuinely excellent skill. As it stands, it is a well-designed skill that cannot currently be tested or used outside of environments with Codex pre-installed.

Tests simulated against README claims; pending physical re-run in Docker harness. Ran 2026-05-26.

Overall: broken. 0 tests passed, 0 partial, 2 failed; key blocker: 'codex' CLI not available, preventing MCP server installation and subsequent doc tool invocations.

Inferred dependencies: codex CLI (not installed), node (for scripts/resolve-latest-model-info.js), python>=3.10 (implied by Codex environment).

Test	Status	Notes
install	fail	The install command requires the 'codex' CLI, which is not installed in the test environment. SKILL.md does not list codex as a dependency, and the command is expected to fail with 'command not found'.
smoke-invocation	fail	The MCP tool 'mcp__openaiDeveloperDocs__search_openai_docs' is not available because the MCP server was not installed (install step failed). SKILL.md states to fall back to web search only if MCP server is missing, but no fallback is tested here.