Tier-2 Review: elasticsearch-onboarding

What We Attempted

We evaluated the elasticsearch-onboarding skill, which presents itself as a guided playbook for onboarding users to Elasticsearch search within the Kibana Agent Builder environment. The skill claims to take users from intent through data setup, mapping, and Dev Tools API snippets (SENSE) using a conversational, one-question-at-a-time approach. We attempted to install the skill as a standalone tool and invoke it via CLI to verify its functionality.

What Failed

Both automated test harness checks failed completely:

• Install (fail): The SKILL.md file contains no install command, package reference, or dependency declaration. The skill is described as a conversational playbook, not a pip-installable package or any other standard installation artifact. There is no setup.py, requirements.txt, or equivalent.

• Smoke-invocation (fail): No CLI binary, entry point, or invocation method is documented in SKILL.md. The skill is explicitly a "guided playbook for Kibana Agent Builder," meaning it is designed to be loaded as a conversational prompt within that specific environment, not executed as a standalone command. There is no main(), no shebang, no argument parser.

What We Observed

The skill's content is well-structured for its intended purpose. The trigger clarity scored a perfect 5.0, and the scope precision at 4.5 reflects tight alignment with the "onboarding" use case. The output specificity (4.0) and self-containment (4.0) are strong for a conversational guide. However, the reusability score of 3.5 hints at the core issue: this skill is tightly coupled to the Kibana Agent Builder environment.

The failure mode is not a bug or broken logic—it is a fundamental architectural mismatch. The skill is a playbook, not a tool. It expects to be loaded into an LLM-powered agent that already has context about Kibana and Elasticsearch, and it provides a structured conversation flow. Our test harness attempted to treat it as an installable, invocable package, which it never claimed to be.

Rating Acknowledgment

The composite score of 4.3 / 5.0 must be considered theoretical until physical re-run resolves the failures. The skill likely performs excellently within its intended environment (Kibana Agent Builder), but we cannot confirm this because our test harness cannot simulate that environment. The failures are not indicative of poor quality—they are artifacts of testing a conversational playbook with tool-oriented integration tests.

Value Assessment

Despite the test failures, this skill appears valuable in principle. The structured onboarding flow—intent → data → mapping → API snippets—is pedagogically sound and addresses a real need for new Elasticsearch users. The trigger clarity is exceptional, covering both formal queries ("how do I build search") and casual openers ("hi", "help"). The skill's design as a single-threaded, step-by-step guide reduces cognitive load for beginners.

The skill would be highly useful if integrated into a conversational agent that already has access to Kibana's Dev Tools and can execute the SENSE snippets it describes. For teams using Kibana Agent Builder, this skill likely accelerates the learning curve significantly.

Recommendation: Rerun the test harness within the actual Kibana Agent Builder environment to validate the skill functionally. Until then, the rating stands as a well-informed estimate, not a verified score. The skill's design philosophy is sound, but its value is contingent on the presence of the correct hosting platform.

Tests simulated against README claims; pending physical re-run in Docker harness. Ran 2026-07-22.

Overall: broken. 0 tests passed, 0 partial, 2 failed; key blocker: SKILL.md describes a conversational playbook, not an installable or invocable tool.

Test	Status	Notes
install	fail	No install command or package is documented in SKILL.md; the skill is a guided playbook, not a pip-installable package.
smoke-invocation	fail	No CLI binary or invocation method is described in SKILL.md; the skill is a conversational playbook for Kibana Agent Builder, not a standalone command.