Tier-2 Review: elasticsearch-tutorial

What we attempted

We pulled the elasticsearch-tutorial skill into a clean Docker container and ran it through our standard two-stage harness: an install step (does the skill declare a package, binary, or setup command we can execute?) and a smoke-invocation step (can we trigger the skill end-to-end and observe a concrete result?).

What failed

Both stages failed to produce a passing result, and the failure mode is the same root cause in each case.

Install — skipped. The SKILL.md is a prompt/instruction document. It contains no install command, no package reference, no binary, and no setup script. There is nothing to install into a clean container, so the install stage had no work to do and was marked skipped rather than passed. This is not a defect on its own — many prompt-only skills legitimately have nothing to install — but it means the harness could not establish a working baseline before invocation.

Smoke-invocation — failed. The skill defines itself as a "topic-driven, hands-on Elasticsearch tutorial flow that runs in Kibana Dev Console." Its trigger surface is conversational ("walk me through", "teach me", "show me how X works"), and its output contract is a sequence of SENSE snippets the user is expected to paste into Dev Tools against a live Elasticsearch cluster. Neither prerequisite exists in our clean container. There is no README, no runnable entrypoint, and no way to synthesize the interactive Kibana surface the skill assumes. We could not exercise the skill at all, so we recorded a hard failure rather than a partial.

The harness result was therefore: 0 passed, 0 partial, 1 failed, 1 skipped.

What we observed

The failure is environmental, not obviously authorial. The skill is explicitly scoped to an interactive Kibana Dev Console session against a live cluster — that is the whole point of the format. Our harness is built for containerized, headless, non-interactive execution, which is the wrong substrate for this class of skill. We are not claiming the skill is broken; we are claiming we could not test it as written.

Observed dependencies: none declared. That is itself informative — the skill implicitly depends on (a) a reachable Elasticsearch cluster, (b) a Kibana instance with Dev Tools, and (c) a human or agent willing to paste snippets and read responses. None of these are declared, so even a more capable harness would have to infer them.

Rating status

The composite score of 4.2 / 5.0 (D1 4.5, D2 4.0, D3 4.5, D4 4.0, D5 3.5) is theoretical. It rests on reading the SKILL.md against our rubric, not on observed behavior. Until a physical re-run resolves the two failures above — most plausibly by provisioning a throwaway ES + Kibana pair and driving the flow interactively — the dimension scores should be treated as provisional. D4 (self-containment) in particular is the dimension most likely to move once we can actually run it, because the skill's real dependencies are undeclared.

Does the skill still seem valuable in principle?

Yes. The trigger language is unusually crisp — it draws a clean line between "teach me how X works" and "build my real use case," which is exactly the distinction that trips up general-purpose assistants. The scope is tight (sample data, isolated resources, cleanup, docs pointers), the topic list is broad and current (semantic_text, kNN, RRF, ES|QL, reranking), and the output format — every step as a runnable SENSE snippet — is the right shape for a tutorial skill. Nothing we saw suggests the design is wrong; it suggests the skill targets an execution surface our harness does not yet provide. The honest next step is a Kibana-backed test rig, not a rewrite.

Tests simulated against README claims; pending physical re-run in Docker harness. Ran 2026-10-06.

Overall: broken. 0 tests passed, 0 partial, 1 failed, 1 skipped; key blocker: the skill is a prompt-only tutorial flow with no install command, README, or runnable entrypoint, and requires an interactive Kibana Dev Console plus a live Elasticsearch cluster that a clean Docker container does not provide.

Test	Status	Notes
install	skipped	SKILL.md is a prompt/instruction document for a Kibana Dev Console tutorial flow and does not document any install command, package, or binary; there is nothing to install in a clean container.
smoke-invocation	fail	No README or runnable entrypoint is provided; the skill only describes an interactive tutorial flow triggered by phrases like 'walk me through' or 'teach me', which cannot be exercised without a Kibana Dev Console and a live Elasticsearch cluster.