Tier-2 Review: experimental-design (Slug: experimental-design)

What We Attempted

We attempted to run the skill experimental-design in a clean test harness to validate its installability, basic invocation, and functional correctness. The skill is described as a domain-specific agent for designing experiments and studies before data is collected — covering randomization, blocking, factorial designs, response-surface optimization, and related topics. The auto-summary and composite score (4.3/5.0) indicated strong trigger clarity (5.0) and scope precision (4.5), but flagged output specificity (3.5) as a weaker dimension.

What Failed

No tests passed. All two test cases (install and smoke-invocation) were skipped. The key blocker was that the SKILL.md file provided only a textual description of the skill's purpose and triggers — it contained no executable code, no installation instructions, no CLI or Python API invocation examples, and no dependencies to install. The test harness could not determine a minimal invocation command or package name. Attempting to infer a pip package (e.g., pip install experimental-design) would fail because no such package exists on PyPI or is referenced in the skill metadata.

What We Observed

The skill appears to be a pure description / prompt-engineering artifact rather than a runnable software package. It defines a set of natural-language triggers (e.g., "how should I set up this experiment", "assign these mice to conditions") and a domain scope, but does not include any code, configuration, or instructions to instantiate it as an agent or tool. The SKILL.md content is a well-written, comprehensive prompt for an LLM, but it is not executable in any standard sense. The test harness could not:

• Install the skill (no package, no requirements file, no install command).
• Invoke the skill (no entry point, no CLI, no Python function signature).
• Validate outputs (no example outputs, no test cases, no expected behavior).

Implications for the Rating

The composite score (4.3) and dimension ratings are theoretical until a physical re-run can succeed. The high trigger clarity (5.0) and scope precision (4.5) are plausible given the well-structured trigger list and domain definition, but they cannot be empirically verified. The output specificity (3.5) is consistent with a text-only description that lacks concrete output examples. The self-containment (4.0) and reusability (4.0) are also speculative — a description-only skill is inherently less self-contained and reusable than a packaged tool.

Is the Skill Still Valuable in Principle?

Yes, in principle. The domain of experimental design is a legitimate and valuable area for an LLM agent. A well-crafted trigger list and scope definition can significantly improve the quality of LLM responses in this domain by narrowing the prompt to relevant statistical concepts (blocking, randomization, factorial designs, pseudoreplication, etc.). The skill's trigger list is unusually thorough and covers many edge cases (e.g., “plate layouts”, “batch/run-order effects”, “sequential/adaptive/group-sequential designs”). If the skill were paired with a minimal executable wrapper (e.g., a Python function that calls an LLM with the system prompt), it could become a useful tool. The failure mode here is not a flaw in the concept, but a lack of packaging — the skill is a prompt, not a program. For it to be testable and reusable, the author would need to provide:

• A requirements.txt or pyproject.toml with dependencies.
• A skill.py or main.py with a documented entry point.
• At least one example invocation (e.g., python -m skill "How should I design a 3-factor experiment with blocking?").

Until then, the skill remains a high-quality but non-executable specification, and its rating should be treated as provisional.

Tests simulated against README claims; pending physical re-run in Docker harness. Ran 2026-07-28.

Overall: broken. 0 tests passed, 0 partial, 2 skipped; key blocker: SKILL.md is a description-only skill with no executable code or install instructions.

Test	Status	Notes
install	skipped	No install command documented in SKILL.md; assuming pip install experimental-design would fail because package not on PyPI.
smoke-invocation	skipped	No CLI or Python API invocation example in SKILL.md; cannot determine minimal invocation.