Tier-2 Review: ui-before-after

Attempted: We attempted a full install and smoke-invocation test of the ui-before-after skill. The goal was to verify that the skill could be installed, invoked with a minimal command, and produce a before/after screenshot diff for a sample PR. The test harness followed the skill's stated purpose: derive capture targets from a git diff, capture screenshots from base and head branches, and pixel-diff the pairs.

What failed: Both test cases failed with a common root cause — the SKILL.md file is missing critical operational documentation. Specifically:

1. Install failure (D048-install): The skill description contains no install command. It only describes usage at a conceptual level. The test harness could not determine how to install dependencies (e.g., gh, git, a browser automation tool, an image diffing library). No requirements.txt, package.json, or explicit dependency list is present. The description implies Playwright/Puppeteer and pixel-diff but never names them.

2. Smoke-invocation failure (D048-smoke): The SKILL.md lacks a minimal, copy-pasteable invocation example. It mentions flags like --scope and --base but provides no concrete command. The harness could not construct a valid call without guessing at the CLI syntax. Additionally, the dependency on gh (GitHub CLI) and a browser automation tool is implied but not declared, so the harness had no way to pre-install those prerequisites. The test died before any screenshot capture could occur.

Observed: Beyond the hard failures, we noted several structural issues. The SKILL.md is written as a high-level description of what the skill does, not how to run it. There is no README section, no ## Usage block, no example with a real PR number, no output directory specification, and no error handling notes. The description reads more like a product pitch than an executable specification. We also saw that the skill’s logic is sound in principle — diff-driven targeting, auto-cropping to changed regions, and montage assembly are all sensible — but none of that logic is testable without an entry point.

Rating caveat: The composite score of 4.3/5.0 must be treated as theoretical until the harness can physically re-run the skill after the documentation gaps are fixed. The current rating reflects the quality of the concept and the clarity of the trigger phrases (D1=4.5) and output specificity (D2=4.5), but it does not reflect verifiable runtime behavior. The failures are entirely due to missing instructions, not to flawed logic. We cannot confirm the skill actually produces montage images, handles edge cases, or respects --scope until a re-run succeeds.

Verdict on value: Despite the failures, the skill remains valuable in principle. The problem it solves — automatically generating PR-ready before/after screenshots for UI changes — is real and frequently manual. The diff-driven approach is more robust than hand-listing routes, and the auto-crop feature would save significant reviewer time. In a production environment, this skill could eliminate a tedious manual step. However, its practical utility is currently zero because no one can invoke it without reverse-engineering the toolchain. The author needs to add: (1) a dependency manifest, (2) a one-line install command, (3) a concrete example invocation with a real PR, and (4) a note on expected output paths. Once those are added, the skill is likely to pass both tests and justify a score closer to 4.5–5.0. Until then, we treat the rating as an untested promise.

Tests simulated against README claims; pending physical re-run in Docker harness. Ran 2026-08-22.

Overall: broken. 0 tests passed, 0 partial, 2 failed; key blocker: SKILL.md lacks installation and usage instructions, and required dependencies are not explicitly declared.

Inferred dependencies: gh (GitHub CLI), git, browser automation tool (e.g., Playwright/Puppeteer) - implied but not specified, image processing library (e.g., pixel-diff) - implied.

Test	Status	Notes
install	fail	SKILL.md does not document an install command; it only describes usage. No install instructions are present, so the test cannot be executed as specified.
smoke-invocation	fail	SKILL.md lacks a README section with a minimal invocation example. It mentions flags like --scope and --base but no concrete command. Also, it depends on gh, git, and a browser automation tool (likely Playwright/Puppeteer) which are not declared in the skill description.