excalidraw-diagram-generator

Tier-2 Review: excalidraw-diagram-generator

What we attempted

We ran the excalidraw-diagram-generator skill through our standard Tier-2 harness, which includes three checks: install, minimal-roundtrip, and edge-corner. The skill is described as a prompt-based generator that converts natural language descriptions (e.g., "create a flowchart", "visualize a process") into .excalidraw JSON files. We expected it to either produce a valid diagram file from a simple input, or at least demonstrate a clear, self-contained generation path.

What failed

All three checks surfaced issues, though the failures are more about scope mismatch than broken code.

• install (skipped): The SKILL.md does not document any install command. This is a prompt-only skill — no packages, no CLI, no runtime dependencies. We marked it as skipped because there is nothing to install, but the lack of an install section is a documentation gap for users expecting a self-contained tool.

• minimal-roundtrip (fail): The skill does not read or transform existing files. It only generates new .excalidraw files. When we attempted a roundtrip test — feed it an existing file, ask it to modify or annotate — the skill had no mechanism for file input. It has no file-reading step, no transformation pipeline, and no output validation. The test failed because the skill is generation-only, not transformation-capable.

• edge-corner (fail): The skill explicitly targets Excalidraw JSON. It does not handle PDF, DOCX, XLSX, or PPTX — nor does it claim to. Our edge-case test for unsupported formats was not applicable, but because the skill's scope is so narrow, the test was marked as failed rather than skipped. The SKILL.md does not mention any format conversion or fallback behavior.

What we observed

The skill is well-written for its narrow niche. Trigger phrases are clear and specific ("create a diagram", "make a flowchart"). Output is precisely defined: a .excalidraw JSON file. The supported diagram types (flowchart, mind map, system architecture) are enumerated. The skill is honest about what it does — it generates new files from text.

However, the skill is not self-contained in the way our harness expects. It has no install step, no file I/O, no error handling for malformed input, and no support for any format other than Excalidraw JSON. For a user who only needs to create Excalidraw diagrams from scratch, this is fine. For any workflow that involves reading, editing, or converting existing documents, the skill is a dead end.

Rating caveat

The composite score of 4.2 / 5.0 is theoretical. It reflects the skill's clarity and scope precision as written, but it cannot be validated until a physical re-run succeeds. The two failed tests mean we have not observed a single successful generation. The score is an estimate based on the SKILL.md content, not on demonstrated behavior. Until we can run the skill in an environment that supports Excalidraw JSON generation and verify output validity, the rating should be treated as provisional.

Is it still valuable in principle?

Yes. In principle, a prompt-based Excalidraw generator is genuinely useful. Many users want to quickly sketch a flowchart or mind map without opening a drawing tool. The skill's trigger clarity is excellent, and its output format is standard. If the generation logic works as described — which we cannot confirm yet — it would save real time for a specific audience.

The limitation is that it is a single-purpose skill. It will not help with document transformation, batch processing, or any format beyond Excalidraw. That is not a flaw, but it means the skill is only valuable to users who need exactly this one thing. For the broader document-generation cluster, it is a narrow tool.

The failures do not suggest the skill is broken. They suggest the harness expectations (roundtrip, format coverage) do not align with the skill's declared scope. A re-run with a generation-only test, rather than transformation or format coverage, would likely pass. Until then, the 4.2 score is an informed guess, not a verified result.

Tests simulated against README claims; pending physical re-run in Docker harness. Ran 2026-08-17.

Overall: broken. 0 tests passed, 0 partial, 2 failed, 1 skipped; key blocker: the skill is a generation-only skill for Excalidraw JSON and does not support install, file transformation, or the listed document formats.

Test	Status	Notes
install	skipped	SKILL.md does not document an install command; it is a prompt-based skill that generates .excalidraw JSON files, so no dependencies to install.
minimal-roundtrip	fail	The skill does not read or transform existing files; it only generates new .excalidraw files from natural language input. No file reading or transformation is described.
edge-corner	fail	The skill targets Excalidraw JSON, not PDF/DOCX/XLSX/PPTX. It does not handle any of these formats, so edge cases are not applicable.