cli-creator

Tier-2 Review: cli-creator (Slug: cli-creator)

What We Attempted

We evaluated the cli-creator skill, which is designed to build composable CLIs for Codex from various sources such as API docs, OpenAPI specs, curl examples, SDKs, web apps, admin tools, or local scripts. The skill aims to create durable command-line tools that future Codex threads can run by command name from any working directory.

What Failed

All four test harness checks failed:

1. install-and-auth (fail): The skill is a meta-skill for generating CLIs, not an installable package itself. No pip-installable package exists; the skill expects users to scaffold a new CLI project. Authentication concepts are not applicable to the skill itself.

2. list-or-read (fail): No list command exists within the skill itself. The skill generates CLIs but does not expose read operations. The only valid invocation is providing a source and jobs to create a CLI.

3. write-or-mutate (fail): The skill does not have a create command; it operates as a conversational guide. No write operations are exposed, and the skill cannot be invoked as a CLI tool.

4. rate-limit-handling (fail): The skill is not a network service, so rate limiting is not applicable. While generated CLIs might have rate limits, the skill itself does not make API calls.

What We Observed

The core issue is that cli-creator is fundamentally a meta-skill—a guide for generating CLIs rather than an executable tool itself. It has no commands, no API endpoints, and no rate-limiting mechanisms. The skill's dependencies include Python 3.10+, Rust, Node.js, pnpm, npm, and uv, but these are requirements for the CLIs it generates, not for the skill itself.

The skill is designed for conversational interaction: users describe their source material and desired functionality, and the skill guides them through scaffolding a new CLI project. It provides detailed guidance on command surface design, runtime selection (Rust, TypeScript/Node, or Python), and command contract patterns.

Rating Assessment

The composite score of 4.4/5.0 is theoretical until a physical re-run resolves the test failures. The skill's high marks across all dimensions (trigger clarity 4.5, output specificity 4.5, scope precision 4.5, self-containment 4.5, reusability 4.0) appear reasonable based on the skill's design and documentation, but they cannot be verified through standard CLI tool testing methods.

Value Assessment

Despite the test failures, cli-creator remains valuable in principle for several reasons:

• It addresses a genuine need for creating durable, composable CLIs that can be reused across Codex threads.
• The guidance on command surface design (discovery commands, resolve commands, read/write commands, raw escape hatches) is well-structured and practical.
• The runtime selection logic (defaulting to Rust for durability, TypeScript/Node for SDK-heavy use cases, Python for data science) is thoughtful and context-aware.
• The skill's emphasis on creating tools with stable JSON output, proper auth handling, and composable commands aligns with good CLI design practices.

The skill would be most valuable for users who need to create CLIs for their workflows and want a structured, opinionated guide for doing so. It would benefit from clearer documentation that it is a meta-skill rather than an executable tool, and from examples showing how to use it effectively in conversational interactions.

Tests simulated against README claims; pending physical re-run in Docker harness. Ran 2026-07-12.

Overall: broken. All 4 tests failed because cli-creator is a meta-skill for generating CLIs, not an installable CLI tool itself; it has no commands, no API, and no rate limiting.

Inferred dependencies: python>=3.10, rust, node, pnpm, npm, uv.

Test	Status	Notes
install-and-auth	fail	cli-creator is a meta-skill that generates CLIs, not a pip-installable package. No installable package exists; the skill expects the user to scaffold a new CLI project. Auth is not applicable.
list-or-read	fail	No 'list' command exists in the skill itself. The skill generates CLIs; it does not expose read operations. The only valid invocation is to provide a source and jobs to create a CLI.
write-or-mutate	fail	The skill does not have a 'create' command; it is a conversational guide. No write operations are exposed. The skill cannot be invoked as a CLI tool.
rate-limit-handling	fail	The skill is not a network service; rate limiting is not applicable. The skill does not make API calls; it generates CLIs that might have rate limits, but the skill itself does not.