How to Build a Skill From Scratch With the Skill-Creator Tool
There’s a skill for building skills. That sounds like a joke, but it’s the fastest way to go from “I keep explaining this the same way every time” to a file Claude reads on its own. Here’s what actually happens when you use it, start to finish.
Scaffolding the draft
Skill-creator doesn’t start by asking you to write a SKILL.md. It starts by asking what you’re trying to get Claude to do, when it should kick in, what the output should look like, and whether the result is the kind of thing you can check objectively (a file, a specific format, a fixed set of steps) or the kind of thing that’s more a matter of taste (tone, style, art). That distinction matters later, because it decides whether you’ll be able to grade the output with a script or you’ll just have to look at it and say “yeah, that’s right.”
Once that’s settled, it writes an actual skill folder: a SKILL.md with a name and a description up top, then the instructions underneath, and optionally a scripts folder for anything that should run as code instead of getting reasoned through fresh every time. The description matters more than anything else in the file. It’s the only part that’s in context at all times, and it’s the whole reason Claude decides to open the rest.
Writing a trigger description that fires
This is the part people underrate. A description that accurately explains what the skill does can still fail to trigger it, because Claude tends to under-use skills rather than over-use them. The skill creator’s own guidance is blunt about the fix: make the description a little pushy. Don’t just say what the skill does; spell out the specific situations that should count, even the ones where someone wouldn’t think to name the skill by name.
The difference looks like this. “How to build a dashboard to display data” is accurate and still weak. “Make sure to use this whenever the user mentions dashboards, data visualization, internal metrics, or wants to display any kind of data, even if they don’t say ‘dashboard'” gives Claude a lot more surface area to match against.
To actually test whether a description works, skill-creator builds a set of eval queries, a mix of prompts that should trigger the skill and prompts that shouldn’t. The should-not-trigger half is the one worth paying attention to. The easy negatives (“write a Fibonacci function” for a PDF skill) don’t tell you anything. The useful ones are near misses, requests that share a keyword or a domain with your skill but actually need something else. Those are what expose a description that’s too broad or too narrow. From there it’s a real optimization loop: split the queries into training and test sets, score the current description, propose a revision, re-score, repeat up to five times, and keep whichever version scored best on the held-out queries rather than the ones it trained on.
Running an eval instead of just eyeballing it
Before any of that description tuning, there’s a more basic question: does the skill actually produce a good result? Skill-creator answers that by running a handful of realistic test prompts twice, once with the skill available and once without, so you can see the actual delta rather than assume there is one.
The results don’t get dumped into the chat as a wall of text. They go into a viewer, one test case at a time, with the output rendered where possible and a spot to leave feedback on each one. There’s a second tab with the numbers: pass rate, how long each version took, how many tokens it burned. Assertions get written for anything objectively checkable, and left alone for anything that comes down to judgment, because forcing a pass/fail grade onto something like “does this sound right” just produces a fake sense of precision.
The feedback you leave there is what drives the next draft. Not a rewrite from scratch, a targeted fix aimed at whatever specifically went sideways, followed by another round of the same test prompts to see if it actually got better this time.
The part that’s easy to skip
It’s tempting to write a skill, run it once, decide it looks fine, and move on. The loop exists because “looks fine on the one example I already know by heart” and “works on a prompt I haven’t seen yet” are different claims. A skill that only survives the exact scenario you built it against isn’t really a skill yet; it’s a script with your name still attached.
The version worth keeping is the one that’s been wrong at least once, in a way you noticed and fixed. That’s what separates a skill from a really elaborate way of remembering what you meant to type.
