course-builder
Use when turning "I want to teach X" into a defensible course skeleton — measurable outcomes (Bloom + ABCD), assessment that proves each one, sequenced modules, and an outcome×module×assessment matrix — for a workshop, bootcamp, cohort or onboarding track. NOT making one concept
Install
npx skills add https://github.com/ericrisco/rsc-harness/tree/main/skills/course-builder
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install ericrisco-rsc-harness@llmmart
git clone https://github.com/ericrisco/rsc-harness.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole ericrisco/rsc-harness collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Course Builder — The Course Is a Contract
Outcomes promise what the learner can DO. Assessment is the proof. Modules are the build order. If the three don't line up, you have a content dump, not a course. You design them backward — outcomes first, assessment second, content last — and you emit a matrix that makes the alignment checkable.
This skill owns the skeleton: the measurable outcomes, the assessment that certifies each one, the module sequence that builds toward them, and the alignment matrix tying it all together. It does not own the teaching of any one concept.
The litmus. If the question is "what must the learner be able to DO, how do we prove it, and in what order do we build it?" → here. If it's "how do I make THIS concept click and stick?" → ../course-storytelling/SKILL.md.
Route elsewhere when: auditing a finished course for gaps, redundancy, anachronisms with a written report → ../review/SKILL.md; turning a designed module into slides/decks → ../presentations/SKILL.md (+ ../design/SKILL.md for the visual system); sales/landing copy and launch emails → ../marketing/SKILL.md (you design the learning contract, never the pitch); fact-checking or sourcing the subject matter → ../research-ops/SKILL.md (you structure what the user already knows); a repeatable internal procedure with steps but no outcomes/assessment → ../sop-builder/SKILL.md (an SOP tells someone the steps; a course changes what they can DO and proves it).
The grounding gate (read first — STOP if unmet)
You cannot design a course backward without knowing the destination. Do not write a single outcome until you have all three. Why: an outcome scoped for a senior engineer in a 90-minute workshop is wrong for a beginner in a 12-week cohort — same topic, different contract.
- WHO — the learner and their current level (absolute beginner? practitioner? what can they already do?).
- WHAT transformation — what must they be able to DO at the end that they cannot do now.
- FORMAT + constraints — duration, modality (live cohort vs self-paced), group size, prerequisites, certification stakes.
If a 02-DOCS/wiki/teaching/ profile exists (the convention shared with ../course-storytelling/SKILL.md), read it first and reuse it. Otherwise interview in ONE batch — ask all three at once, do not dribble questions. Incomplete grounding = STOP and ask. Depth, the interview script, and scope right-sizing by format → references/grounding-and-scoping.md.
The build workflow (one backward pass)
Run it in order. Do not jump to modules. UbD has exactly three stages in order — desired results → acceptable evidence → learning experiences (Wiggins & McTighe, Understanding by Design). Content designed first is a dump with no destination, and "I already have these slides, build a course around them" inverts the contract: outcomes decide what content survives.
Stage 0 Grounding gate WHO + WHAT transformation + FORMAT. Incomplete → STOP.
Stage 1 Outcomes 3–8 course-level outcomes. Measurable Bloom verb + ABCD.
Verb banlist enforced. Right-sized to format.
Stage 2 Assessment For EACH outcome, the evidence that proves it.
Verb-match the outcome. Place formative + summative.
Check content validity (covers the breadth of outcomes).
Stage 3 Modules + sequence One focus per module, prerequisite order. Each module
maps to >=1 outcome. No orphans, no unproven outcomes.
Stage 4 Alignment matrix Emit outcome x module x assessment. The checkable artifact.
Stage 5 QA gate Run scripts/verify.sh over the curriculum doc. Fix warnings
or justify them. Then hand modules to course-storytelling.
Stage 1 — Writing measurable outcomes
Pick the verb at the level the learner must actually operate. Recall ≠ build. The revised taxonomy (Anderson & Krathwohl, 2001) is six ascending levels of verbs, each supplying observable action verbs — a verb you can't observe, you can't assess.
Level What the learner does Sample verbs
Remember recall facts list, define, name, label, recall
Understand explain in own words explain, summarize, classify, compare
Apply use in a new situation apply, use, implement, solve, run
Analyze break apart, find relations analyze, differentiate, debug, diagram
Evaluate judge against criteria evaluate, critique, justify, prioritize, review
Create produce something new design, build, compose, construct, ship
Banlist — never "understand", "know", "learn about", "appreciate", "be aware", "be familiar". They name a private mental state, not an observable behavior. Replace with what the learner does: list, explain, build, debug, evaluate, design.
Every outcome is ABCD-complete — a bare verb without a condition and a degree is not yet assessable ("write code" vs "given a failing test, write code that makes it pass"):
[Audience] the learner
[Behavior] <Bloom verb> + <object>
[Condition] given <situation / tools / inputs>
[Degree] <criterion that counts as success>
Bad → Good (the banlist verb is the tell):
Bad: Students will understand REST APIs.
Good: Given a spec, the learner builds a REST endpoint that returns the correct
status code for 3 named error cases (400, 404, 500).
Bad: Learners will know SQL joins.
Good: Given two tables, the learner writes a query joining them that returns the
expected rows for 2 of 2 test cases.
Bad: Participants will appreciate good test design.
Good: Given a 20-line module, the learner writes 3 tests that cover the happy path
and 2 edge cases, all passing.
Full per-level verb tables, the banlist, more worked ABCD examples, and course-level vs module-level granularity → references/outcomes-and-blooms.md.
Stage 2 — Designing aligned assessment
Assessment is the proof, not an afterthought. For every outcome, ask: what would I have to SEE the learner do to believe they achieved it? — and make that the assessment. Place both kinds: an outcome with no evidence is a promise you never keep, and a course with only the final exam gives the learner no feedback loop.
Formative (during) Summative (at the end)
Job feedback, guide improvement certify mastery / accountability
Stakes low / none high — the grade, the cert
Examples quizzes, skill checklists, capstone project, final exam,
drafts, peer review, exit portfolio, graded build
tickets
Bloom fit Remember/Understand/Apply Apply/Analyze/Evaluate/Create
Two hard checks:
- Verb match (constructive alignment, Biggs). The assessment must require the same verb as the outcome. Outcome "build" → assessment is a build, not a multiple-choice quiz. Outcome "evaluate" → assessment asks for a judgement with justification, not recall. If the outcome says "design" and the quiz tests "recall", the assessment proves the wrong thing.
- Content validity. The set of assessments must cover the breadth of the outcomes — every outcome touched, none over-weighted into a vanity exam. Competency-based design maps this via the matrix and certifies demonstrated mastery, not seat time.
The full formative↔summative menu mapped to Bloom levels, the content-validity / blueprint checklist, and the verb-match rule worked end-to-end → references/assessment-design.md.
Stages 3–4 — Sequencing modules + the alignment matrix
Order modules by prerequisite (you can't build before you can run), give each one focus, and map each to ≥1 outcome. Then emit the matrix — this is the artifact scripts/verify.sh checks.
| Outcome | Module(s) | Assessment (F=formative, S=summative) |
|----------------------------------|------------------|---------------------------------------|
| O1 build a REST endpoint | M2, M3 | F: M2 quiz · S: capstone API |
| O2 debug a failing request | M4 | F: M4 debug drill · S: capstone API |
| O3 evaluate an API's error model | M5 | F: M5 peer review · S: capstone rubric|
Read the matrix two ways: down a column finds orphan modules (a module in no row → content the contract never asked for, so cut it or write the outcome it serves); across the outcome list finds unproven outcomes (an outcome with an empty assessment cell → design evidence). A complete matrix has no empty cells.
Decision table (branch only where the flow actually splits)
Situation Then
Live cohort Schedule synchronous formative checkpoints; peer
assessment is cheap and valuable.
Self-paced Formative must be self-graded/auto-graded (quizzes,
tests that run); no instructor in the loop.
Short workshop (<= half day) 1-2 outcomes, mostly Apply; one summative artifact, no exam.
Full course / bootcamp 5-8 outcomes spanning up to Create; staged formative +
a capstone summative.
Knowledge outcome Assess with explanation/application, not recognition alone.
Skill outcome Assess with a performance/build, never a quiz.
Attitude/disposition outcome Assess with reflection + observed behavior; hardest to
prove — keep few and honest.
Anti-patterns
| Bad | Why it fails | Do instead |
|---|---|---|
| Content-first: "build a course around my slides" | Inverts the contract; content with no destination | Write outcomes first; let them decide what content survives |
| Vanity outcome: "students will understand X" | Names a private state, not observable → unassessable | Use a measurable Bloom verb in ABCD form |
| Orphan module: a module mapped to no outcome | Content the contract never asked for | Cut it, or write the outcome it serves |
| Unproven outcome: an outcome with no assessment | A promise you never verify | Design evidence that requires the outcome's verb |
| Verb mismatch: outcome "design", quiz tests "recall" | Proves the wrong thing | Make the assessment require the outcome's verb |
| Summative-only: just a final exam | No feedback loop during learning | Place formative checkpoints throughout |
| Scope creep: bootcamp outcomes in a workshop | None are actually achievable in the time | Right-size outcome count to the format |
Stage 5 — QA gate and hand-off
scripts/verify.sh checks your curriculum's STRUCTURE and ALIGNMENT (measurable verbs, the matrix, proven outcomes, formative + summative). Fix its warnings or justify them.
Then hand each module to ../course-storytelling/SKILL.md to make the teaching land (epiphany, named models, grounded analogies). You built what and in what order; that skill makes it stick. The boundary is executable: this skill's verify.sh does not judge whether the teaching lands, and course-storytelling's checks narrative, not structure.
Files (rsc-harness)
-
evals
-
cases.yaml 4 KB
skill: course-builder should_trigger: - prompt: "I have 30 topics on Kubernetes scattered in a doc — turn them into a structured curriculum with modules and a sensible order." why: "Raw topic pile -> structured curriculum with sequenced modules. Core sequencing/scoping job." - prompt: "Write measurable learning outcomes for my intro Python course; right now I just have 'students will understand functions'." why: "Outcomes + measurability/Bloom, including rewriting a banlist verb ('understand') into an observable one." - prompt: "What should the final project and quizzes be for my data-analytics bootcamp, and do they actually prove what I claim students will be able to do?" why: "Assessment design plus alignment — does the evidence prove the outcomes (constructive alignment)." - prompt: "My course just ends with no real endpoint. What should a learner be able to DO when they finish, and how do I build toward that?" why: "NON-OBVIOUS: no Bloom/curriculum jargon, but it's pure backward design — outcomes-first, build toward the DO." - prompt: "Estructura el temario de mi curso de onboarding y dime qué evaluación demuestra cada objetivo." why: "Spanish: structure the curriculum and map assessment to each objective — outcomes + alignment." - prompt: "Map my existing modules to outcomes — I want to check the course is actually aligned and nothing is orphaned." why: "Alignment matrix / constructive-alignment check; orphan-module and unproven-outcome detection is the emitted artifact." - prompt: "Dissenya el pla docent d'un taller de 2 hores sobre Git per a principiants i quina avaluació demostra cada objectiu." why: "Catalan: design the syllabus for a short workshop and the assessment per objective — grounding + right-sized scope + alignment." should_not_trigger: - prompt: "This module on indexing is correct but students nod and forget it — make it land with a story and a sticky name." route_to: course-storytelling why: "Narrative landing of a single concept (epiphany, sticky name), not the skeleton/structure." - prompt: "Audit module 14 of my finished course for gaps, redundancy, and outdated references, with a written report." route_to: review why: "Diagnostic content-audit pass over finished material, not design-from-intent." - prompt: "Turn my designed module into a polished slide deck with speaker notes." route_to: presentations why: "Slides/visuals from an already-designed module, not the learning structure." - prompt: "Write the landing-page copy and launch emails to sell my course." route_to: marketing why: "Sales/launch copy, not the learning contract." - prompt: "Document the exact repeatable steps our support team follows to process a refund." route_to: sop-builder why: "A repeatable procedure with steps but no outcomes/assessment — an SOP, not a course." capability: - scenario: > A user gives a learner profile (junior dev who can write small scripts but has never built a web API) and a goal ("ship a small REST API"), in a 2-week self-paced format, and asks course-builder to design the course. must_include: - "Works backward: defines measurable outcomes (Bloom verbs in ABCD form) BEFORE any module or content." - "Bans vanity verbs — no 'understand'/'know'; outcomes are observable ('build', 'deploy', 'debug')." - "Designs assessment that PROVES each outcome, with BOTH formative checkpoints and a summative artifact." - "Verb-matches: the assessment requires the same verb as the outcome (constructive alignment), not a recall quiz for a 'build' outcome." - "Sequences modules in prerequisite order, each module mapped to >=1 outcome (no orphan modules, no unproven outcomes)." - "Respects self-paced format: formative checks self-grade (auto-graded quiz / a test that runs); right-sizes outcome count to 2 weeks." - "Emits an outcome x module x assessment alignment matrix as the artifact." - "Hands off narrative landing to course-storytelling rather than doing the storytelling here." -
README.md 1.3 KB
# Eval harness — `course-builder` This is an **agent-run** eval, not an automated grader. `cases.yaml` has three blocks: `should_trigger` (7), `should_not_trigger` (5), and `capability` (1 scenario with a rubric). To run it, load **only** `course-builder` into a fresh agent (no sibling skills, so a miss can't be masked), paste each `should_trigger` / `should_not_trigger` prompt verbatim over 3–5 trials, and record whether the agent invokes the skill — `should_trigger` should fire, `should_not_trigger` should stay quiet and ideally route to the named sibling. Aim for ≥90% correct decisions across trials; a prompt that misses on a majority of its trials is a real defect — fix the description/triggers in `SKILL.md`, don't loosen the case. For the `capability` scenario, run it once and score the output against the `must_include` rubric by hand: did it work backward (outcomes first), ban vanity verbs, verb-match assessment to outcomes with both formative and summative, sequence modules with no orphans/unproven outcomes, emit the alignment matrix, and hand storytelling off rather than doing it? Grading is judgement, not grep — `scripts/verify.sh` only checks the greppable structure of a finished curriculum doc (banlist verbs, matrix presence, proven outcomes, formative+summative); the rubric here is the real bar.
-
-
references
-
assessment-design.md 3.7 KB
# Assessment design Assessment is the proof that an outcome was achieved — not a grade-generating afterthought. The backward-design question for every outcome is: *what would I have to SEE the learner do to believe this outcome is met?* That sight IS the assessment. This file is the formative↔summative menu, the content-validity check, and the verb-match rule worked end to end. ## Formative vs summative — different jobs ```text Formative (DURING learning) Summative (AT THE END) Purpose feedback; guide improvement certify mastery; accountability Stakes low / none high — the grade, the certificate When throughout each module after the module / the course Owner of action the learner adjusts the institution decides Examples quizzes, skill checklists, capstone project, final exam, drafts + revision, peer review, portfolio, graded build, defense exit tickets, self-checks ``` A course needs both. Summative-only gives the learner no chance to course-correct; formative-only never certifies the contract was met. ## Assessment type mapped to Bloom level Match the assessment to where the outcome's verb sits. A quiz can't prove "build". ```text Outcome level Formative options Summative options Remember flashcards, recall quiz section of a written exam Understand explain-back, concept map short-answer / explanation exam Apply guided exercise, lab drill applied problem set, practical task Analyze debug drill, code review case analysis, diagnostic task Evaluate peer review w/ rubric critique + justification, defense Create draft + feedback cycles capstone build / portfolio / project ``` ## The verb-match rule, worked Constructive alignment (Biggs): the verb in the outcome must be the verb the learner *performs* in the assessment. Walk it through: ```markdown Outcome: Given a spec, the learner BUILDS a REST endpoint that returns the correct status code for 3 named error cases. (verb = build, level = Create) Wrong assessment: A 10-question multiple-choice quiz on HTTP status codes. Why wrong: the learner RECALLS (Remember). Recall ≠ build. It proves a different, lower outcome — the contract is unverified. Right assessment: The learner ships a working endpoint; a test run hits all 3 error cases and asserts the status codes. (verb performed = build) ``` If you find yourself assessing with a quiz an outcome that says "design" / "build" / "evaluate", that's the mismatch anti-pattern — fix the assessment, not the outcome. ## Content-validity / blueprint check Competency-based design certifies *demonstrated mastery*, not seat time, and the assessment set must have **content validity**: it covers the breadth of the outcomes and aligns with them. Build a one-row-per-outcome blueprint and confirm: ```text [ ] Every course-level outcome has at least one assessment that requires its verb. [ ] No outcome is over-weighted (one vanity exam carrying the whole grade). [ ] No outcome is under-assessed (touched once, in passing, never certified). [ ] Both formative AND summative evidence exist across the course. [ ] The summative assessment(s) together cover all course-level outcomes. [ ] Self-paced? Every formative check grades itself (auto-graded / a test that runs). ``` When the blueprint has no gaps and no over-weighting, the assessment plan is content-valid and the outcomes are provable. That blueprint feeds straight into the alignment matrix in SKILL.md. -
grounding-and-scoping.md 4.3 KB
# Grounding & scoping Backward design starts from a destination. If you don't know WHO is travelling, WHAT they must be able to DO at the end, and the FORMAT/constraints of the trip, every outcome you write is a guess. This is the hard gate — incomplete grounding means STOP and ask, do not design on assumption. ## The three things you must know 1. **WHO — the learner and current level.** Not a demographic; a *capability baseline*. What can they already do today? Absolute beginner who has never opened a terminal? Practitioner who ships daily but never wrote a test? The baseline sets the floor of your first outcome. 2. **WHAT transformation.** State it as a DO, not a topic. "They can deploy a containerized service to production and roll it back" — not "Docker". The gap between the baseline and this is the course. 3. **FORMAT + constraints.** Duration (90 min / 1 day / 6 weeks / 12 weeks), modality (live cohort vs self-paced vs hybrid), group size, prerequisites you can assume, and stakes (does completion certify anything?). These bound how many outcomes are honest. ## The one-batch interview Ask all of it at once. Dribbling one question per message wastes the learner's patience and yours. ```text Before I design anything, I need three things: 1. WHO is this for, and what can they already DO today (their starting point)? 2. By the end, what must they be able to DO that they can't do now? (a verb, not a topic) 3. FORMAT: how long, live cohort or self-paced, group size, any prerequisites, and does finishing certify anything? If you have a teaching profile in 02-DOCS/wiki/teaching/, point me at it and I'll reuse it. ``` If any answer is a topic ("they should know React"), push back once: *"What should they be able to DO with React that they can't now?"* Topics don't scope; verbs do. ## The shared teaching profile (02-DOCS/wiki/teaching/) This is the **same persistence convention** `course-storytelling` uses. If the workspace follows the project harness, look for `02-DOCS/wiki/teaching/` first: - `02-DOCS/wiki/teaching/learner-profile.md` — WHO + baseline (reusable across both skills). - `02-DOCS/wiki/teaching/<course>-outcomes.md` — the outcomes you write here. - `02-DOCS/wiki/teaching/<course>-matrix.md` — the alignment matrix you emit. Reuse the learner profile if it exists rather than re-interviewing. When you write outcomes and the matrix, persist them there so `course-storytelling` can pick up the same grounding for the hand-off. Note in the file that the profile is shared, so neither skill clobbers the other's section. Each persisted `02-DOCS/wiki/teaching/*.md` file is an OKF v0.1 wiki article: open it with YAML frontmatter carrying a non-empty `type:` (`type: course-outline`), then the H1 and body. Use standard markdown links between articles (`[Outcomes](./<course>-outcomes.md)`), never wikilinks. Mirror the harness `wiki-article-template.md` shape: ```markdown --- type: course-outline title: <Course> — Outcomes description: The measurable, ABCD-complete learning outcomes for <course>. tags: [course, learning-outcomes] timestamp: YYYY-MM-DDTHH:MM:SSZ topic: teaching status: stable --- # <Course> — Outcomes > Sources: {user interview, YYYY-MM-DD} ... ``` ## Right-sizing scope by format The single most common failure is too many outcomes for the time. Honest ceilings: ```text Format Outcomes Bloom ceiling Assessment shape Workshop (<= half day) 1-2 Apply one hands-on artifact, no exam Full-day workshop 2-3 Apply / Analyze one build + exit checklist Multi-week cohort 4-6 up to Evaluate staged formative + capstone Bootcamp (8-12+ weeks) 5-8 up to Create weekly formative + graded capstone Onboarding track 3-5 Apply on-the-job checklists + sign-off Self-paced course 3-6 up to Create auto-graded quizzes + a final build ``` Rules of thumb: - If you can't assess an outcome in the available time, it isn't an outcome for this course — it's the next course. Cut it or split the course. - Self-paced means every formative check must grade itself (auto-graded quiz, a test suite that runs). There is no instructor in the loop to give feedback. - A cohort buys you cheap, high-value peer assessment and synchronous checkpoints — use them. -
outcomes-and-blooms.md 4 KB
# Outcomes & Bloom's revised taxonomy An outcome is a promise about what the learner can DO. Two things make it defensible: it sits on a **measurable verb** from Bloom's revised taxonomy, and it is **ABCD-complete** so it's actually assessable. This file is the verb reference, the banlist, and worked examples. ## Bloom's revised taxonomy (Anderson & Krathwohl, 2001) The 2001 revision turned Bloom's original nouns into **verbs** and reordered the top two levels. Six ascending levels — pick the verb at the level the learner must operate at. Recall is not build. ```text Level 1 Remember recall, list, define, name, label, identify, state, recognize, repeat Level 2 Understand* explain, summarize, classify, compare, describe, paraphrase, interpret Level 3 Apply apply, use, implement, execute, run, solve, demonstrate, compute Level 4 Analyze analyze, differentiate, debug, diagram, distinguish, examine, deconstruct Level 5 Evaluate evaluate, critique, justify, prioritize, defend, review, assess, recommend Level 6 Create design, build, compose, construct, develop, ship, formulate, generate ``` \* "Understand" is a LEVEL NAME, not an outcome verb. Write the *verb* at that level ("explain", "compare") — never the bare word "understand" in an outcome. See the banlist below. ## The verb banlist These name a private mental state no one can observe, so they can't be assessed. Replace every one. ```text Banned Why Use instead (observable) understand can't see "understanding" explain, apply, predict, classify know can't see "knowing" list, define, recall, use learn (about) describes activity, not result produce, demonstrate, solve appreciate a feeling, not a behavior justify, critique, prioritize be aware of awareness is invisible identify, flag, distinguish be familiar familiarity is unmeasurable describe, locate, recognize ``` A quick test: can two independent graders watch the learner and agree on whether they did it? If not, the verb is too vague — go down to a concrete action. ## The ABCD template A bare verb is not yet an outcome. Add the conditions and the success criterion. ```text [A] Audience the learner / the participant [B] Behavior <Bloom verb> + <object> [C] Condition given <inputs / tools / situation> [D] Degree <what counts as success: count, accuracy, time, rubric> ``` Read as one sentence: *"Given <C>, the learner will <B> to <D>."* ## Worked examples ```markdown Bad: Students will understand functions in Python. Why: banlist verb, no condition, no degree — unassessable. Good: Given a problem statement, the learner writes a Python function with correct parameters and return value that passes 3 of 3 provided unit tests. [A] learner [B] writes a function [C] given a problem statement [D] passes 3/3 unit tests Bad: Learners will know how to handle errors. Good: Given an endpoint that can fail 3 ways, the learner adds error handling that returns the correct status code (400/404/500) for each, verified by a test run. Bad: Participants will appreciate clean architecture. Good: Given a 200-line module, the learner critiques its layering against 4 named criteria and proposes one concrete refactor with justification. ``` ## Course-level vs module-level outcomes - **Course-level outcomes** (3–8) are the contract: what the learner can DO when the whole course ends. They tend to sit at Apply→Create and are proven by the *summative* assessment. - **Module-level outcomes** are the steps that build toward a course outcome. They can sit lower (Remember/Understand/Apply) and are proven by *formative* checks. Each module-level outcome should ladder up to at least one course-level outcome — if it doesn't, the module is an orphan. Keep the course-level list short and the module-level list a build sequence beneath it. The matrix (see assessment-design.md and SKILL.md) ties module work back to course outcomes so nothing floats.
-
-
scripts
-
verify.sh 10.4 KB
#!/usr/bin/env bash # # verify.sh — STRUCTURE/ALIGNMENT QA gate for the `course-builder` skill. # # WHAT IT DOES (read-only; never edits a file) # Static, network-free heuristics over your curriculum/syllabus source files. It # checks the SKELETON is defensible — not whether the teaching lands (that is # course-storytelling's job). Per scanned file it checks: # 1. Banlist verbs — outcome-ish lines using "understand / know / learn about / # appreciate / be aware / be familiar" (a vanity verb you cannot assess). # 2. Alignment matrix — a section mapping outcomes to modules/assessment. # 3. Proven outcomes — every declared outcome id (O1, LO2, Outcome 3...) is # referenced somewhere in an assessment context (flag unproven outcomes). # 4. Formative + summative — BOTH assessment kinds are present (flag if missing). # 5. Modules present — module sections exist and reference outcomes (orphans). # # Every finding is a WARNING by default (course design is judgement, not pass/fail). # Use --strict to turn any warning into a failure (exit 1) so CI can gate on it. # A missing tool is reported yellow and SKIPPED — never a failure. # # HOW TO RUN (inside YOUR project, not the skills repo) # ./verify.sh # scan ./ for curriculum sources (*.md, *.mdx, *.txt) # ./verify.sh --path course # scan a subdirectory # ./verify.sh --strict # treat any warning as a failure (exit 1) # # EXIT CODES # 0 clean, or warnings only without --strict # 1 a real failure, or --strict with any warning # 2 bad usage # # Runs on stock macOS bash 3.2: no mapfile, no associative arrays, arrays are # initialised and only expanded when non-empty under `set -u`. set -euo pipefail # --- portability: runs on stock macOS bash 3.2 ------------------------------ if [ -z "${BASH_VERSION:-}" ]; then printf 'This script requires bash (any version >= 3.2). Run: bash %s\n' "$0" >&2 exit 2 fi # --- color helpers (no escape codes when not a TTY) ------------------------- if [ -t 1 ]; then RED=$'\033[31m'; GREEN=$'\033[32m'; YELLOW=$'\033[33m'; NC=$'\033[0m' else RED=''; GREEN=''; YELLOW=''; NC='' fi ok_count=0; skip_count=0; warn_count=0; fail_count=0 ok() { printf '%s[ ok ]%s %s\n' "$GREEN" "$NC" "$*"; ok_count=$((ok_count + 1)); } skip() { printf '%s[skip]%s %s\n' "$YELLOW" "$NC" "$*"; skip_count=$((skip_count + 1)); } warn() { printf '%s[warn]%s %s\n' "$YELLOW" "$NC" "$*"; warn_count=$((warn_count + 1)); } fail() { printf '%s[fail]%s %s\n' "$RED" "$NC" "$*"; fail_count=$((fail_count + 1)); } usage() { # print the header comment block (lines 2..43), stripping the leading "# " sed -n '2,43p' "$0" | sed 's/^# \{0,1\}//' } # --- arg parse -------------------------------------------------------------- SCAN_PATH="." STRICT=0 while [ $# -gt 0 ]; do case "$1" in --path) SCAN_PATH="${2:?--path needs a value}"; shift 2 ;; --strict) STRICT=1; shift ;; -h|--help) usage; exit 0 ;; *) printf '%sUnknown argument: %s%s\n\n' "$RED" "$1" "$NC"; usage; exit 2 ;; esac done if [ ! -e "$SCAN_PATH" ]; then printf '%sPath not found: %s%s\n' "$RED" "$SCAN_PATH" "$NC"; exit 2 fi have() { command -v "$1" >/dev/null 2>&1; } # count_lines <file>: number of lines in a file; 0 if missing/empty. count_lines() { if [ -s "$1" ]; then wc -l < "$1" 2>/dev/null | tr -dc '0-9' else printf '0' fi } # find curriculum source files (markdown / text). Prints one path per line. # Excludes the conventional 02-DOCS wiki/raw so we lint the CURRICULUM, not the profile. list_curricula() { find "$SCAN_PATH" \ \( -name '*.md' -o -name '*.mdx' -o -name '*.txt' \) \ -type f \ ! -path '*/node_modules/*' \ ! -path '*/.git/*' \ ! -path '*/02-DOCS/*' \ 2>/dev/null || true } # file_has <file> <pattern>: 0 if the (case-insensitive) ERE pattern appears in the file. file_has() { grep -iqE "$2" "$1" 2>/dev/null; } # --- ensure we have a searcher ---------------------------------------------- if ! have grep; then skip "grep not found — cannot scan curriculum sources; install grep" printf '\nok=%d skip=%d warn=%d fail=%d\n' "$ok_count" "$skip_count" "$warn_count" "$fail_count" exit 0 fi # --- gather curriculum files ------------------------------------------------ TMPDIR_V="$(mktemp -d 2>/dev/null || printf '/tmp/verify-cb.%s' "$$")" mkdir -p "$TMPDIR_V" 2>/dev/null || true cleanup() { rm -rf "$TMPDIR_V" 2>/dev/null || true; } trap cleanup EXIT DOCS="$TMPDIR_V/docs" list_curricula > "$DOCS" 2>/dev/null || true n_docs="$(count_lines "$DOCS")" if [ "$n_docs" -eq 0 ]; then skip "no curriculum sources (*.md, *.mdx, *.txt) found under $SCAN_PATH" printf '\nok=%d skip=%d warn=%d fail=%d\n' "$ok_count" "$skip_count" "$warn_count" "$fail_count" exit 0 fi printf 'Scanning %s curriculum file(s) under: %s\n\n' "$n_docs" "$SCAN_PATH" # --- heuristic markers (case-insensitive ERE) ------------------------------- # A "curriculum-ish" file is one that talks about outcomes/objectives at all. # Files with no outcome language are skipped silently from the alignment checks. OUTCOME_LANG_RE='outcome|objective|resultado|objetiu|objectiu|learning goal' BANLIST_RE='\b(understand|understands|understanding|knows?|know about|learn about|learns about|appreciate|appreciates|be aware|are aware|be familiar|familiar with)\b' MATRIX_RE='alignment matrix|outcome.*(module|assessment)|module.*outcome|matrix' FORMATIVE_RE='\bformativ|quiz|checklist|exit ticket|draft|peer review|self.?check|practice' SUMMATIVE_RE='\bsummativ|capstone|final (project|exam)|portfolio|certif|graded (build|project)' MODULE_RE='^#{1,6}.*\bmodule\b|^#{1,6}.*\bunit\b|^#{1,6}.*\bweek\b|\bmódulo\b|\bmòdul\b' # An outcome ID like O1 / LO2 / Outcome 3 / Objetivo 1 (at most-ish line starts). OUTCOME_ID_RE='\b(O|LO|OBJ)[- ]?[0-9]+\b|\boutcome[- ]?[0-9]+\b|\bobjetivo[- ]?[0-9]+\b|\bobjectiu[- ]?[0-9]+\b' # per-file result buckets BANHITS="$TMPDIR_V/banhits"; : > "$BANHITS" NO_MATRIX="$TMPDIR_V/no_matrix"; : > "$NO_MATRIX" NO_FORM="$TMPDIR_V/no_form"; : > "$NO_FORM" NO_SUMM="$TMPDIR_V/no_summ"; : > "$NO_SUMM" NO_MODULE="$TMPDIR_V/no_module"; : > "$NO_MODULE" UNPROVEN="$TMPDIR_V/unproven"; : > "$UNPROVEN" n_relevant=0 while IFS= read -r f; do [ -z "$f" ] && continue file_has "$f" "$OUTCOME_LANG_RE" || continue # not a curriculum doc; skip quietly n_relevant=$((n_relevant + 1)) # 1. banlist verbs on lines that look like outcomes/objectives. # An outcome line either carries outcome/objective language or an outcome ID # (O1/LO2/Outcome 3/Objetivo 1...). Flag the file if any such line uses a banlist verb. if grep -inE "$OUTCOME_LANG_RE|$OUTCOME_ID_RE" "$f" 2>/dev/null \ | grep -iqE "$BANLIST_RE"; then printf '%s\n' "$f" >> "$BANHITS" fi # 2. alignment matrix / mapping present file_has "$f" "$MATRIX_RE" || printf '%s\n' "$f" >> "$NO_MATRIX" # 4. formative AND summative markers file_has "$f" "$FORMATIVE_RE" || printf '%s\n' "$f" >> "$NO_FORM" file_has "$f" "$SUMMATIVE_RE" || printf '%s\n' "$f" >> "$NO_SUMM" # 5. module sections exist file_has "$f" "$MODULE_RE" || printf '%s\n' "$f" >> "$NO_MODULE" # 3. proven outcomes: each outcome ID must also appear near assessment language. # Heuristic: collect distinct outcome IDs; for each, check it co-occurs in an # assessment-ish line (formative/summative/assess). If a file has IDs but none in # an assessment context, flag it as possibly-unproven. ids="$(grep -ioE "$OUTCOME_ID_RE" "$f" 2>/dev/null | tr '[:lower:]' '[:upper:]' \ | tr -d ' -' | sort -u)" if [ -n "$ids" ]; then proven_any=0 for id in $ids; do # normalise file IDs the same way, then look for the id on an assessment line if grep -iE "$FORMATIVE_RE|$SUMMATIVE_RE|\bassess" "$f" 2>/dev/null \ | tr '[:lower:]' '[:upper:]' | tr -d ' -' | grep -qF "$id"; then proven_any=1 fi done [ "$proven_any" -eq 0 ] && printf '%s\n' "$f" >> "$UNPROVEN" fi done < "$DOCS" || true # read returns 1 at EOF; harmless under set -e if [ "$n_relevant" -eq 0 ]; then skip "no files contain outcome/objective language — nothing to check for alignment" printf '\nok=%d skip=%d warn=%d fail=%d\n' "$ok_count" "$skip_count" "$warn_count" "$fail_count" exit 0 fi printf 'Found %s file(s) with outcome/objective language.\n\n' "$n_relevant" report_files() { label="$1"; listfile="$2" n="$(count_lines "$listfile")" if [ "$n" -gt 0 ]; then warn "$label ($n file(s)):" head -n 8 "$listfile" | sed 's/^/ /' if [ "$n" -gt 1 ]; then warn_count=$((warn_count + n - 1)); fi # warn() already counted 1 else ok "no $label" fi } # --- report ------------------------------------------------------------------ report_files "outcomes using a vanity/banlist verb (use a measurable Bloom verb)" "$BANHITS" report_files "curricula with no alignment matrix (emit outcome x module x assessment)" "$NO_MATRIX" report_files "curricula missing formative assessment (add during-learning checks)" "$NO_FORM" report_files "curricula missing summative assessment (add an end-of-course artifact)" "$NO_SUMM" report_files "curricula with no module/unit/week sections (sequence the build)" "$NO_MODULE" report_files "outcomes not referenced in any assessment context (unproven outcomes)" "$UNPROVEN" # --- markdownlint (optional) ------------------------------------------------- if have markdownlint; then ml_out="$TMPDIR_V/mdlint" # shellcheck disable=SC2046 # word-splitting the file list is intended here markdownlint $(cat "$DOCS") > "$ml_out" 2>&1 || true if [ -s "$ml_out" ]; then warn "markdownlint findings in curriculum sources:" head -n 10 "$ml_out" | sed 's/^/ /' else ok "markdownlint clean on curriculum sources" fi else skip "markdownlint not found — skipping markdown lint (npm i -g markdownlint-cli)" fi # --- summary ----------------------------------------------------------------- printf '\nok=%d skip=%d warn=%d fail=%d\n' "$ok_count" "$skip_count" "$warn_count" "$fail_count" cat <<'EOF' Note: every finding above is a heuristic WARNING — course design is judgement, not pass/fail. This gate checks STRUCTURE and ALIGNMENT (measurable verbs, the matrix, proven outcomes, formative + summative), NOT whether the teaching lands — that is course-storytelling's job. Review each finding: fix the skeleton or justify it. Re-run with --strict to gate CI. EOF if [ "$fail_count" -gt 0 ]; then exit 1; fi if [ "$STRICT" -eq 1 ] && [ "$warn_count" -gt 0 ]; then exit 1; fi exit 0
-
-
SKILL.md 11.8 KB
--- name: course-builder description: "Use when turning \"I want to teach X\" into a defensible course skeleton — measurable outcomes (Bloom + ABCD), assessment that proves each one, sequenced modules, and an outcome×module×assessment matrix — for a workshop, bootcamp, cohort or onboarding track. NOT making one concept land emotionally with story or analogy (that is course-storytelling)." tags: [course, curriculum, instructional-design, learning-outcomes, assessment] recommends: [course-storytelling, presentations, sop-builder] origin: risco --- # Course Builder — The Course Is a Contract *Outcomes promise what the learner can DO. Assessment is the proof. Modules are the build order. If the three don't line up, you have a content dump, not a course. You design them backward — outcomes first, assessment second, content last — and you emit a matrix that makes the alignment checkable.* This skill owns the **skeleton**: the measurable outcomes, the assessment that certifies each one, the module sequence that builds toward them, and the alignment matrix tying it all together. It does not own the *teaching* of any one concept. **The litmus.** If the question is *"what must the learner be able to DO, how do we prove it, and in what order do we build it?"* → here. If it's *"how do I make THIS concept click and stick?"* → `../course-storytelling/SKILL.md`. Route elsewhere when: auditing a *finished* course for gaps, redundancy, anachronisms with a written report → `../review/SKILL.md`; turning a designed module into slides/decks → `../presentations/SKILL.md` (+ `../design/SKILL.md` for the visual system); sales/landing copy and launch emails → `../marketing/SKILL.md` (you design the learning contract, never the pitch); fact-checking or sourcing the subject matter → `../research-ops/SKILL.md` (you structure what the user already knows); a repeatable internal procedure with steps but no outcomes/assessment → `../sop-builder/SKILL.md` (an SOP tells someone the steps; a course changes what they can DO and proves it). ## The grounding gate (read first — STOP if unmet) You cannot design a course backward without knowing the destination. Do not write a single outcome until you have all three. Why: an outcome scoped for a senior engineer in a 90-minute workshop is wrong for a beginner in a 12-week cohort — same topic, different contract. 1. **WHO** — the learner and their *current* level (absolute beginner? practitioner? what can they already do?). 2. **WHAT transformation** — what must they be able to DO at the end that they cannot do now. 3. **FORMAT + constraints** — duration, modality (live cohort vs self-paced), group size, prerequisites, certification stakes. If a `02-DOCS/wiki/teaching/` profile exists (the convention shared with `../course-storytelling/SKILL.md`), read it first and reuse it. Otherwise interview in ONE batch — ask all three at once, do not dribble questions. **Incomplete grounding = STOP and ask.** Depth, the interview script, and scope right-sizing by format → `references/grounding-and-scoping.md`. ## The build workflow (one backward pass) Run it in order. Do not jump to modules. UbD has exactly three stages in order — desired results → acceptable evidence → learning experiences (Wiggins & McTighe, *Understanding by Design*). Content designed first is a dump with no destination, and "I already have these slides, build a course around them" inverts the contract: outcomes decide what content survives. ```text Stage 0 Grounding gate WHO + WHAT transformation + FORMAT. Incomplete → STOP. Stage 1 Outcomes 3–8 course-level outcomes. Measurable Bloom verb + ABCD. Verb banlist enforced. Right-sized to format. Stage 2 Assessment For EACH outcome, the evidence that proves it. Verb-match the outcome. Place formative + summative. Check content validity (covers the breadth of outcomes). Stage 3 Modules + sequence One focus per module, prerequisite order. Each module maps to >=1 outcome. No orphans, no unproven outcomes. Stage 4 Alignment matrix Emit outcome x module x assessment. The checkable artifact. Stage 5 QA gate Run scripts/verify.sh over the curriculum doc. Fix warnings or justify them. Then hand modules to course-storytelling. ``` ## Stage 1 — Writing measurable outcomes Pick the verb at the level the learner must actually operate. Recall ≠ build. The revised taxonomy (Anderson & Krathwohl, 2001) is six ascending levels of *verbs*, each supplying observable action verbs — a verb you can't observe, you can't assess. ```text Level What the learner does Sample verbs Remember recall facts list, define, name, label, recall Understand explain in own words explain, summarize, classify, compare Apply use in a new situation apply, use, implement, solve, run Analyze break apart, find relations analyze, differentiate, debug, diagram Evaluate judge against criteria evaluate, critique, justify, prioritize, review Create produce something new design, build, compose, construct, ship ``` **Banlist — never "understand", "know", "learn about", "appreciate", "be aware", "be familiar".** They name a private mental state, not an observable behavior. Replace with what the learner *does*: list, explain, build, debug, evaluate, design. Every outcome is **ABCD-complete** — a bare verb without a condition and a degree is not yet assessable ("write code" vs "given a failing test, write code that makes it pass"): ```text [Audience] the learner [Behavior] <Bloom verb> + <object> [Condition] given <situation / tools / inputs> [Degree] <criterion that counts as success> ``` Bad → Good (the banlist verb is the tell): ```markdown Bad: Students will understand REST APIs. Good: Given a spec, the learner builds a REST endpoint that returns the correct status code for 3 named error cases (400, 404, 500). Bad: Learners will know SQL joins. Good: Given two tables, the learner writes a query joining them that returns the expected rows for 2 of 2 test cases. Bad: Participants will appreciate good test design. Good: Given a 20-line module, the learner writes 3 tests that cover the happy path and 2 edge cases, all passing. ``` Full per-level verb tables, the banlist, more worked ABCD examples, and course-level vs module-level granularity → `references/outcomes-and-blooms.md`. ## Stage 2 — Designing aligned assessment Assessment is the proof, not an afterthought. For every outcome, ask: *what would I have to SEE the learner do to believe they achieved it?* — and make that the assessment. Place **both** kinds: an outcome with no evidence is a promise you never keep, and a course with only the final exam gives the learner no feedback loop. ```text Formative (during) Summative (at the end) Job feedback, guide improvement certify mastery / accountability Stakes low / none high — the grade, the cert Examples quizzes, skill checklists, capstone project, final exam, drafts, peer review, exit portfolio, graded build tickets Bloom fit Remember/Understand/Apply Apply/Analyze/Evaluate/Create ``` Two hard checks: - **Verb match** (constructive alignment, Biggs). The assessment must require the *same* verb as the outcome. Outcome "build" → assessment is a build, not a multiple-choice quiz. Outcome "evaluate" → assessment asks for a judgement with justification, not recall. If the outcome says "design" and the quiz tests "recall", the assessment proves the wrong thing. - **Content validity.** The set of assessments must cover the *breadth* of the outcomes — every outcome touched, none over-weighted into a vanity exam. Competency-based design maps this via the matrix and certifies demonstrated mastery, not seat time. The full formative↔summative menu mapped to Bloom levels, the content-validity / blueprint checklist, and the verb-match rule worked end-to-end → `references/assessment-design.md`. ## Stages 3–4 — Sequencing modules + the alignment matrix Order modules by **prerequisite** (you can't build before you can run), give each **one focus**, and map each to ≥1 outcome. Then emit the matrix — this is the artifact `scripts/verify.sh` checks. ```markdown | Outcome | Module(s) | Assessment (F=formative, S=summative) | |----------------------------------|------------------|---------------------------------------| | O1 build a REST endpoint | M2, M3 | F: M2 quiz · S: capstone API | | O2 debug a failing request | M4 | F: M4 debug drill · S: capstone API | | O3 evaluate an API's error model | M5 | F: M5 peer review · S: capstone rubric| ``` Read the matrix two ways: down a column finds **orphan modules** (a module in no row → content the contract never asked for, so cut it or write the outcome it serves); across the outcome list finds **unproven outcomes** (an outcome with an empty assessment cell → design evidence). A complete matrix has no empty cells. ## Decision table (branch only where the flow actually splits) ```text Situation Then Live cohort Schedule synchronous formative checkpoints; peer assessment is cheap and valuable. Self-paced Formative must be self-graded/auto-graded (quizzes, tests that run); no instructor in the loop. Short workshop (<= half day) 1-2 outcomes, mostly Apply; one summative artifact, no exam. Full course / bootcamp 5-8 outcomes spanning up to Create; staged formative + a capstone summative. Knowledge outcome Assess with explanation/application, not recognition alone. Skill outcome Assess with a performance/build, never a quiz. Attitude/disposition outcome Assess with reflection + observed behavior; hardest to prove — keep few and honest. ``` ## Anti-patterns | Bad | Why it fails | Do instead | |-----|--------------|------------| | Content-first: "build a course around my slides" | Inverts the contract; content with no destination | Write outcomes first; let them decide what content survives | | Vanity outcome: "students will understand X" | Names a private state, not observable → unassessable | Use a measurable Bloom verb in ABCD form | | Orphan module: a module mapped to no outcome | Content the contract never asked for | Cut it, or write the outcome it serves | | Unproven outcome: an outcome with no assessment | A promise you never verify | Design evidence that requires the outcome's verb | | Verb mismatch: outcome "design", quiz tests "recall" | Proves the wrong thing | Make the assessment require the outcome's verb | | Summative-only: just a final exam | No feedback loop during learning | Place formative checkpoints throughout | | Scope creep: bootcamp outcomes in a workshop | None are actually achievable in the time | Right-size outcome count to the format | ## Stage 5 — QA gate and hand-off `scripts/verify.sh` checks your curriculum's STRUCTURE and ALIGNMENT (measurable verbs, the matrix, proven outcomes, formative + summative). Fix its warnings or justify them. Then hand **each module** to `../course-storytelling/SKILL.md` to make the teaching land (epiphany, named models, grounded analogies). You built *what* and *in what order*; that skill makes it stick. The boundary is executable: this skill's verify.sh does not judge whether the teaching lands, and `course-storytelling`'s checks narrative, not structure.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.