coverage-analysis
Project-wide code coverage and CRAP (Change Risk Anti-Patterns) score analysis for .NET projects. Calculates CRAP scores per method and surfaces risk hotspots — complex code with low coverage that is dangerous to modify. Use to diagnose why coverage is stuck or plateaued, identif
Install
npx skills add https://github.com/dotnet/skills/tree/main/plugins/dotnet-test/skills/coverage-analysis
claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install dotnet-skills@llmmart
git clone https://github.com/dotnet/skills.git
The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole dotnet/skills collection as a plugin from our marketplace. Git is the plain clone.
Skill manifest
Coverage Analysis
Purpose
Explain what .NET coverage evidence proves, reconcile target arithmetic, and identify the code blocking progress. Add complexity/CRAP ranking only when the user explicitly asks for risk hotspots, CRAP, priorities by risk, or refactoring safety.
When to Use
Use this skill for interpreting supplied .NET line/branch/condition evidence, coverage gaps and plateaus, target arithmetic, or explicit project-wide coverage-backed risk analysis.
When Not to Use
- Named method, class, or file CRAP/refactoring-safety analysis — use the
crap-scoreskill instead - Static source-to-test pairing or listing files with no tests — use
find-untested-sources - Behavioral or pseudo-mutation gaps in existing tests — use
test-gap-analysis - Test trait/category distributions or coverage shape by test type — use
test-tagging - Writing or generating tests — this skill identifies where tests are needed, not write them
- General test execution unrelated to coverage or CRAP analysis
- Only collecting .NET coverage or printing a raw percentage with no diagnosis — use
run-tests; use native tooling for non-.NET coverage collection or analysis. Interpreting .NET line/branch gaps remains in scope here
Inputs
| Input | Required | Default | Description |
|---|---|---|---|
| Project/solution path | No | Current directory | Path to the .NET solution or project |
| Line coverage threshold | No | 80% | Minimum acceptable line coverage |
| Branch coverage threshold | No | 70% | Minimum acceptable branch coverage |
| Existing Cobertura path | No | Discover only if not supplied | Preferred input; never rerun tests when usable |
| CRAP threshold | No | 30 | Used only for explicit risk/CRAP requests |
| Hotspot count | No | 3 | Explicit risk requests only; cap at 10 unless the user asks for more |
Discover optional inputs from the workspace. Do not ask for a project path when the current directory or a supplied report is sufficient.
Choose the smallest matching path
| User intent | Required work | Do not do |
|---|---|---|
| Explain a supplied excerpt, condition, or summary | Answer directly from the supplied evidence | Tools, CRAP, discovery, report files |
| Interpret a supplied Cobertura path or diagnose a plateau | Read that report, reconcile totals, name all material gaps, answer directly | Rerun tests, install tools, compute CRAP, or generate files unless explicitly requested |
| Rank risk hotspots, compute project-wide CRAP, or assess refactoring safety | Use the supplied/existing report, read references/guidelines.md, and compute CRAP before ranking |
Coverage-only ranking or a full report template unless requested |
| Analyze coverage when no report exists | Invoke run-tests to collect coverage with the repository-compatible runner, then analyze the generated report |
Choose or execute a test command independently; CRAP unless risk was requested |
| Produce a full markdown/HTML/CSV report | First deliver the direct answer; then read references/output-format.md or references/report-generation.md |
Report generation before the answer |
Words such as analyze coverage, what is blocking coverage, or why is coverage stuck do not by themselves request CRAP. Explicit signals include risk hotspot, CRAP, complexity-weighted priority, safe to refactor, or an equivalent request to combine complexity with coverage.
Existing-data fast path
When the user supplies a coverage excerpt, summary, or valid Cobertura path:
- Treat it as authoritative input and start there.
- Do not discover the solution or test projects unless source mapping is necessary.
- Do not run
dotnet test, install ReportGenerator, add a coverage package, or readreferences/setup-discovery.mdorreferences/report-generation.md. - Do not write
coverage-analysis.mdor create a report directory unless the user requested a saved/full report. - For interpretation and plateau questions, parse only the evidence needed to
answer. For explicit project-wide risk requests, use the bundled scripts as
described in
references/guidelines.md.
A failed read/view operation is not proof that a named path does not exist. Classify the failure, then make one targeted existence probe and use a normalized path or alternate reader only for confirmed tool availability, transport, or path-normalization failures and only after verifying the canonical path remains inside the workspace. Stop on content-exclusion, permission/policy, workspace-boundary, or unknown failures. Report a missing path only when the independent probe also fails; do not broaden the search or substitute another artifact.
Collection path
Use this path only when no usable coverage evidence exists and the user asked for analysis that requires it.
- Read
references/setup-discovery.md. - Prefer existing Cobertura discovered under the requested root.
- If none exists, invoke
run-testsfor repository overlay, platform, runner, and command selection, and have it collect Cobertura once per entry point. - Analyze the resulting Cobertura. Compute CRAP only if risk analysis was explicitly requested.
When the workspace contains the project/test manifests needed by this path,
perform discovery directly and delegate collection to run-tests. Do not ask
the user to attach files or pre-run coverage that the current workspace lets
you inspect and generate.
Do not modify production code. Any incidental test-project change needed for
collection is owned by run-tests; include that change and its revert command
in the final analysis.
The automatic collection path is for SDK-style projects. For classic non-SDK or
packages.config projects, use only a repository-owned coverage command. If none
exists, stop and request Cobertura XML. Never migrate the project, inject an
SDK-style provider, create a wrapper project, or report substitute coverage from
another assembly.
Interpretation and arithmetic invariants
- A line hit proves execution, not both decision outcomes.
condition-coverage="50% (1/2)"proves one reported outcome ran, but not which one. Recommend forcing the opposite outcome. Without source, never invent likely predicates or claim whether true or false is missing. State that compound predicates need independently exercised operands and short-circuit combinations when applicable; do not infer the exact combinations without source or fuller XML.- Derive overall totals from Cobertura's covered/valid line counts. For target
T, required covered lines areceiling(valid lines × T). - Projected coverage is
(current covered lines + newly covered distinct lines) / valid lines. State assumptions such as fully covering a method. - When asked whether one member can reach a target, show its maximum projected total and at least one concrete sufficient combination of supplied members or line gains. If no supplied combination is sufficient, say so.
- For a multi-member target, name the exact combination and resulting covered / valid total; saying only "combine it with another member" is incomplete.
- Reconcile member gaps against project totals. Method line ranges can overlap or omit class-level lines, so do not sum method counts as project truth.
- Never call one member the sole, entire, or all remaining gap unless its distinct uncovered lines exactly reconcile with the project total and no other below-threshold member remains.
- Name every supplied or extracted below-threshold member, but keep detail proportional: lead with the blockers, summarize the remainder in one sentence or a compact table.
Response contract
Answer the user's question in the first 2–4 sentences.
- Excerpt or arithmetic question: one explanation plus the next test or member priority. No dashboard.
- Existing-report interpretation or plateau: overall line/branch coverage, blocking members, reconciled target impact, and 1–3 recommendations. Use at most one compact table.
- Explicit risk/CRAP request: top 3 actual hotspots by default, supporting complexity/coverage/CRAP values, remaining flagged count, and 1–3 priorities. Exclude fully covered low-risk methods from the hotspot table. Never exceed 10 rows unless the user requests a larger count. For refactoring safety, rank risky methods by CRAP rather than raw coverage alone, then name comparatively safe well-covered methods separately.
- Explicit full report request: read
references/output-format.md. Save the report only then.
Report only artifacts that exist. Do not announce inaccessible output paths or failed optional file writes when no file was requested.
Optional reports
HTML/CSV/markdown files are not part of normal analysis. Generate them only when the user explicitly requests report files or a CI artifact.
- Deliver the direct coverage/risk answer first.
- For a full markdown report, read
references/output-format.mdand save it. - For HTML/CSV, then read
references/report-generation.md. Do not install ReportGenerator before the direct answer, and do not retry a failed install.
Validation
- Confirm a supplied report was used without test execution or tool installation.
- Reconcile covered, valid, and uncovered line totals before projecting impact.
- Confirm every stated blocker comes from supplied or extracted evidence.
- If CRAP was requested, spot-check one score using
references/guidelines.md. - If files were requested, verify they exist before reporting their paths.
Common Pitfalls
- Existing report triggers setup work — stop. Analyze the named artifact first.
- Generic analysis triggers CRAP — stop. CRAP requires explicit risk intent.
- One method is called the entire gap — reconcile distinct uncovered lines and the full below-threshold set first.
- A small request gets a dashboard — scale down to a direct answer or one table.
- No Cobertura from collection — report the collection failure; do not invent substitute coverage.
- Compiler-generated method names — verify suspicious async, lambda, or local function names against source before presenting them as user-authored members.
- Test exit code 1 — coverage may still exist; proceed with a warning. Other nonzero build failures stop the collection path.
Files (skills)
-
references
-
guidelines.md 2.6 KB
# Explicit Risk/CRAP Analysis Read this file only when the user explicitly asks for project-wide risk hotspots, CRAP scores, complexity-weighted priorities, or refactoring safety. Do not read it for supplied-excerpt interpretation, plateau diagnosis, or target arithmetic alone. ## Compute the risk data Resolve the scripts relative to this skill's `SKILL.md`: ```powershell & "<skill-directory>/scripts/Compute-CrapScores.ps1" ` -CoberturaPath @(<all Cobertura paths>) ` -CrapThreshold <crap_threshold> ` -TopN <top_n> & "<skill-directory>/scripts/Extract-MethodCoverage.ps1" ` -CoberturaPath @(<all Cobertura paths>) ` -CoverageThreshold <line_threshold> ` -BranchThreshold <branch_threshold> ` -Filter below-threshold ``` `Compute-CrapScores.ps1` emits aggregate line/branch coverage, method counts, flagged counts, and sorted hotspots. `Extract-MethodCoverage.ps1` emits every below-threshold method. Use both for explicit project-wide risk work. CRAP is: `CRAP(m) = complexity² × (1 − lineCoverage)³ + complexity` A method at 100% coverage therefore has CRAP equal to its complexity. Use 30 as the default flagged threshold; treat 15–30 as moderate rather than catastrophic. ## Scale the output - Show the top 3 actual risk hotspots by default. - Exclude fully covered low-risk methods from the hotspot table. - State how many additional methods exceeded the threshold instead of listing them all. - Honor a user-supplied count up to 10. Exceed 10 only when explicitly requested. - For five or fewer below-threshold members, name all of them. For larger sets, show the top hotspots and summarize the remaining count and range. - Give 1–3 recommendations ordered by expected risk reduction. ## Prioritize - **HIGH** — both CRAP and coverage exceed their risk thresholds. - **MED** — either CRAP or coverage exceeds its threshold. - **LOW** — below coverage threshold but complexity is at most 2. Prefer complex uncovered critical paths (authentication, payment, data access, error handling). Deprioritize trivial getters, generated code, migrations, and configuration glue. Do not project a CRAP reduction from an arbitrary target without showing the assumption. Recalculate with the stated projected method coverage. Do not generate tests during analysis. Recommend focused test cases; implementation is a separate follow-up. ## Style - Lead with the risk verdict, not setup narration. - Quantify recommendations only from actual line/coverage evidence. - Use one compact hotspot table. Do not append the full report template unless the user requested a report. -
output-format.md 4.1 KB
# Output Format Read and use this template only when the user explicitly requests a full markdown coverage report. Normal interpretation, plateau, arithmetic, and hotspot requests use the scaled response contract in `SKILL.md` instead. For an explicit full report, copy the template below **verbatim** for all fixed elements (headings, table headers, emoji, symbols). Only replace `<placeholder>` values with actual data. Do not substitute emoji with text equivalents, do not change `·` to `-`, do not change `×` to `x`, and do not drop section emoji prefixes. ```markdown # Coverage Analysis - <ProjectName> | Metric | Value | |--------|-------| | **Date** | <YYYY-MM-DD> | | **Line Coverage** | <N>% | | **Branch Coverage** | <N>% | | **Risk Hotspots** | <N> (CRAP > <crap_threshold>) | | **Tests** | <N> passed · <N> failed | ## Summary | Metric | Value | Threshold | Status | |--------|-------|-----------|--------| | **Line Coverage** | <N>% | <line_threshold>% | ✅ / ❌ | | **Branch Coverage** | <N>% | <branch_threshold>% | ✅ / ❌ | | **Methods Analyzed** | <N> | — | — | | **Risk Hotspots** | <N> | 0 | ✅ / ⚠️ | | **Test Result** | <Passed / N tests failed> | — | ✅ / ⚠️ | > Coverage collected from **<N> of <M> test project(s)**. > Outputs saved to: `<coverageDir>/` (markdown summary + raw Cobertura XML). > *If HTML/CSV generation ran:* additional reports are at `<coverageDir>/reports/`. If any coverage provider package was added to test projects, include this note after the summary: > ℹ️ **Coverage provider package updates** > - `coverlet.collector` added to `<K>` project(s): `<TestProject1.csproj>`, `<TestProject2.csproj>` > - `Microsoft.Testing.Extensions.CodeCoverage` added to `<M>` project(s): `<TestProject3.csproj>` > > To revert: `git checkout -- <path-to-each-modified-csproj>` If all test projects already had a coverage provider, omit this note. --- ## 🔥 Risk Hotspots (Top <N> by CRAP Score) Methods flagged as high-risk: complex code with low test coverage that is dangerous to change. | Rank | Method | Class | File | Complexity | Coverage | CRAP Score | |------|--------|-------|------|-----------|---------|-----------| | 1 | `<method>` | `<class>` | `<file>` | <N> | <N>% | **<score>** | | … | … | … | … | … | … | … | > **CRAP Score** = `Complexity² × (1 − Coverage)³ + Complexity`. > Scores above <crap_threshold> are flagged. A score ≤ 5 is considered safe. --- ## 📋 Coverage Gaps by File Files below the line or branch coverage threshold, ordered by uncovered lines descending: | File | Line Coverage | Branch Coverage | Uncovered Lines | Priority | |------|--------------|----------------|----------------|---------| | `<file>` | <N>% | <N>% | <N> | 🔴 HIGH / 🟡 MED / 🟢 LOW | | … | … | … | … | … | --- ## 💡 Recommendations 1. **Write tests for the top risk hotspot first** — `<method>` in `<class>` has a CRAP score of <N> (complexity <N>, <N>% coverage). Reducing it to 80% coverage would drop the score to ~<projected>. 2. **Focus on `<file>`** — <N> uncovered lines, below threshold. <Brief reasoning.> 3. **<Up to 5 actionable items total, ordered by expected risk reduction.>** --- ## 📁 Reports | Report | Path | |--------|------| | Markdown summary (this file) | `<coverageDir>/coverage-analysis.md` | | Raw Cobertura XML | `<coberturaXmlPathsUsedForAnalysis>` | | HTML (browsable) | `<coverageDir>/reports/index.html` *or* `Not generated (optional — request HTML reports to enable)` | | Text summary | `<coverageDir>/reports/Summary.txt` *or* `Not generated` | | GitHub markdown | `<coverageDir>/reports/SummaryGithub.md` *or* `Not generated` | | CSV data | `<coverageDir>/reports/Summary.csv` *or* `Not generated` | ``` If ReportGenerator has not run, mark the HTML/Text/GitHub-markdown/CSV rows as `Not generated (optional — request HTML reports to enable)`. Do not invent paths for files that have not been produced. For **Raw Cobertura XML**, list the actual XML file path(s) used in analysis (for from-scratch runs this is typically under `<coverageDir>/raw/`; for existing-data runs this may be under `TestResults/` or another user-supplied location). -
report-generation.md 2 KB
# Coverage Analysis — ReportGenerator HTML/CSV reports Read this file **only** when the user explicitly asked for HTML/CSV reports and the direct coverage/risk answer has already been delivered. ## Verify or install ReportGenerator ```powershell $rgAvailable = $false $rgCommand = Get-Command reportgenerator -ErrorAction SilentlyContinue if ($rgCommand) { $rgAvailable = $true Write-Host "RG_INSTALLED:already-present" } else { $rgToolPath = Join-Path "<COVERAGE_DIR>" ".tools" dotnet tool install dotnet-reportgenerator-globaltool --tool-path $rgToolPath if ($LASTEXITCODE -eq 0) { $env:PATH = "$rgToolPath$([System.IO.Path]::PathSeparator)$env:PATH" $rgCommand = Get-Command reportgenerator -ErrorAction SilentlyContinue if ($rgCommand) { $rgAvailable = $true Write-Host "RG_INSTALLED:true (tool-path: $rgToolPath)" } else { Write-Host "RG_INSTALLED:false" Write-Host "RG_INSTALL_ERROR:reportgenerator-not-available" } } else { Write-Host "RG_INSTALLED:false" Write-Host "RG_INSTALL_ERROR:reportgenerator-not-available" } } Write-Host "RG_AVAILABLE:$rgAvailable" ``` If installation fails (no internet), keep `RG_AVAILABLE:false`, leave the existing user-facing summary as the final output, and note that HTML reports were skipped. ## Step 7: Generate HTML/CSV reports ```powershell $reportsDir = Join-Path "<COVERAGE_DIR>" "reports" if ($rgAvailable) { reportgenerator ` -reports:"<semicolon-separated COBERTURA paths>" ` -targetdir:$reportsDir ` -reporttypes:"Html;TextSummary;MarkdownSummaryGithub;CsvSummary" ` -title:"Coverage Report" ` -tag:"coverage-analysis-skill" Get-Content (Join-Path $reportsDir "Summary.txt") -ErrorAction SilentlyContinue } else { Write-Host "REPORTGENERATOR_SKIPPED:true" } ``` After report generation completes successfully, you may follow up with a short message pointing the user to the generated HTML report (one paragraph, no need to repeat the summary). -
setup-discovery.md 9.9 KB
# Coverage Analysis — setup and discovery Read this file only when the user did not supply usable coverage evidence and the request requires workspace discovery or fresh collection. Do not read or run these probes for a supplied excerpt or valid Cobertura path. ## Step 1: Locate the solution or project Given the user's path (default: current directory), find the entry point: ```powershell $root = "<user-provided-path-or-current-directory>" # Prefer solution file; fall back to project file $sln = Get-ChildItem -Path $root -Filter "*.sln" -Recurse -Depth 2 -ErrorAction SilentlyContinue | Select-Object -First 1 if ($sln) { Write-Host "ENTRY_TYPE:Solution"; Write-Host "ENTRY:$($sln.FullName)" } else { $project = Get-ChildItem -Path $root -Filter "*.csproj" -Recurse -Depth 2 -ErrorAction SilentlyContinue | Select-Object -First 1 if ($project) { Write-Host "ENTRY_TYPE:Project"; Write-Host "ENTRY:$($project.FullName)" } else { Write-Host "ENTRY_TYPE:NotFound" } } # Test projects: search path first, then git root, then parent $searchRoots = @($root) $gitRoot = (git -C $root rev-parse --show-toplevel 2>$null) if ($gitRoot) { $gitRoot = [System.IO.Path]::GetFullPath($gitRoot) } if ($gitRoot -and $gitRoot -ne $root) { $searchRoots += $gitRoot } $parentPath = Split-Path $root -Parent if ($parentPath -and $parentPath -ne $root -and $parentPath -ne $gitRoot) { $searchRoots += $parentPath } $testProjects = @() foreach ($sr in $searchRoots) { # Primary: match by .csproj content (test framework references) $testProjects = @(Get-ChildItem -Path $sr -Filter "*.csproj" -Recurse -Depth 5 -ErrorAction SilentlyContinue | Where-Object { $_.FullName -notmatch '([/\\]obj[/\\]|[/\\]bin[/\\])' } | Where-Object { (Select-String -Path $_.FullName -Pattern 'Microsoft\.NET\.Test\.Sdk|xunit|nunit|MSTest\.TestAdapter|"MSTest"|MSTest\.TestFramework|TUnit' -Quiet) }) if ($testProjects.Count -gt 0) { if ($sr -ne $root) { Write-Host "SEARCHED:$sr" } break } } # Fallback: match by file name convention if ($testProjects.Count -eq 0) { foreach ($sr in $searchRoots) { $testProjects = @(Get-ChildItem -Path $sr -Filter "*.csproj" -Recurse -Depth 5 -ErrorAction SilentlyContinue | Where-Object { $_.Name -match '(?i)(test|spec)' }) if ($testProjects.Count -gt 0) { if ($sr -ne $root) { Write-Host "SEARCHED:$sr" } break } } } Write-Host "TEST_PROJECTS:$($testProjects.Count)" $testProjects | ForEach-Object { Write-Host "TEST_PROJECT:$($_.FullName)" } # Project-system classification controls whether the automatic dotnet/provider path is safe. $classicTestProjects = @($testProjects | Where-Object { $text = Get-Content $_.FullName -Raw $hasSdk = $text -match '<Project[^>]+\bSdk\s*=' -or $text -match '<Sdk\b' $hasPackagesConfig = Test-Path (Join-Path $_.DirectoryName "packages.config") $hasClassicSignals = $text -match '\bToolsVersion\s*=' -or $text -match 'Microsoft\.(Common\.props|CSharp\.targets)' -or $text -match '<Compile\s+Include=' $hasPackagesConfig -or (-not $hasSdk -and $hasClassicSignals) }) Write-Host "CLASSIC_TEST_PROJECTS:$($classicTestProjects.Count)" $classicTestProjects | ForEach-Object { Write-Host "CLASSIC_TEST_PROJECT:$($_.FullName)" } $sdkTestProjects = @($testProjects | Where-Object { $classicTestProjects.FullName -notcontains $_.FullName }) Write-Host "SDK_TEST_PROJECTS:$($sdkTestProjects.Count)" $sdkTestProjects | ForEach-Object { Write-Host "SDK_TEST_PROJECT:$($_.FullName)" } # Resolve the test output root (where coverage-analysis artifacts will be written) if ($testProjects.Count -eq 0) { if ($gitRoot) { $testOutputRoot = $gitRoot } else { $testOutputRoot = $root } } elseif ($testProjects.Count -eq 1) { $testOutputRoot = $testProjects[0].DirectoryName } else { # Multiple test projects — find their deepest common parent directory $dirs = $testProjects | ForEach-Object { $_.DirectoryName } $common = $dirs[0] foreach ($d in $dirs[1..($dirs.Count-1)]) { $sep = [System.IO.Path]::DirectorySeparatorChar while (-not $d.StartsWith("$common$sep", [System.StringComparison]::OrdinalIgnoreCase) -and $d -ne $common) { $prevCommon = $common $common = Split-Path $common -Parent # Terminate if we can no longer move up (at filesystem root or no parent) if ([string]::IsNullOrEmpty($common) -or $common -eq $prevCommon) { $common = $null break } } } if ([string]::IsNullOrEmpty($common)) { # Fallback when no common parent directory exists (e.g., projects on different drives) if ($gitRoot) { $testOutputRoot = $gitRoot } else { $testOutputRoot = $root } } else { $testOutputRoot = $common } } Write-Host "TEST_OUTPUT_ROOT:$testOutputRoot" ``` - If `ENTRY_TYPE:NotFound` and SDK-style test projects were found → use the test projects directly as `dotnet test` entry points. - If `ENTRY_TYPE:NotFound` and classic test projects were found → use only the repository's documented coverage command; do not infer `dotnet test`. - If `ENTRY_TYPE:NotFound` and no test projects found → stop: `No .sln or test projects found under <path>. Provide the path to your .NET solution or project.` - If `TEST_PROJECTS:0` and `EXISTING_COBERTURA_COUNT` > 0 (Step 2b) → continue with existing Cobertura XML analysis (no `dotnet test` run). - If `TEST_PROJECTS:0` and `EXISTING_COBERTURA_COUNT` == 0 → stop: `No test projects found (expected projects with 'Test' or 'Spec' in the name), and no existing Cobertura XML was provided. Add a test project or provide a Cobertura file path.` - If `CLASSIC_TEST_PROJECTS` is nonzero and no existing Cobertura XML is found, search scripts/CI/docs for a repository-owned coverage command. Use it if it emits Cobertura. - If classic projects are the only test projects and no repository command exists, stop: `Classic non-SDK or packages.config test project detected. The automatic SDK-style coverage-provider path would modify this project incorrectly. Run the repository's supported coverage workflow and provide its Cobertura XML.` This is a hard stop: do not create or run a temporary SDK project against the classic source, because its coverage would belong to the substitute assembly, not the requested test project. - In a mixed solution, run automatic collection only for `SDK_TEST_PROJECTS`. Never run the solution entry point if it would include classic projects. Clearly label the result partial until repository-owned Cobertura data for the classic projects is also available. ## Step 2: Create the output directory ```powershell $coverageDir = Join-Path $testOutputRoot "TestResults" "coverage-analysis" if (Test-Path $coverageDir) { Remove-Item $coverageDir -Recurse -Force } New-Item -ItemType Directory -Path $coverageDir -Force | Out-Null Write-Host "COVERAGE_DIR:$coverageDir" ``` This step only manages the `TestResults/coverage-analysis/` subdirectory (skill-owned outputs). It must never delete user-supplied Cobertura files — those live one level up at `TestResults/coverage.cobertura.xml` (or wherever the user pointed). If the user provided a path that *is* `TestResults/coverage-analysis/...`, copy the file aside before this step recreates the directory. ## Step 2b: Discover or accept existing Cobertura XML (required for the existing-data path) If the user supplied a Cobertura XML path explicitly, use it. Otherwise probe well-known locations and any path the user mentioned: ```powershell # 1. Honor a user-supplied path first (highest priority) $coberturaFiles = @() if ($userSuppliedCoberturaPath -and (Test-Path $userSuppliedCoberturaPath)) { $coberturaFiles = @(Get-Item $userSuppliedCoberturaPath) } # 2. Otherwise scan TestResults/ at the repo/test root for any *.cobertura.xml if ($coberturaFiles.Count -eq 0) { $searchPaths = @( (Join-Path $testOutputRoot "TestResults"), (Join-Path $root "TestResults") ) | Where-Object { $_ -and (Test-Path $_) } | Select-Object -Unique foreach ($sp in $searchPaths) { $found = @(Get-ChildItem -Path $sp -Filter "*.cobertura.xml" -Recurse -ErrorAction SilentlyContinue | Where-Object { $_.FullName -notmatch '[/\\]coverage-analysis[/\\]raw[/\\]' }) if ($found.Count -gt 0) { $coberturaFiles = $found; break } } } Write-Host "EXISTING_COBERTURA_COUNT:$($coberturaFiles.Count)" $coberturaFiles | ForEach-Object { Write-Host "EXISTING_COBERTURA:$($_.FullName)" } ``` - If `EXISTING_COBERTURA_COUNT` > 0 → skip fresh collection and analyze these paths. - If `EXISTING_COBERTURA_COUNT` == 0 and all test projects are SDK-style → invoke `run-tests` to select the repository-compatible platform/provider command and collect Cobertura. - If `EXISTING_COBERTURA_COUNT` == 0 and only classic/packages.config projects exist → use a repository-owned coverage command that emits Cobertura; otherwise stop with the message above. - If `EXISTING_COBERTURA_COUNT` == 0 and both classic and SDK-style projects exist → ask `run-tests` to collect only for `SDK_TEST_PROJECTS` and mark the result partial until classic-project Cobertura is available. ## Step 2c: Recommend ignoring `TestResults/` ```powershell $pattern = "**/TestResults/" $gitRoot = (git -C $testOutputRoot rev-parse --show-toplevel 2>$null) if ($gitRoot) { $gitRoot = [System.IO.Path]::GetFullPath($gitRoot) } if ($gitRoot) { $gitignorePath = Join-Path $gitRoot ".gitignore" $alreadyIgnored = $false if (Test-Path $gitignorePath) { $alreadyIgnored = (Select-String -Path $gitignorePath -Pattern '^\s*(\*\*/)?TestResults/?\s*$' -Quiet) } if ($alreadyIgnored) { Write-Host "GITIGNORE_RECOMMENDATION:already-present" } else { Write-Host "GITIGNORE_RECOMMENDATION:$pattern" } } else { Write-Host "GITIGNORE_RECOMMENDATION:$pattern" } ```
-
-
scripts
-
Compute-CrapScores.ps1 6.6 KB · in bundle
-
Extract-MethodCoverage.ps1 7.9 KB · in bundle
-
-
SKILL.md 11.1 KB
--- name: coverage-analysis description: > Activation requires either supplied .NET coverage reports/percentages/line, branch, or condition metrics, or an explicit request to collect .NET coverage for analysis. USE FOR: interpreting that evidence, including Cobertura data, partial conditions, plateaus, target arithmetic, project-wide coverage-backed CRAP, and coverage-backed refactoring safety. Analyze supplied reports directly without rerunning tests or installing tools. DO NOT USE FOR: requests with neither coverage evidence nor explicit coverage-collection intent, including hypothetical change-survival questions (use test-gap-analysis); CRAP or refactoring safety for one named target (use crap-score); or requests owned by test-tagging, find-untested-sources, test-anti-patterns, run-tests, or code-testing-agent. license: MIT --- # Coverage Analysis ## Purpose Explain what .NET coverage evidence proves, reconcile target arithmetic, and identify the code blocking progress. Add complexity/CRAP ranking only when the user explicitly asks for risk hotspots, CRAP, priorities by risk, or refactoring safety. ## When to Use Use this skill for interpreting supplied .NET line/branch/condition evidence, coverage gaps and plateaus, target arithmetic, or explicit project-wide coverage-backed risk analysis. ## When Not to Use - **Named method, class, or file CRAP/refactoring-safety analysis** — use the `crap-score` skill instead - **Static source-to-test pairing or listing files with no tests** — use `find-untested-sources` - **Behavioral or pseudo-mutation gaps in existing tests** — use `test-gap-analysis` - **Test trait/category distributions or coverage shape by test type** — use `test-tagging` - **Writing or generating tests** — this skill identifies where tests are needed, not write them - **General test execution** unrelated to coverage or CRAP analysis - **Only collecting .NET coverage or printing a raw percentage with no diagnosis** — use `run-tests`; use native tooling for non-.NET coverage collection or analysis. Interpreting .NET line/branch gaps remains in scope here ## Inputs | Input | Required | Default | Description | |-------|----------|---------|-------------| | Project/solution path | No | Current directory | Path to the .NET solution or project | | Line coverage threshold | No | 80% | Minimum acceptable line coverage | | Branch coverage threshold | No | 70% | Minimum acceptable branch coverage | | Existing Cobertura path | No | Discover only if not supplied | Preferred input; never rerun tests when usable | | CRAP threshold | No | 30 | Used only for explicit risk/CRAP requests | | Hotspot count | No | 3 | Explicit risk requests only; cap at 10 unless the user asks for more | Discover optional inputs from the workspace. Do not ask for a project path when the current directory or a supplied report is sufficient. ## Choose the smallest matching path | User intent | Required work | Do not do | |-------------|---------------|-----------| | Explain a supplied excerpt, condition, or summary | Answer directly from the supplied evidence | Tools, CRAP, discovery, report files | | Interpret a supplied Cobertura path or diagnose a plateau | Read that report, reconcile totals, name all material gaps, answer directly | Rerun tests, install tools, compute CRAP, or generate files unless explicitly requested | | Rank risk hotspots, compute project-wide CRAP, or assess refactoring safety | Use the supplied/existing report, read `references/guidelines.md`, and compute CRAP before ranking | Coverage-only ranking or a full report template unless requested | | Analyze coverage when no report exists | Invoke `run-tests` to collect coverage with the repository-compatible runner, then analyze the generated report | Choose or execute a test command independently; CRAP unless risk was requested | | Produce a full markdown/HTML/CSV report | First deliver the direct answer; then read `references/output-format.md` or `references/report-generation.md` | Report generation before the answer | Words such as **analyze coverage**, **what is blocking coverage**, or **why is coverage stuck** do not by themselves request CRAP. Explicit signals include **risk hotspot**, **CRAP**, **complexity-weighted priority**, **safe to refactor**, or an equivalent request to combine complexity with coverage. ## Existing-data fast path When the user supplies a coverage excerpt, summary, or valid Cobertura path: - Treat it as authoritative input and start there. - Do not discover the solution or test projects unless source mapping is necessary. - Do not run `dotnet test`, install ReportGenerator, add a coverage package, or read `references/setup-discovery.md` or `references/report-generation.md`. - Do not write `coverage-analysis.md` or create a report directory unless the user requested a saved/full report. - For interpretation and plateau questions, parse only the evidence needed to answer. For explicit project-wide risk requests, use the bundled scripts as described in `references/guidelines.md`. A failed read/view operation is not proof that a named path does not exist. Classify the failure, then make one targeted existence probe and use a normalized path or alternate reader only for confirmed tool availability, transport, or path-normalization failures and only after verifying the canonical path remains inside the workspace. Stop on content-exclusion, permission/policy, workspace-boundary, or unknown failures. Report a missing path only when the independent probe also fails; do not broaden the search or substitute another artifact. ## Collection path Use this path only when no usable coverage evidence exists and the user asked for analysis that requires it. 1. Read `references/setup-discovery.md`. 2. Prefer existing Cobertura discovered under the requested root. 3. If none exists, invoke `run-tests` for repository overlay, platform, runner, and command selection, and have it collect Cobertura once per entry point. 4. Analyze the resulting Cobertura. Compute CRAP only if risk analysis was explicitly requested. When the workspace contains the project/test manifests needed by this path, perform discovery directly and delegate collection to `run-tests`. Do not ask the user to attach files or pre-run coverage that the current workspace lets you inspect and generate. Do not modify production code. Any incidental test-project change needed for collection is owned by `run-tests`; include that change and its revert command in the final analysis. The automatic collection path is for SDK-style projects. For classic non-SDK or `packages.config` projects, use only a repository-owned coverage command. If none exists, stop and request Cobertura XML. Never migrate the project, inject an SDK-style provider, create a wrapper project, or report substitute coverage from another assembly. ## Interpretation and arithmetic invariants - A line hit proves execution, not both decision outcomes. - `condition-coverage="50% (1/2)"` proves one reported outcome ran, but not which one. Recommend forcing the opposite outcome. Without source, never invent likely predicates or claim whether true or false is missing. State that compound predicates need independently exercised operands and short-circuit combinations when applicable; do not infer the exact combinations without source or fuller XML. - Derive overall totals from Cobertura's covered/valid line counts. For target `T`, required covered lines are `ceiling(valid lines × T)`. - Projected coverage is `(current covered lines + newly covered distinct lines) / valid lines`. State assumptions such as fully covering a method. - When asked whether one member can reach a target, show its maximum projected total and at least one concrete sufficient combination of supplied members or line gains. If no supplied combination is sufficient, say so. - For a multi-member target, name the exact combination and resulting covered / valid total; saying only "combine it with another member" is incomplete. - Reconcile member gaps against project totals. Method line ranges can overlap or omit class-level lines, so do not sum method counts as project truth. - Never call one member the **sole**, **entire**, or **all** remaining gap unless its distinct uncovered lines exactly reconcile with the project total and no other below-threshold member remains. - Name every supplied or extracted below-threshold member, but keep detail proportional: lead with the blockers, summarize the remainder in one sentence or a compact table. ## Response contract Answer the user's question in the first 2–4 sentences. - **Excerpt or arithmetic question:** one explanation plus the next test or member priority. No dashboard. - **Existing-report interpretation or plateau:** overall line/branch coverage, blocking members, reconciled target impact, and 1–3 recommendations. Use at most one compact table. - **Explicit risk/CRAP request:** top 3 actual hotspots by default, supporting complexity/coverage/CRAP values, remaining flagged count, and 1–3 priorities. Exclude fully covered low-risk methods from the hotspot table. Never exceed 10 rows unless the user requests a larger count. For refactoring safety, rank risky methods by CRAP rather than raw coverage alone, then name comparatively safe well-covered methods separately. - **Explicit full report request:** read `references/output-format.md`. Save the report only then. Report only artifacts that exist. Do not announce inaccessible output paths or failed optional file writes when no file was requested. ## Optional reports HTML/CSV/markdown files are not part of normal analysis. Generate them only when the user explicitly requests report files or a CI artifact. 1. Deliver the direct coverage/risk answer first. 2. For a full markdown report, read `references/output-format.md` and save it. 3. For HTML/CSV, then read `references/report-generation.md`. Do not install ReportGenerator before the direct answer, and do not retry a failed install. ## Validation - Confirm a supplied report was used without test execution or tool installation. - Reconcile covered, valid, and uncovered line totals before projecting impact. - Confirm every stated blocker comes from supplied or extracted evidence. - If CRAP was requested, spot-check one score using `references/guidelines.md`. - If files were requested, verify they exist before reporting their paths. ## Common Pitfalls - **Existing report triggers setup work** — stop. Analyze the named artifact first. - **Generic analysis triggers CRAP** — stop. CRAP requires explicit risk intent. - **One method is called the entire gap** — reconcile distinct uncovered lines and the full below-threshold set first. - **A small request gets a dashboard** — scale down to a direct answer or one table. - **No Cobertura from collection** — report the collection failure; do not invent substitute coverage. - **Compiler-generated method names** — verify suspicious async, lambda, or local function names against source before presenting them as user-authored members. - **Test exit code 1** — coverage may still exist; proceed with a warning. Other nonzero build failures stop the collection path.
Comments (0)
Sign in to join the conversation.
Reviews (0)
No reviews yet.
No comments yet.