Test generation loop for GitHub Copilot

The recipe

For a codebase where tests are missing, weak, or inconsistent.

The failure mode this exists to prevent: asking for tests, receiving four hundred lines of plausible-looking test code, committing it, and discovering later that half of it never ran and the half that did asserts nothing meaningful. Generated tests are drafts. Execution is the gate.

1. If the project has no test harness, start with /setupTests

Ask Copilot to recommend the framework that matches the language and the patterns already in the repo — not the one it likes best. If there's a sibling project with tests, point at it: matching an existing convention is worth more than picking the theoretically better framework.

Verify the harness works on a trivial test (assert 1 + 1 == 2) before generating anything real. Debugging a broken harness through generated tests is miserable.

2. Generate the smallest relevant tests with /tests

Scope it to one function, class, or bug fix. Select the code first. Do not ask for broad suite generation — a hundred tests you haven't read is a hundred assertions you haven't agreed to, and it takes longer to audit them than to have written them.

3. Run the tests immediately

A generated test that has never been run is a draft, not a test. Run them before you read them; the failures tell you where to focus.

4. If they fail for setup or syntax reasons, use /fixTestFailure

This is the right tool for wiring problems: imports, fixtures, configuration gaps, mock setup, obviously broken assertions.

It is the wrong tool for a test that fails because the production code is wrong. If the assertion is correct and the code isn't, you've found a bug — stop and treat it as one. Deleting the assertion converts a discovery into a silent regression.

Use /fixTestFailure to repair this failing test without changing the production behaviour. If the test is correct and the code is wrong, say so instead of fixing the test.

5. Re-run the exact same test command

Don't widen scope until the targeted slice is green. And read the diff /fixTestFailure produced — the common bad outcome is a test "repaired" by weakening its assertion into something that can't fail.

6. Audit what the tests actually assert

The step people skip, and the one that decides whether this was worth doing. For each generated test ask: would this fail if the function were wrong?

Delete on sight:

• Tests that assert the mock was called, and nothing about the result
• assertNotNull as the only assertion
• Tests that re-implement the function's logic in the assertion (they pass by construction)
• Tests asserting on log strings, dictionary ordering, or timestamps
• Four tests that are the same test with different variable names

Then apply the real check: break the function on purpose and re-run. If everything still passes, you have coverage, not tests. This takes thirty seconds and it is the only honest measure of a generated suite.

7. Add one edge case manually through chat

Add one more test for the closest edge case these tests are still missing. Keep it behaviour-focused, and explain what bug it would catch.

The explanation requirement is what stops it generating a seventh variation of the happy path.

Common gaps in generated tests: empty and boundary inputs, the error path, concurrent access, the "already exists" case, and anything involving time.

8. Only after the slice is stable, widen coverage

Move to the next function. Resist the urge to generate the whole suite now that it's working — the audit step doesn't scale, and skipping it is how you get back to where you started.

Prompts that work well

• Use /tests to generate focused tests for the selected method only.
• Use /fixTestFailure to repair this failing test without changing the production behavior.
• Which of these tests would still pass if the function returned a constant?
• Suggest the next highest-value edge case after these tests pass, and say what bug it catches.
• Rewrite these tests to assert on behavior rather than on implementation details.

Make it stick

In .github/copilot-instructions.md:

## Tests
- Framework: <yours>. Match the style of the nearest existing test file.
- Every test must be able to fail: no assert-not-null-only tests, no asserting
  on mock invocation without asserting on the result.
- Never change production code to make a test pass, or a test to accommodate
  production behavior, without saying so explicitly.
- Test names describe the behavior and the condition, not the method name.

Failure modes

• Tests pass on first run against known-broken code. They're asserting nothing. Go to step 6.
• Everything is mocked. A test where every dependency is a mock tests your mock configuration. Ask for one integration-level test that exercises the real path.
• /fixTestFailure keeps weakening assertions. Restate the rule and revert. If it happens twice, the production code is probably wrong and it's routing around the evidence.
• Coverage went up, confidence didn't. Coverage measures lines executed, not behaviour verified. The break-it-on-purpose check is the one that matters.
• The generated tests are brittle — they break on every harmless refactor. They're asserting on implementation. Rewrite them against the public behaviour.

Why it works

Copilot is genuinely good at drafting tests: the boilerplate, the fixtures, the arrange-act-assert shape. What it can't do is know which assertions matter. This loop puts execution and a human audit between "drafted" and "committed", which is exactly where the value is.