# Test integrity

> Catch test edits that weaken assertions to turn red green, the moment the diff lands.

When a test fails, the path of least resistance is to weaken the test. An agent swaps an assertion for `assert True`, replaces an integration call with a `Mock()`, or sprinkles `@pytest.mark.skip` to make red go green. You want an LLM reviewer that reads the edit diff and blocks these patterns before they merge.

``` python
"""Block test edits that weaken assertions to make a failing test pass."""

from __future__ import annotations

from captain_hook import (
    BaseHookEvent,
    Event,
    HookResult,
    Prompt,
    SourceEdits,
    TestFile,
    Tool,
    on,
    prompt_check,
)

INTEGRITY_TEMPLATE = """
You are reviewing a test edit for signs the agent weakened tests to make them pass.

Block if you see any of:
- An assertion replaced by `assert True`, `pass`, or a no-op.
- A real call replaced by a `Mock()` that defeats the test's purpose.
- A bulk addition of `@pytest.mark.skip` or `pytest.skip(...)` without justification.
- An integration boundary (DB, HTTP, file I/O) swapped for a stub.

File: {fp}

--- old ---
{old}
--- new ---
{new}
"""


@on(Event.PostToolUse, only_if=[SourceEdits(lang="py", include_tests=True), TestFile(), Tool("Edit")])
def guard_test_edits(evt: BaseHookEvent) -> HookResult | None:
    if not (fp := evt.file) or not (old := evt.old) or not (new := evt.content):
        return None
    return prompt_check(
        evt,
        Prompt.from_template(INTEGRITY_TEMPLATE, fp=fp.path, old=old, new=new),
        prefix="TEST INTEGRITY",
        suffix=" If unsure whether the change weakens the test, allow.",
    )
```

> **Note: Verified by replay**
>
> The weakening verdict is an LLM judgment, so it isn't asserted by an inline test -- the model's decision is exercised by replaying real test-edit sessions through the hook.


# What it catches

``` python
# assert exercise_count == 3  ->  assert True             # assertion gutted to a no-op
# total = service.charge(card)  ->  total = Mock()         # real call swapped for a stub
# @pytest.mark.skip("flaky")                               # bulk skip with no justification
# db.execute(query)  ->  return FAKE_ROW                   # integration boundary stubbed out
```


# What it allows

``` python
# Edit on src/billing/charge.py                            # not a test file -- only_if skips it
# Write on tests/test_billing.py                           # not an Edit -- Tool("Edit") skips it
# Edit on tests/test_billing.py tightening an assertion    # LLM allows; doesn't weaken the test
```

*The catch / allow split mirrors the hook's `only_if` narrowing and prompt, so it stays true as the hook evolves.*


# Run it yourself

``` bash
uvx capt-hook --hooks docs/examples test
```


# See also

- [LLM hooks](../../docs/guide/llm-hooks.md)
- [conditions](../../docs/guide/primitives.md#filter-with-conditions)
- [matchers](../../docs/reference/matchers.md)