Files
blocker/docs/TESTING.md
T
SellPL 5073c7321d docs: add ADR log, architecture, limitations, permissions, perf, testing
The documentation set CLAUDE.md sections 2, 6, 8 and 10.1 call for, written as
records of decisions actually made rather than as a description of intent.

**DECISIONS.md** — 17 ADRs. Two are deliberately left open and say so: ADR-6
(the desktop UI toolkit, which section 3 requires be decided on measurement and
which needs the owner's call, partly because Slint is GPL-3.0) and ADR-17 (the
licence, which is a legal commitment nobody should make on the owner's behalf).
Where CLAUDE.md gives a reason for a decision, that reason is recorded; where it
does not, the file says the reason is unrecorded rather than inventing one.

**LIMITATIONS.md** — all eight section 6 limits plus the section 1.5.4 safety
guarantees, stated as facts about the operating systems. The words
"unbypassable", "tamper-proof", "guaranteed" and "secure" appear nowhere, per
sections 6.1 and 6.7. Ends with a separate list of shortfalls in the *current
code* — storage has no I/O, the golden tables cannot yet be parsed, LockState
always returns Normal — so a known gap in this implementation is never mistaken
for a platform limit.

**ARCHITECTURE.md** — the layering, the purity contract and its mechanical
enforcement, why `next_wakeup` must be armed over inactive schedules too, the
overnight weekday-of-start rule, and the end-to-end flow for one block event.

**PERF.md** — targets and results as separate tables, so an aspiration is never
read as a measurement. The one real number recorded (0.279 MB stripped daemon) is
labelled as nearly meaningless, since the daemon is two print statements. Lists
the common causes of idle-wakeup regressions.

**PERMISSIONS.md** — per platform, what each permission is for and what degrades
without it. Flags the two invasive ones (Android accessibility, extension host
permissions) rather than burying them, and records that unprivileged eBPF is
*assumed* unavailable here because the sysctls were never read.

**TESTING.md** — the golden-table contract, the add-the-case-first rule, and why
clock tampering needs code fixtures instead of data.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-10-01 15:22:24 +03:00

196 lines
7.9 KiB
Markdown

# Testing
## What the golden tables are for
CLAUDE.md section 7 requires that the same decision table runs against the core
and serves as the parity contract for all platforms.
The point is not to test the engine — unit tests do that. It is to make
cross-platform divergence a **build failure** rather than a bug report from
whichever platform nobody was testing that week.
```
golden/*.json ──┬──▶ core::evaluate ──▶ must match `expect`
├──▶ Linux identity mapping ──▶ must match `expect`
└──▶ Android identity mapping ──▶ must match `expect`
```
Each platform implements a mapping from its own app identity (Android package
name, Linux desktop-file id, Windows exe path) onto a `Target`. The engine is
identical. If the mappings disagree about what an app *is*, a table case catches it.
## The rule that keeps tables meaningful
**A case is added before the engine changes.**
A table written after the code records current behaviour. It can confirm nothing
changed and can never catch a regression, because it was fitted to whatever the
code happened to do.
So a PR that changes engine behaviour is reviewed on one question: *is there a
case added or updated in this same commit?* If not, the change is undocumented
behaviour, and that is a review failure, not a nit.
## Format
JSON, one file per concern, under `tests/golden/`. The full type contract is in
`tests/src/golden.rs`; that file is written before any parse layer exists, so M1
implements against a fixed format.
```json
{
"concern": "overnight",
"cases": [
{
"name": "mon 22:00-06:00 window still active at tue 03:00",
"covers": "7:overnight time ranges",
"schedules": [
{ "id": "s1", "name": "Night", "mode": "Blocklist",
"targets": [{"type": "app", "app": "com.example.social"}],
"conditions": [{"type": "Time", "days": [0],
"start": "22:00", "end": "06:00"}] }
],
"input": { "civil_time": "2026-10-06T03:00:00+02:00" },
"expect": {
"blocked_apps": [{"app": "com.example.social", "schedule": "s1"}],
"next_wakeup": "2026-10-06T06:00:00+02:00"
}
}
]
}
```
Two conventions that matter:
- **`days` is Monday = 0**, matching `time::Weekday`.
- **`start > end` means overnight.** No `crosses_midnight` flag — the comparison
already encodes it.
- **`civil_time` always carries an explicit UTC offset**, so a DST case is
unambiguous.
- **`next_wakeup: "none"`** is a legal, meaningful expectation. It asserts the
platform may sleep forever, which is the zero-idle-wakeup budget made
testable.
## Mandatory coverage
Every entry in `golden::MANDATORY_COVERAGE` must be claimed by at least one
`covers` field, and a test asserts it — so a scenario cannot silently lose its
test when someone reorganises files.
| Tag | Scenario |
|---|---|
| `7:overnight time ranges` | Windows spanning midnight, including the weekday-of-start bug |
| `7:DST changes` | Spring-forward (a local time that does not exist) and fall-back (an ambiguous one) |
| `7:timezone changes` | Same instant, different local offset |
| `7:clock tampering` | System date moved; a lock must not shorten |
| `7:strictest-wins` | Overlapping schedules, blocklist and allowlist together |
| `7:allowlist mode` | Complement semantics, and the empty-inventory fail-open case |
| `7:invalid condition combos` | Each forbidden pairing from section 1.2 |
| `7:strict mode lock rules` | Section 1.4: no editing, pausing, or uninstalling |
| `1.2:time+usage rejected` | |
| `1.2:time+launch rejected` | |
| `1.2:usage+launch rejected` | |
| `1.2:usage+location rejected` | |
| `1.2:launch+location rejected` | |
| `1.5.4:never-block list is unblockable` | |
| `1.5.4:worst case 24h` | |
## Tables cannot express the passage of time
Most cases are pure data. Clock tampering is not: proving that moving the system
date never shortens a lock requires actually moving a clock, which no JSON file
can express.
Those cases live as code fixtures in `tests/src/purity.rs` instead, and the two
coexist deliberately:
- **Tables describe decisions.** Given a state, what is the decision?
- **Fixtures describe time.** Given that time moved, does the lock behave?
Current fixtures, all of which pass:
| Fixture | What it proves |
|---|---|
| `backwards_wall_clock_never_shortens_a_lock` | Setting the date back 3 days does not shorten a 12h lock |
| `wall_clock_correction_respects_both_bounds` | A skew extends, and clamps to the 24h cap |
| `an_undisturbed_clock_costs_nothing` | An honest clock adds no waiting |
| `reboot_resumes_conservatively` | A reboot does not reset a lock |
| `worst_case_disable_path_is_capped` | No level exceeds 24h |
| `never_block_list_is_unblockable` | The safety list is protected and does not over-match |
| `engine_cannot_block_a_never_blocked_app` | The engine itself enforces it, not just the list |
| `dependency_purity` | `blocker-core` still has zero dependencies |
## Engine unit tests
Alongside the tables, `core/src/engine.rs` carries tests for the logic that is
hard to express as data — overnight windows, weekday indexing, boundaries, the
allowlist complement, and `next_wakeup` behaviour.
`weekday_indexing_is_monday_zero` is the anchor test. If weekday arithmetic breaks,
every other overnight test is quietly testing the wrong day, so this one asserts
its own calendar dates directly.
Two tests exist specifically to protect the idle budget:
- `wifi_schedule_declares_no_time_wakeup` — a schedule only an OS event can change
must not arm a timer.
- `inactive_schedule_yields_no_wakeup` — a schedule that is inactive now but starts
later must still arm a wakeup. This is the subtle direction: over-sleeping means
the schedule never fires.
## Property tests
To be added in M1, once the tables exist. Candidates, in rough priority order:
- `time_window_contains` never panics for any `days`/`start`/`end` combination.
- `evaluate` is deterministic: same input, same output, always.
- A never-blocked target never appears in `blocked_apps`, for any schedule set.
- `next_wakeup`, when present, is strictly in the future relative to `now`.
- Strictest-wins is monotonic: adding a schedule never unblocks anything.
## Platform testing
### Linux
The X11 path is testable headless with Xvfb, which makes it the CI-exercised
backend. The KWin path cannot be — it needs a real Plasma session — so its
testing is manual, against the checklist below.
Manual, on a real session:
- [ ] Focusing a blocked native Wayland app raises the block screen within 150 ms
- [ ] Focusing a blocked XWayland app does the same
- [ ] Focusing an allowed app shows nothing
- [ ] The daemon survives a suspend/resume cycle
- [ ] The daemon survives logout/login
- [ ] Killing the daemon restarts it, and the tamper log records it
- [ ] The backend actually selected is the one expected on this session type, and
the troubleshooting screen says which and why
- [ ] With KWin scripting disabled, the daemon falls back and says so
- [ ] Idle over 10 minutes: no wakeups, RSS under 15 MB
### Android
Instrumented tests for the AccessibilityService path, which cannot run on a
desktop JVM. M3-Android.
### Manual checklist, per OS
The full checklist lives in `docs/TESTING.md` and grows with each milestone. Every
entry maps to a section 1 requirement, so an uncovered requirement shows as an
uncovered checklist line.
## Running
```sh
scripts/ci.sh # everything, stop at first failure
scripts/ci.sh --keep-going # everything, summarise all failures
cargo test --workspace # just the tests
```
A milestone is not done until its acceptance checks pass **and** its performance
numbers are recorded in `PERF.md` (section 7).
If a check fails, it gets reported and fixed. Weakening a test to make it pass is
prohibited by section 10.6, and a test deleted to make a build green is the same
mistake by another route.