A 200 that carried nothing
RuleVerify through the surface people use.
In one working day, four pieces of work were reported as done and were not. I went back through them expecting four different causes and found one.
The first was an API feature checked with a command that printed only the HTTP status. It printed 200. The response body was empty, and nobody had read it.
The second was wired to one endpoint and tested there. The product calls a different one, the streaming version, so the test passed on a path no user ever takes.
The third was a build of two services chained with a start command. The second build failed, the chain stopped, and the test ran against a stale copy of the first service.
The fourth was a change to a screen, declared finished three times without anyone opening a browser. The first version shipped a control that was invisible. The fix shipped one that covered the text underneath it. An element can be present in the page structure and still be invisible to a person.
Each check was real and each one passed. Every one of them tested something next to the feature: a status line, a sibling endpoint, a stale container, a tree of elements. Since then the rule has been to check the thing itself.
What I check now
- For a screen, I open it in a browser and look. Present in the page structure is not the same as visible to a person.
- For an API, I read the payload, and I call the endpoint the product calls.
- For configuration, I read the value back from the running service, never from the source file or the deploy log.
- For a count, I compare it with a figure computed somewhere else.