Method and definitions

How the lab records are produced, what the five outcomes mean, what is published and what is not, how to get the data, and how to cite it.

What this is

We take reports of broken WordPress sites from public forums, turn each one into a repeatable recipe, recreate the failure on our own test servers, and run it through the recovery station the way a customer would.

One record per case: what the site owner saw, which server setups reproduced it, the outcome on each setup for each connection mode, and, when the outcome was not "recovered", what we changed and the version that shipped it.

There are also capability sweeps: every problem type from the "What we fix" page is planted on one test site at once, and each run records "cleared" or "still present" for each problem type. Where a sweep result exists, the matrix cell uses it.

These are records from our test servers, for a given product version on a given day. They are not a promise about the result on any site. Other hosts, PHP versions, and WordPress versions can behave differently.

Lab records contain no customer data. Every case is recreated on our own test sites, and we link to the forum thread without copying posts or poster names.

Test servers

Four server setups. We publish the software layer only:

nginx + PHP-FPM
Apache + PHP-FPM
Apache + mod_php
Apache + php-cgi (mod_cgid)

All four run the same test site. After each run, the server is reset to that copy and compared against a snapshot taken before the run. When a new environment comes up, we add another server rather than changing the recipe.

How a case is run

Write the recipe first: how to break the site and how to confirm the breakage, following the symptoms described in the forum thread.
Recreate it on the test server that matches the environment, and record the HTTP status of the homepage and the login page. If no server matches, the case waits; the recipe is not changed to fit a server.
Open a case through the recovery station the way a customer would (payment is simulated for testing). The failure decides which connection modes run: if wp-admin cannot be reached, both SSH and FTP; if it can, all three.
Read the result: what the health check listed, what the recovery did, and the state of the site after write-back.
If the outcome is not "recovered", record a gap. After the recovery station is changed, run the same recipe again on the same server.

The five outcomes

"Recovered" is judged by the same re-check used in customer cases: the homepage, sampled inner pages, and page assets all have to load. The re-check is the re-verification referred to in Article 9(5) of the Terms of Service.

RecoveredA case was opened, the health check listed this problem, the recovery passed the re-check (homepage, sampled inner pages, page assets), was written back and passed the check after write-back (in WP admin mode, the recovered package was delivered), and every item in the recovery report was completed.

Declined at the pre-payment checkThe pre-payment check did not let the site in. No case was opened and nothing was charged.

Not detectedA case was opened, but the health check did not list this problem.

Not recoveredThe health check listed the problem, but the recovery could not be completed as defined in Article 9(5) of the Terms of Service. The recovery fee is refunded in full; the refund is carried out automatically by the system, with no request needed.

Case did not completeA case was opened but did not reach the health check or the recovery (for example, copying the site stalled). The health check fee is refunded in full, automatically.

ClearedIn a capability sweep, the planted fault for this problem type was gone after write-back.

Still presentIn a capability sweep, the planted fault for this problem type was still present after write-back. A gap row explains it.

What we publish and what we do not

Published

The public source of the case, the symptoms, and the WordPress and PHP versions of the test site.
The software layer of each server setup, and the HTTP status of the homepage and the login page after the failure.
For each run: the date (UTC, day only), server setup, connection mode, outcome, and product version.
Gaps: what we changed, the version it shipped in, and the re-test date and outcome.

Not published

Test server addresses, hostnames, data center, or machine size.
Times of day, how long each step took, or case numbers.
Details of how the fix works.
Any percentage, success rate, or summary claim. Readers can count the table themselves.

Before a record is published, code checks it and blocks IP addresses, host and machine names, machine sizes, data center names, case numbers, times of day and time measurements, percentages and success rates, promise wording, adjectives, non-English text, and HTML. Fix details and summary claims are reviewed by a person. Problems with the lab tooling itself (a test server that did not reproduce the failure, a stuck tool, a recipe that could not be built) are not recovery-station outcomes and are not listed.

When a gap is published

A run that did not end in "recovered" is published together with the fix. If the fix has not shipped when the record is published, the gap is marked "In progress". If a customer's case runs into the same gap in the meantime, the refund terms in Article 9 of the Terms of Service apply.

Corrections

Every edit after a record is published gets a line on the changes page. We fix mistakes, we note the fix, and we do not delete the old entry.

Disclosure

The recovery station under test is our own product. These pages carry no sponsorship and no affiliate links.

Data license and citation

The data files (CSV and JSON) are licensed under CC BY 4.0. The linked forum threads are not part of the data files. To cite, use "WP119 lab records" with the date you retrieved the data and the URL. Frozen versions (fixed date, file hashes, DOI) are listed under "Frozen versions" on the data page.

FIELD

FIELD	Meaning
`id / recipe`	Record id: the date the recipe was written plus a short name. It does not change. "recipe" is the name of the fault recipe; one recipe can be run on several days.
`kind`	"Forum case": one failure taken from a public forum thread. "Capability sweep": every problem type from the "What we fix" page planted on one test site at once, with a result per problem type.
`title / symptom`	What the site owner reported, in plain words, with the HTTP status we saw.
`source`	The public forum thread the case was taken from (wordpress.org, wordpress.stackexchange.com or stackoverflow.com). Empty for a capability sweep.
`items`	Which of the problem types on the "What we fix" page this case exercises.
`wordpress / php`	Versions on the test site.
`observed`	Per server setup: HTTP status of the homepage and the login page after the failure was recreated.
`runs`	One row per attempt: date (UTC), server setup, connection mode, outcome, product version. A sweep run also carries a result per problem type ("cleared" or "still present").
`gaps`	Every run that did not end in "recovered", and every problem type still present after a sweep, has a row here: what we changed, the version it shipped in, and the re-test.
`changes`	Dated list of edits to this record after it was first published.