test suite

From IndieWeb


A test suite is a set of tests, usually for an entire specification, that implementations can use to check their conformance and often generate an implementation report with the results; nearly all IndieWeb standards have test suites.

Why

Test suites are much better for interoperability over time, e.g. a many years old implementation that passes a test suite will have a very good chance of interoperating with a brand new implementation that passes the test suite (depends on unambiguous spec text + reasonably thorough test suites of course). Webmention and https://webmention.rocks/ is a good example of this.

Why better than interop testing

"interop testing setup" is roughly what IETF has done in the past for some specs, and the problem is that it only serves as a synchronous checkpoint in time, and does not persist over time. It creates a two tier (or more) dynamic to implementations, those that got to participate in the "interop testing setup" and those that did not, e.g. future implementations. That disadvantage and discouragement to additional, future implementations is an anti-plurality bias.

Examples

Full test suites:

Test files:

Splash page / coming "soon":

See Also