This plugin implements a simple framework to produce a TAP stream. TAP stands for the “Test Anything Protocol”, and according to its website, it “is a simple text-based interface between testing modules in a test harness”. Although TAP has its origins tied to Perl, it has since then spread to a number of other programming languages and environments, the most recent of which is Praat!
This plugin is only concerned with generating a compliant TAP stream, which will look something like this:
1..4 ok 1 - Input file opened not ok 2 - First line of the input valid ok 3 - Read the rest of the file not ok 4 - Summarized correctly # TODO Not written yet
This means that four tests were planned, out of which two passed and two failed. However, one of the failing tests has been explicitly marked as “TODO”, meaning that it was expected to fail… so it’s OK.
To analyse these results, a “harness” is needed, which will read this input stream and provide a report on the outcome of the entire test suite. Different harnesses exist providing different features, and they should all work with this same output.
The procedures in this plugin do not provide a harness. They only produce the stream.
Every TAP stream must either begin or end with a plan line, which specifies how may tests will run, or have run. This is a safeguard in case your test file dies silently in the middle of its run.
For testsimple, it is an error to begin a test suite without having first
specified a plan by calling this procedure. In the case that a test suite does
not know how many tests will be run, this must be stated explicitly with
Sometimes a test suite will have an undetermined number of tests. In this case,
testsimple requires this to be explicitly stated before the first test takes
place by calling this procedure. Every test script using this plugin should
begin with a call to this procedure or to
It is an error to call this procedure more than once, or to call it before either of the planning procedures has been called.
The core of TAP is the test line. Test lines begin with either
depending on the result of the test. A number is expected to follow the result
of the test, although this is optional. After this, a test description may be
This procedure takes care of generating all of these fragments, and internally
keeps track of the test number (so that successive calls to
print increasing numbers). The description provided as the second argument can
be the empty string.
@ok, this procedure also generates a test line, but the
value of the first argument is not interpreted as a boolean, but executed as a
formula string. This will be passed to
Calculator... command, and the test will pass if this is a
true value (i.e., non-zero).
A test can be skipped if the test suite decides it does not make sense to run
it. A skipped test will be marked as passing regardless of their output, and the
test line that reports it will bear a special description with the
It might be desirable to simply wrap the test in a conditional block and never
run it in the first place, but that would mean test numbers would change. This
procedure makes it possible to mark tests as skipped while keeping the test
indices intact, which means
@plan can still be used.
The first argument to
@skip specifies the number of tests to skip,
while the second argument specifies the reason why they were skipped. If the
number of tests to skip is
undefined, all tests until the next call of
@end_skip will be skipped.
This procedure ends a SKIP block. If a previous call to
been made with an
undefined number of tests to skip, tests following the call
to this procedure will no longer be skipped. Otherwise, this procedure has no
Similar to calling
@skip with an undefined number of tests to skip,
calling this procedure will mark following tests as skipped. However, unlike
@skip, the effects of this procedure are not affected by subsequent
@end_skip. Once called, all remaining tests will be
An alternative to skipping tests is to mark them as to-do, which is useful for example when the test is testing code that has not been written yet. Tests marked as to-do will still be run, and their results will be accurately reported. But unlike regular tests, failing a to-do test does not affect the outcome of the test suite. Passing them is a nice bonus.
Using this feature, a test can be written when a bug is discovered even if the bug has still not been fixed. This not only works as a reminder that the bug is there, but it also has the benefit that its result will change as soon as the bug has been fixed (even if the solution was unintentional).
@skip, the first argument states the number of tests to
marks as to-do, while the second one gives them a common description. If the
number of tests is
undefined, a TODO block begins and lasts until the next
@end_skip, this procedure finishes a TODO block. If
there has been a prior call to
@todo with an
undefined number of
tests, tests following this procedure will no longer be marked as to-do.
Otherwise, this procedure has no effect.