This plugin provides a framework to produce a TAP stream. TAP stands for the “Test Anything Protocol”, and according to its website, it “is a simple text-based interface between testing modules in a test harness”. Although TAP has its origins tied to Perl, it has since then spread to a number of other programming languages and environments, the most recent of which is Praat!
This plugin is only concerned with generating a compliant TAP stream, which will look something like this:
1..4 ok 1 - Input file opened not ok 2 - First line of the input valid ok 3 - Read the rest of the file not ok 4 - Summarized correctly # TODO Not written yet
This means that four tests were planned, out of which two passed and two failed. However, one of the failing tests has been explicitly marked as “TODO”, meaning that it was expected to fail… so it’s OK.
To analyse these results, a “harness” is needed, which will read this input stream and provide a report on the outcome of the entire test suite. Different harnesses exist providing different features, and they should all work with this same output.
The procedures in this plugin do not provide a harness. They only produce the stream.
The procedures made available using
simple.proc are the same as those
provided by the testsimple plugin.
Every TAP stream must either begin or end with a plan line, which specifies how may tests will run, or have run. This is a safeguard in case your test file dies silently in the middle of its run.
For tap, it is an error to begin a test suite without having first
specified a plan. That can be done by calling this procedure or, in the case
that a test suite does not know how many tests will be run, by explicitly
stating this with
Sometimes a test suite will have an undetermined number of tests. In this case,
tap requires this to be explicitly stated before the first test takes
place by calling this procedure. Every test script using this plugin should
begin with a call to this procedure or to
It is an error to call this procedure more than once, or to call it before either of the planning procedures has been called.
The core of TAP is the test line. Test lines begin with either
depending on the result of the test. A number is expected to follow the result
of the test, although this is optional. After this, a test description may be
This procedure takes care of generating all of these fragments, and internally
keeps track of the test number (so that successive calls to
print increasing numbers). The description provided as the second argument can
be the empty string.
@ok, this procedure also generates a test line, but the
value of the first argument is not interpreted as a boolean, but executed as a
formula string. This will be passed to the
Calculator... command, and the test will pass if this results in a true value
A test can be skipped if the test suite decides it does not make sense to run
it. A skipped test will be marked as passing regardless of their output, and the
test line that reports it will bear a special description with the
It might be desirable to simply wrap the test in a conditional block and never
run it in the first place, but that would mean test numbers would change. This
procedure makes it possible to mark tests as skipped while keeping the test
indices intact, which means
@plan can still be used.
The first argument to
@skip specifies the number of tests to skip,
while the second argument specifies the reason why they were skipped. If the
number of tests to skip is
undefined, all tests until the next call of
@end_skip will be skipped.
This procedure ends a SKIP block. If a previous call to
been made with an
undefined number of tests to skip, tests following the call
to this procedure will no longer be skipped. Otherwise, this procedure has no
Similar to calling
@skip with an undefined number of tests to skip,
calling this procedure will mark following tests as skipped. However, unlike
@skip, the effects of this procedure are not affected by subsequent
@end_skip. Once called, all remaining tests will be
An alternative to skipping tests is to mark them as to-do, which is useful for example when the test is testing code that has not been written yet. Tests marked as to-do will still be run, and their results will be accurately reported. But unlike regular tests, failing a to-do test does not affect the outcome of the test suite. Passing them is a nice bonus.
Using this feature, a test can be written when a bug is discovered even if the bug has still not been fixed. This not only works as a reminder that the bug is there, but it also has the benefit that its result will change as soon as the bug has been fixed (even if the solution was unintentional).
@skip, the first argument states the number of tests to
marks as to-do, while the second one gives them a common description. If the
number of tests is
undefined, a TODO block begins and lasts until the next
@end_skip, this procedure finishes a TODO block. If
there has been a prior call to
@todo with an
undefined number of
tests, tests following this procedure will no longer be marked as to-do.
Otherwise, this procedure has no effect.
All the procedures made available by
simple.proc are also available through
more.proc, such that tests written using
simple.proc can be executed with
more.proc instead without changing the source code at all.
The procedures described below are available exclusively using
The main difference between
main.proc is that the
procedures in the latter try to provide better diagnostics for failing tests.
With the simple ok-style procedures, this diagnostic cannot be any more than
the state of the test. The new is-style of tests provide more information
about the existing and the expected outcome of the test, and allow for much more informative error messages.
@is takes a numeric value and its expected outcome (as well as a test
message), and tests both for equality.
@is, but tests string values.
@is, but tests what a numeric value is not.
@isnt, but tests string values.
The first three arguments in this case are a numeric value, a string representing an operation to test, and a second numeric value. The result of the test is the result of applying that operator to both values.
@cmp_ok, but tests string values.
Checks whether an object with the specified ID exists in the Objects list.
@ok_exists, but checks that a given object does not
Checks whether an object with the specified ID exists in the Objects list and is currently selected.
@ok_selected, but checks that a given object exists
and is not selected.
Checks whether two objects specified by IDs are the same or not. Additional checks for whether the objects exist or not are also made.
Automatically pass a test. This can be useful if a test is the result of a complicated process not easily reduced to one of the procedures above.
@pass, but automatically marks the test as failing.
If you realise that testing after a given point is futile, you can call this to aborts the test suite altogether.