Unit Testing Shell Scripts:
Part Three

This is the third in a series of posts about unit testing shell scripts. So far, we’ve rolled our own test script and used an Open Source testing framework, shunit2, to test script that checks disk usage on a system and sends a notification email when usage exceeds a threshold.

This time, we’ll try out another Open Source testing framework that takes a slightly different approach.

bash-spec and its spin-off korn-spec are behavioral-style tools that use “expect” syntax similar to that used in Jasmine and Rspec.

Behavioral-Style Testing

Most shell scripts out in the wild are not modular. They tend to be monolithic and to perform several operations in sequence. It’s challenging to write fine-grained microtests or unit tests that isolate any one of the operations from a long, monolithic script. A monolithic design also makes it hard to extract any single action from a long script for reuse in other scripts.

The design of bash-spec and korn-spec make them a poor choice for testing long, monolithic scripts. They can add value in situations where new scripts are being developed.

If we take a test-first approach to developing shell scripts, then these tools will tend to guide us toward a modular design, because that sort of design will fall out naturally from the microtest cases. If we go with the flow and let that happen, we end up with short scripts that encapsulate single tasks in functions and that can be reused by sourcing them into a “main” script as needed.

With the growing importance of devops (that is, cross-pollinating development skills and methods with operations skills and methods), it becomes sensible to design and organize shell scripts using some basic software engineering principles such as separation of concerns and single responsibility principle. Modular, reusable, composable scripts will serve us better than monolithic ones.

Of course, bash-spec doesn’t automatically result in such designs, nor do other tools prevent such designs. The behavioral style of unit testing seems to help guide developers to modular designs in the application programming world, and it may offer similar help in the scripting world.

Testing diskusage.sh with bash-spec

bash-spec and korn-spec are so similar that there’s no value in using both of them for this exercise. We’ll just use bash-spec.

Unlike shunit2, BATS, and zunit, bash-spec doesn’t have any logic to discover and execute test cases. The implementation is relatively crude. You write a test script that sources bash-spec and consists of test functions that you write. At the end of your script you call the test functions explicitly. There’s less “magic” than in the other frameworks we’ve looked at so far.

There’s no built-in capability to skip or ignore test cases; however, due to the way bash-spec is implemented, if you don’t call a test function it won’t be executed automatically. You can achieve the same result as a “skip” or “ignore” feature by deleting or commenting-out the call to a test function.

On the downside, you can easily omit a test case unintentionally by forgetting to call it. The tool doesn’t warn you or offer you any hints about that.

Revamping the test script to work with bash-spec, we end up with the following (walkthrough is after the source):

#!/bin/bash
#==================================================================================
# Specs for diskusage.sh
#==================================================================================
. bash_spec

shopt -s expand_aliases

function before_all {
    alias mail="touch mailsent;false"
}

function after_all {
    unalias mail
    unalias df
}

function before_each {
    rm -f mailsent
}

function it_does_nothing_when_disk_usage_is_below_threshold {
    it "does not issue notification when disk usage is below 90%"
    before_each
    alias df="echo 'Filesystem Size Used Avail Use% Mounted on';echo '/dev/sda2 100G 89.0G 11.0G 89% /'"
    . ./diskusage.sh
    expect mailsent not to_exist
}

function it_sends_notification_when_disk_usage_reaches_threshold {
    it "issues notification email when disk usage is at 90%"
    before_each
    alias df="echo 'Filesystem Size Used Avail Use% Mounted on';echo '/dev/sda2 100G 90.0G 10.0G 90% /'"
    . ./diskusage.sh
    expect mailsent to_exist
}

print_header diskusage.sh
before_all
it_does_nothing_when_disk_usage_is_below_threshold
it_sends_notification_when_disk_usage_reaches_threshold
after_all
print_trailer

exit 0

This honors the shopt command so we are able to mock system commands by defining aliases. This works if we source the script under test rather than executing it in a subshell.

This time we use the existence of the file ‘mailsent’ rather than its contents as the indicator that diskusage.sh called the ‘mail’ command.

The function names before_all, after_all, and before_each are not significant to bash-spec. I used those names because they are likely to be understood by people who have used other unit testing frameworks.

This example shows the ‘it’, ‘expect’, ‘not’, and ‘to_exist’ methods of bash-spec in a working context. This is how the behavioral-style assertion syntax looks. Compare it to the syntax for shunit2 assertions, which uses the ‘assertThat’ style.

There’s more hand-coding to do with bash-spec (and korn-spec) than with shunit2, BATS, or zunit because the tool does not attempt to discover test cases or to perform much under-the-covers “magic”. It does count the test cases and number them and displays the strings passed to ‘it’. However, even for that functionality you have to call ‘print_header’ and ‘print_trailer’ explicitly.

The main benefit of bash-spec is that it helps us maintain self-discipline with respect to encapsulation and separation of concerns when writing or restructuring a large script that performs many operations, resulting in easier maintenance, reuse, and understandability.

Conclusions for General-Purpose *nix script testing

On balance, the best choice for general-purpose unit testing of scripts in *nix shell languages is shunit2, in my view. That tool supports a number of useful features, is straightforward to use, and is well supported. Try out bash-spec or korn-spec if you want to experiment with a behavioral style of unit tests.

BATS and zunit are also good tools. If you don’t need to mock system commands via aliases, or if you can come up with a practical workaround for that limitation, then these tools are also viable choices.

What’s Next?

We’ve covered general-purpose test frameworks for *nix shell scripts. There are a few special-purpose test frameworks that are of interest, too. In the next installment we’ll take a look at Pester, a Powershell unit testing framework. Also coming up: ChefSpec for Chef and rspec-puppet for Puppet.