Continuous Unit Checking:
Part Two

Where were we?

• Part 1: Command-line utilities and wrapper scripts
• Part 2 (this):Guard (with Ruby) and SBT (with Scala)
• Part 3: Infinitest (with Java and IntelliJ IDEA)

In Part 1, we looked at some simple continuous checking solutions based on shell commands and wrapper scripts. Let’s start climbing the tool pile and see what we find.

Guard for Ruby

Guard exemplifies the next level “up” from plain command-line tools.

In the Ruby world there are several continuous unit testing tools, of which two are significant and widely-used: ZenTest and guard.

ZenTest is a suite of four tools, of which autotest supports automatic running of unit checks when any file in a monitored directory changes. Both autotest and guard are configurable to filter files by filename and to control whether the full test suite is executed after a failed test has been corrected. These features go beyond what we can easily accomplish using basic command-line utilities and shell scripts.

As mentioned earlier, the additional functionality comes at the cost of a little more complexity. These tools have configuration files that have to be set up and, in the case of guard, initialization that has to be done to enable certain features.

Another cost to obtain more functionality is some limitation in what code can be supported. ZenTest and guard are specifically intended to support Ruby application development. While straight command-line tools don’t care what language we’re writing in, as they can execute any script we care to write, these tools are designed to understand something about Ruby unit test frameworks like Test::Unit, minitest, and rspec, as well as the conventions typically used in common types of Ruby projects like Rails.

In principle, you could use autotest or guard with languages other than Ruby, but in practice people generally don’t. The effort to wrangle a language-specific tool into supporting a different language can be significant, and when there are already tools available for the other language, why bother?

Let’s set up guard to monitor a trivial Ruby application similar to the Python example we looked at before. Here’s the “application” code:

class Incrementer

  def increment number
    number += 1
  end

end 

and here’s the complete “unit test suite”. It uses rspec.

require "spec_helper"
require_relative "../app/incrementer"

RSpec.describe Incrementer do 

  context "incrementing numbers" do
    before(:each) do 
      @incr = Incrementer.new
    end

    it "increments 1 resulting in 2" do
      expect(@incr.increment(1)).to eq(2)
    end
  end   
end 

Obviously, even for this extremely simple example that isn’t a sufficient set of unit checks to cover al the possibilities of the increment method. Our purpose is to demonstrate the setup for a continuous testing tool, and not to exercise rspec or Ruby to any extent. This will do.

We use bundler to define the dependencies for our application. For this example, the Gemfile contains the following:

source "http://rubygems.org"

group :test, :development do 
  gem "guard"
  gem "guard-rspec"
  gem "rspec"
  gem "rake"
end

At a minimum we need the gem guard. We’re using rake, the standard Ruby make utility, to build our project, and rspec, a behavior-oriented unit checking tool popular in the Ruby community. We need to pull in gems for those, as well as an additional gem to provide rspec support in guard. The bundle install command installs those gems unless they are already present on the system. We run the command from our project root directory.

bundle install --path vendor/bundle

The –path option tells bundler to save the gems in subdirectory ./vendor/bundle under our project, rather than installing them globally. We run the command without sudo, as this will store the gems in user space. This is a common convention for Ruby applications.

With those gems in place, we need to configure guard to work with our application. This is where things get slightly more complicated than with basic command-line tools. We’ll use bundler to run a couple of setup commands for guard. These are executed from our project root directory.

bundle exec guard init
bundle exec guard init rspec

The bundle exec guard init command creates a template Guardfile in the project root directory. The Guardfile contains the configuration for controlling how guard is to run.

The bundle exec guard init rspec command adds rspec-specific configuration settings to the Guardfile. When you run it on your own system, you’ll notice it adds specifications for Rails, Capybara, and Turnip. We aren’t using those in our “incrementer” application, so we can remove them from the Guardfile. For our simplistic example, all we need in the Guardfile is this:

guard :rspec, cmd: "bundle exec rspec", failed_mode: :keep do
  watch(%r{^app/(.+)\.rb$}) { |m| "spec" }
  watch(%r{^spec/.+_spec\.rb$}) { |m| "spec" }
end  

If you want to get serious about guard there’s plenty of documentation available. For our immediate purposes, those specifications tell guard to run the command bundle exec rspec whenever it detects a change to a file in the app subdirectory whose name ends in .rb, or in the spec directory whose name ends in _spec.rb. The bit that reads ‘failed_mode: :keep’ tells guard to continue running any examples that failed until they pass. Otherwise, it will run only the specs that relate to the files that were modified.

Here’s how the developer work flow could look when using guard.

As I see it, there are two distinct advantages to using guard over a simple command-line setup:

• It’s the same on all platforms, as it’s a Ruby application; it doesn’t matter which shell languages the platform supports or what the operating system commands look like. A straight command-line solution would probably require some tweaking between platforms.
• It only runs the examples that pertain to the code that was just changed, plus re-running any failed examples until they pass (due to the ‘failed_mode’ setting in our Guardfile). A simple loop on the command-line will run everything every time, unless we go to the trouble of writing a sophisticated script to make decisions about which examples to run.

The only disadvantage is that it works for Ruby applications, but not (easily) for Java, C#, Python, C++, and so forth. That is characteristic of continuous testing tools tailored for specific languages. The more the tool “knows” about a particular language and its conventions, the less useful it is for other languages. Even when it’s technically possible, people aren’t in the habit of searching beyond their own development community for tools. Someone working with, say, C# will likely stay within the .NET ecosystem.

Sbt for Scala

I touched on sbt in an earlier post exploring the feasibility of a minimal development environment. Let’s look at it again in the context of continuous unit checking.

sbt was designed to support Scala development. Like other tools in this category, you could configure it to support other languages, but few people do so. It’s based on Java, and in principle it cansupport other JVM languages besides Scala, just as Gradle and Maven can support other languages besides Java/Groovy/Kotlin, but the configuration may become complicated.

You need to have JDK 8 or later installed. Either Oracle or OpenJDK will work. With that in place, I installed sbt on a MacBook Pro this way:

brew install sbt@1
sbt

That initial execution of sbt downloads dependencies and gets things set up for using sbt.

This tool is part of the Java ecosystem, and as such it assumes a default project directory structure similar to the Maven convention, plus an optional ‘project’ subdirectory for Scala helper files.

project-root/
  |
  + -- project/
  |
  +-- src/
        |
        +-- main/
        |     |
        |     +-- resources/
        |     |
        |     +-- scala/
        |     |
        |     +-- java/
        |
        +-- test/
              |
              +-- resources/
              |
              +-- scala/
              |
              +-- java/

Generated artifacts are written to the ‘target’ subdirectory off the project root.

Like many build tools, sbt uses a configuration file to control aspects of its operation. By convention, there’s a ‘build.sbt’ in the project root directory. Configuration settings can be organized in some logical way and separated into different files, if that makes sense in your situation. All files named ‘*.sbt’ will be read.

The ‘sbt’ command starts a shell wherein you can execute sbt tasks. There’s a lot to learn about sbt tasks (out of scope for this post).

Continuous build and test functionality is built into sbt. If you start a shell with ‘sbt’ and then enter a command prefixed with tilde (~), sbt will monitor the default directory structure and run that command whenever a file is changed. So you could do continuous unit checking with these commands:

sbt
> ~testQuick

You can stop monitoring for changes by pressing Enter. To terminate the repl, enter the command ‘exit’. (Enter exit. Ha. Funny.)

Let’s try our “increment” application again to check out sbt’s continuous testing work flow. Here’s the “application” code:

package example

object Incrementer extends Increment with App {}

trait Increment {
  def increment(a:Int): Int = a + 1
}

and here’s the comprehensive, thoughtfully-designed test suite:

package example

import org.scalatest._

class IncrementSpec extends FlatSpec with Matchers {
  "The Incrementer object" should "return 3" in {
    Incrementer.increment(2) shouldEqual 3
  }
}

And, here’s how a developer work flow could look:

You can see that the pattern is the same as it has been all along so far, but things seem to take longer to execute. That’s partly due to the nature of the Java language. Tools have to download various jars before they can run. Compiles take some time, even when the source code is very small. The repl doesn’t react quickly to input. This is part of the “cost” of using more-sophisticated tools. You get more functionality, but there’s more code involved and more configuration to do.

What’s Next?

In the next installment, we’ll move off the command line and explore continuous checking in an IDE environment.