The Mocks Are Alright

rayo - 20 Mar 2014

We deploy applications on AWS, and we run jobs that check on them. Like any other code, the jobs need tests.

Consider a simple reaper, which terminates Elastic MapReduce (EMR) clusters whose job flows have taken way too long:

trait Reaper {
  def terminateLongJobFlows(minNormalizedInstanceHours: Int) {
    val emr = elasticMapReduce()
    /* filter jobs from emr.describeJobFlows()... */
    /* run emr.terminateJobFlows() on filtered jobs... */
  }

  def elasticMapReduce(): AmazonElasticMapReduce
}

object Reaper extends Reaper {
  def elasticMapReduce(): AmazonElasticMapReduce =
    new AmazonElasticMapReduceClient()
}

If you wanted to write a test/spec for Reaper, you'd need some sort of test double for its collaborator AmazonElasticMapReduce. What kind of double you use is up to you...

Mock Trial

For this post, consider a ScalaTest WordSpec using mocks.

@RunWith(classOf[JUnitRunner])
class ReaperSpec extends WordSpec with Matchers with MockitoSugar {

  "Reaper.terminateLongJobFlows()" should {
    "terminate jobs taking >= a given # of normalized instance hours" in {

      val emr = mockEmr(Seq(("a", 40), ("b", 60), ("c", 20), ("d", 80)))
      val reaper = new Reaper() {
        def elasticMapReduce(): AmazonElasticMapReduce = emr
      }

      reaper.terminateLongJobFlows(50)

      val argCaptor =
        ArgumentCaptor.forClass(classOf[TerminateJobFlowsRequest])

      verify(emr).terminateJobFlows(argCaptor.capture)

      val expectedJobFlowIds = Set("b", "d")
      val actualJobFlowIds = argCaptor.getValue.getJobFlowIds.asScala.toSet

      actualJobFlowIds should equal (expectedJobFlowIds)
    }
  }

  /* ... */
}

What makes this test mockist (?) is that it spells out how the system under test (SUT), our Reaper, interacts with its collaborator, AmazonElasticMapReduce. The test will

  1. verify our Reaper invoked AmazonElasticMapReduce.terminateJobFlows(), and

  2. check that the supplied argument, a TerminateJobFlowsRequest, contains the IDs of the job flows that should be terminated.

(1) and (2) are handled by a mock framework like Mockito.

Contrast this with a stubbist (?) approach (obligatory link). You're generally not concerned about the interaction but rather the state of things at the end. "I terminated some job flows. Are they there anymore?"

Deep Mockery

There was a time when it would've been painful to write a helper method like mockEmr, which creates the mock AmazonElasticMapReduce, plus all of its contained information.

Consider that many AWS API calls return *Result value objects with graphs of other value objects, sometimes going three or four levels deep (e.g. describeJobFlows() -> getJobFlows() -> getExecutionStatusDetail() -> getState()). So mocking chained API calls would require creating mocks for each call in the chain--tedious, to say the least.

With Mockito, however, you can more easily mock deeply by passing to the mock constructor an additional argument, RETURNS_DEEP_STUBS. Mockito handles the deep mock implementation (basically doing what you would've done), and returns you a mock that lets you apply when/thenReturn to chained API calls, and verify where appropriate.

Subsequently, mock creation gets more concise:

def mockEmr(idsAndHours: Seq[(String, Int)]): AmazonElasticMapReduce = {
  val emr = mock[AmazonElasticMapReduce](Mockito.RETURNS_DEEP_STUBS)
  val jobFlows = idsAndHours.map { case (id, hours) =>
    mockJobFlow(id, hours, JobFlowExecutionState.RUNNING)
  }.asJava
  when(emr.describeJobFlows().getJobFlows()) thenReturn (jobFlows)
  emr
}

def mockJobFlow(
    jobFlowId: String,
    normalizedInstanceHours: Int,
    state: JobFlowExecutionState): JobFlowDetail = {
  val jobFlow = mock[JobFlowDetail](Mockito.RETURNS_DEEP_STUBS)
  when(jobFlow.getJobFlowId()) thenReturn(jobFlowId)
  when(jobFlow.getInstances().getNormalizedInstanceHours()) thenReturn(normalizedInstanceHours)
  when(jobFlow.getExecutionStatusDetail().getState()) thenReturn(state.name)
  jobFlow
}

Sharing is Caring

Because mocks tend to be written more specifically to the tests they support, it's easy to forget about sharing them. But that doesn't mean they couldn't be generalized or shared. ScalaTest has advice on how to share fixtures. For example, you could extract the mock creation methods into their own trait:

trait SharedFixtures extends MockitoSugar { this: Suite =>

  def mockEmr(idsAndHours: Seq[(String, Int)]): AmazonElasticMapReduce = {
    /* ... */
  }

  def mockJobFlow(jobFlowId: String, state: JobFlowExecutionState, terminationProtected: Boolean = false): JobFlowDetail = { /* ... */ }
}

Then, any spec can just extend that trait:

class ReaperSpec extends WordSpec with Matchers with SharedFixtures

You Reap What You Sow

I wrote my test and mocks focused on the AWS API calls that the reaper makes. I'm trading a stronger coupling to the reaper's implementation, for the ability to confirm, without a full integration test, that I'm making the calls that will terminate the correct job flows.

But what if the API calls change? Then I'll have to change the reaper and the test mocks.

Sometimes this is self-inflicted. There are a couple of ways I can use a collaborator object; I switch from one approach to the other; and then I curse myself for the mocks I wrote to support the original implementation.

Other times you have no choice. As of this writing, the AmazonElasticMapReduce job flow view is deprecated, to be superseded by a cluster view. The two views provide similar information, but not the same; when the job flow view is removed, I'll have to change my tests. But that is a problem for another day, to be handled if it happens.

Mock Verdict

In this particular case, a framework like Mockito combined with ScalaTest makes writing tests quick and easy. My SUT gets data from a third-party API which fronts a web service, and I want to verify how it uses the API--calls and arguments--as it reacts to that data. To help me write my tests in a focused and concise manner:

  • The when/thenReturn construct lets me concisely mock the calls to get data.

  • The verify construct lets me easily, more fully check the calls that act on the data.

  • The deep stubbing provided by Mockito lets me construct deep mocks because I can apply when/thenReturn and verify on them where needed.

Mock It Stub It Spy It Test It

It sure would be nice to have an AmazonElasticMapReduce already written with a simple implementation of a job flow container, that could support job flow and cluster API views. I suppose I should get around to that.

Furthermore, with Mockito, you can add behavior verification to any object--even a stub!--by spying on it.

Implement AmazonElasticMapReduceStub:

public class AmazonElasticMapReduceStub implements
AmazonElasticMapReduce {

    private final List<JobFlowDetail> jobFlows = new
    ArrayList<JobFlowDetail>();

    public addJobFlow(String jobFlowId, int normalizedInstanceHours, JobFlowExecutionState state) {
        // Add to jobFlows...
    }

    @Override
    public DescribeJobFlowsResult describeJobFlows() {
        return new DescribeJobFlowsResult().withJobFlows(jobFlows);
    }

    @Override
    public ListClustersResult listClusters(final ListClustersRequest listClustersRequest) throws AmazonServiceException, AmazonClientException {
        // Something to transform jobFlows from List<JobFlowDetail> to List<Clusters>...
    }

    @Override
    public terminateJobFlows(request: TerminateJobFlowsRequest) {
        // Remove from jobFlows...
    }

    /* ... */
}

Spy it:

def spyEmr(idsAndHours: Seq[(String, Int)]): AmazonElasticMapReduce = {
  val emr = spy(new AmazonElasticMapReduceStub())
  for ((ids, hours) <- idsAndHours)
    emr.addJobFlow(id, hours, JobFlowExecutionState.RUNNING)
  emr
}

Now use the spy instead of the mock. Both verify() and the argument checking will work as before.

Also:

comments powered by Disqus
Previous Post Next Post