Contract Testing - Spring Cloud Contract

In the previous article, I described the concept of Contract Testing. To recap, this is a technique that allows validating communication between services in isolation. It eliminates the problems which occur when we try to test communication using integration or e2e testing. In this article, I’ll introduce one of the commonly used libraries for contract testing - Spring Cloud Contract.

If you’re not familiarized with the concept of Contract Testing, I encourage you to read my previous article. This will help you to better understand the ideas that stand for Spring Cloud Contract.

All the examples in this article are based on the order-service and payment-service communication done via REST API. The tests are written in Groovy using Spock. Projects use Gradle as a build tool. The code is available in this repository. To recap how the services communicate:

Defining the contract

The most important concepts of this type of testing are contracts. Spring Cloud Contract allows us to define it using a convenient DSL (Domain Specific Language). You can write it in Groovy, Java, Kotlin, and even in YAML. The contract describes how the communication is going between the producer and consumers. The following contract defines using a REST API of payment-service by order-service. It’s written in Groovy and describes a happy path scenario of executing the online payment.

Contract.make {
   name "should execute online payment"
   request {
       method 'PUT'
       urlPath $(consumer(regex('/payment/' + uuid())))
       body([
               accountId  : $(consumer(anyUuid())),
            quota      : $(consumer(regex('^(\\d+\\.\\d+)$')), producer(1024.50)),
               paymentType: $(consumer(regex('^TRANSFER_(OFFLINE|ONLINE)$')), producer('TRANSFER_ONLINE')),
               dueDate    : $(consumer(anyDateTime()))
       ])
       headers {
           accept 'application/json'
           contentType 'application/json'
       }
   }
   response {
       status OK()
       body([
               paymentExternalId: "ext-${fromRequest().path(1)}",
               status           : "FINISHED",
       ])
       headers {
           contentType('application/json')
       }
   }
}

The contracts should be placed in /src/contractTest/resources. They may be grouped into directories and subdirectories. This allows, for example, to organize contracts by consumers. In this example, it’s placed in payment-service/src/contractTest/resources/contracts/orderService as the contracts belong to order-service.

Request

The most important parts of this contract are request and response blocks. The first one defines how the request to the API should look like. It defines a PUT method with a path described using regular expression - it’s /payment/ followed by UUID. This notation may be kind of mysterious at first glance. I’ll explain it later.

Then, the expected request body is defined. We see that there are 4 fields required: accountId, quota, paymentType, and dueDate. The values may look complicated. We will decode them soon. The last part of the request section describes the required headers. This API requires Accept and Content-Type headers, both with application/json.

Now that you know the basics, let’s try to decode skipped ciphers. Let’s focus on the definition of URL.

    urlPath $(consumer(regex('/payment/' + uuid())))

The expression $(consumer(...)) allows us to define so-called dynamic properties. Those are the values that we don’t want to hardcode in the contract. Think about ID or timestamp. Fixing it means that the consumer is forced to use exact values in its test, which is not the best option. We want to give the consumer team flexibility in implementing its test. Thus, the better solution is to define the regular expression. In this case, we used a built-in method uuid() which defines a regular expression for UUID.

The definition above is actually an abbreviation of this expression:

urlPath $(consumer(regex('/payment/' + uuid())), producer('/payment/42cc610b-575d-4a52-9fba-4a43e8d3cfe9'))

The consumer() defines what values are acceptable while calling this API. Those are values that will be defined in the stub. In this case, it accepts all the URLs that start with /payment/ followed by any valid UUID. In other words, the consumer can use any UUID in its tests, not the specific one.

The producer() part defines the value that is used in the producer’s test of the API. It cannot be a regular expression, because the API in the test has to be called with a specific URL (e.g. /payment/42cc610b-575d-4a52-9fba-4a43e8d3cfe9). Because this is quite a long expression, Spring Cloud Contract allows skipping the producer() part as we did in our contract. Then, in the API test, the value is generated based on the regular expression defined in consumer().

The same solution was used to define accountId and dueDate fields in the request body. For quota and paymentType we defined a custom regular expression.

quota      : $(consumer(regex('^(\\d+\\.\\d+)$')), producer(1024.50)),

paymentType: $(consumer(regex('^TRANSFER_(OFFLINE|ONLINE)$')), producer('TRANSFER_ONLINE'))

Quota should be a positive double which unfortunately doesn’t have a built-in method in Spring Cloud Contract as UUID has. You can find the full list of built-in methods with regular expressions in the documentation. Similarly, paymentType is an enum that can be either TRANSFER_ONLINE or TRANSFER_OFFLINE. Thus, we have to define regular expressions by hand and provide the value used in the API test (producer()).

Response

In the response section, we describe the response that the producer returns for a defined request. Most of the parts are quite intuitive. They define HTTP status, response body, and headers. What may be surprising is how the value of paymentExternalId is defined.

paymentExternalId: "ext-${fromRequest().path(1)}"

In this case, we want this field to be always ext- followed by paymentId (provided to the payment-service as an URL path param). Because paymentId is defined as a regular expression (valid UUID) we cannot hardcode it. We want it to be exactly the value, which was passed in the URL. This is what a method fromRequest() helps us. It allows referencing values from the request in the response. In this case, we wanted to use paymentId, the second part of the path. The path(n) method allows us to retrieve it. N is a number of path elements, numbered from 0. There are more similar methods available, which allow getting values from request body, headers, etc. You can find them here.

Testing the producer

Now, as we have the defined contract, we want to test whether the producer’s API really works as expected. As I wrote in the previous article, we don’t have to write any test for that. They are generated based on the contracts. To do that with Spring Cloud Contract, you must add the following plugin and dependency to the producer’s project.

plugins {
    (...)
    id 'org.springframework.cloud.contract' version '4.0.0'
}

(...)

dependencies {
    (...)
    testImplementation 'org.springframework.cloud:spring-cloud-starter-contract-verifier:4.0.0'
}

The plugin is necessary to generate the tests from the contracts. The above snippet assumes you already have the dependencies required to run and test the Spring Boot application (e.g. spring-boot-starter-web and spring-boot-starter-test).

Spring Cloud Contract requires some configuration to generate and run the producer’s tests.

contracts {
    baseClassForTests = "pl.rmaciak.payment.BaseContractTestsSpec"
    testFramework = "SPOCK"
}

In the minimal configuration, only baseClassForTests is required. The second parameter defines that tests will be in Spock. baseClassForTests is the name of the class that all auto-generated tests extends. This class has to set up the server that will be used in the test. We can do it in two ways:

• mock the server using e.g. RestAssured or MockMvc. This is the default and recommended option.
• start up the server using for example @SpringBootTest. In this case, the tests are run against the real server, not a mock. If you want to test this way you have to set testMode in the configuration to EXPLICIT.

In this example, we will use RestAssuredMockMvc. Other configuration options may be found here.

Which option should I choose?

You may think that testing against a real, listening server is a better solution than mocking it. However, remember that in contract testing we focus on the communication aspect. Thus, we don’t need the whole production code working.

Mocked server exposing the controllers with stubbed dependencies is usually good enough. This is a layer that defines the communication aspects. We don’t need the code with business logic which is called by the controller. These may be stubbed.

To generate and run the test you should execute the command:

./gradlew check

If the API is implemented according to the contract the build should pass successfully. In other cases, you can get an error saying about the issue.

This proves that the tests were really generated and executed. Now let’s look under the hood and see what happened.

If you look up the build directory, you will notice that in generated-test-sources, the class OrderServiceSpec exists. It was auto-generated by Spring Cloud Contract.

The class extends BaseContractTestsSpec that we defined in the configuration. It contains the test in Spock (also configured) that checks whether the server works as expected in the contract. By default, the generated test uses Spring MockMvc to test the server. However, it’s possible to use the following ways of testing the API:

• SpringMockMvc from Rest assured. It’s the default.
• SpringWebTestClient from Rest Assured. The testMode parameter must be set to WebTestClient
• JaxRSClient. The testMode parameter must be set to JaxRsClient
• There is also a special mode I already mentioned - EXPLICIT. In this mode, the server is started and the real HTTP calls are executed.

This is what the test for our contract looks like. It prepares the request as defined in the contract. Then it executes the API and asserts that the returned response matches the one expected in the contract. Nothing interesting, except that all this boilerplate code was generated and we don't have to implement it. This is only one scenario. Think if there are others, like specific communication scenarios, exception scenarios, etc. And they multiply for every endpoint. Quite a lot of work is done for us, isn’t it?

def validate_should_execute_online_payment() throws Exception {
    given:
        MockMvcRequestSpecification request = given()
                .header("Accept", "application/json")
                .header("Content-Type", "application/json")
                .body('''{"accountId":"f4459572-b7a8-4d2e-9f49-75009f9e7f6f","quota":1024.50,"paymentType":"TRANSFER_ONLINE","dueDate":"2018-08-18T12:23:34"}''')

    when:
        ResponseOptions response = given().spec(request)
                .put("/payment/5ecbe91f-a1f7-baff-a9b3-55d48cbbf2fb")

    then:
        response.statusCode() == 200
        response.header("Content-Type") ==~ java.util.regex.Pattern.compile('application/json.*')

    and:
        DocumentContext parsedJson = JsonPath.parse(response.body.asString())
        assertThatJson(parsedJson).field("['paymentExternalId']").matches("ext-5ecbe91f-a1f7-baff-a9b3-55d48cbbf2fb")
        assertThatJson(parsedJson).field("['status']").isEqualTo("FAILED")
}

These tests (usually there are more scenarios) validate the API’s correctness. Now we have to validate whether the consumer correctly calls this API.

Testing the consumer

Developers responsible for consumer service have to implement the tests to confirm that it is correctly integrated with the API. Those tests may be defined on different levels. They can test only an HTTP client, but also they can validate whether the logic of the service works correctly when interacting with another service. This is actually what integration tests do. However, the biggest difference is that we don’t have to implement stubs on our own using Spring Cloud Contract. Instead, we should use the ones generated from the contract defined by the producer.

If you look up the build directory in payment-service again, you will notice that there is an additional JAR file built - payment-service-0.0.0.1-SNAPSHOT-stubs.jar

This is the JAR that contains the stubs. It can be used to run a WireMock server in consumer tests. We can do it by using Spring Cloud Contract Stub Runner.

testImplementation 'org.springframework.cloud:spring-cloud-starter-contract-stub-runner:4.0.0'

This brings an annotation AutoConfigureStubRunner, which takes several arguments. In the minimal configuration, only ids parameter is necessary. This allows configuring the JARs with stubs that should be used by Wiremock to mock the server.

@SpringBootTest(classes = [OrderApplication])
@AutoConfigureStubRunner(
        ids = ["com.rmaciak:payment-service:0.0.1-SNAPSHOT:stubs:8088"],
        stubsMode = StubRunnerProperties.StubsMode.LOCAL
)
class OrderCreatorWithContractSpec extends Specification {
    (...)
}

In this example, I specified the JAR built from contracts in payment-service. At the first glance, the string inside the array may look strange. Let’s decode it.

• com.rmaciak is groupId
• payment-service is artifactId
• 0.0.0.1-SNAPSHOT - version of the JAR with stubs. It may be ‘+’ which means to use always the newest version
• stubs is a JAR classifier. This is the default value that may be skipped.
• 8088 is the port of the WireMock server to listen. It’s random if not provided.

The second parameter of AutoConfigureStubRunner used in the example is stubsMode.

It takes the following values:

• LOCAL - means that the stubs are downloaded from the local repository (e.g. /.m2 directory of Maven). This repository should be defined by your build tool. In this example I used Maven.
• REMOTE - means that the stubs are downloaded from a remote repository. If this mode is used, the remote repository must be defined using the repositoryRoot parameter.
• CLASSPATH - means that the classpath is scanned to take the defined stubs. This is the default option.

You can find other parameters of this annotation in the documentation.

With the stubs up and running, we can test our code that requires an HTTP call to another service.

def "should create order and initiate offline payment"() {
    given:
    def accountId = randomUUID()
    def orderId = randomUUID()
    def quota = BigDecimal.valueOf(120.99)

    expect:
    sut.createOrder(accountId, orderId, quota, TRANSFER_OFFLINE).paymentProcessed()
}

As you can see there is no other stub configured here. Stub Runner does everything.

Now we have both producer and consumer tested against the agreed contract. If the tests of both services pass in CI/CD pipeline, we can be sure that the services are correctly integrated. We can safely go to production.

Of course in this article I showed only one contract as an example. To do it correctly we should probably define more contracts. For example, if there are specific responses for exceptional scenarios like errors. However, please remember the trap I explained here.

Now, let’s see how Spring Cloud Contract supports both Consumer Driven and Producer Driven approaches.

Consumer-Driven Contract Testing with Spring Cloud Contract

To recap, in the Consumer Driven Contract approach, the consumer is an upstream service, which defines the communication. The producer service team (downstream) should implement the API as requested by the consumer (with a mindful of common sense). This is a sample flow of the consumer team’s work (explained in the details here)

Spring Cloud Contract supports this flow. When the consumer team is satisfied with the contract, they generate the tests (3) and stubs (5). Then, they can use the generated JAR file with stubs and add it to their project. This is how it works in the default stubsMode (CLASSPATH). However, the better way is to deploy them to the local, for instance, Maven repository (.m2) and use them in the consumer’s test. This is what LOCAL stubsMode does.

This should be only a temporary solution until the producer’s team implements the API. When it’s done and all tests pass, the JAR with stubs should be uploaded to the remote (global) Artifactory. Then, the consumer team may change stubsMode to REMOTE. This will ensure that the stubs are always downloaded from the Artifactory. This should be combined with the stub version defined to the newest (+ in ids). Now, the consumer tests are executed with the stubs, which correspond to the contract defined with the producer. Such a setup ensures that consumer tests are reliable.

Producer Driven Contract Testing with Spring Cloud Contract

There is no specific support for Producer Driven Contract Testing in Spring Cloud Contract. In this approach, the producer is responsible for defining the contracts. The consumers take it for granted. They cannot influence it. Thus, what the payment team has to do is describe the contract and implement the API according to them. Then, the stubs may be generated and passed to the consumers to be used in the tests. How it’s passed depends on the situation. They may be uploaded to Artifactory, Nexus, Git repository, etc. The stubs not only allow testing integration but also are a kind of documentation of how the producer’s API works.

Summary

This article explained how to define a contract of REST API using Spring Cloud Contract. I demonstrated testing both producer and consumer against this contract. Through the tutorial, you learned the main parts of the framework and its basic configuration. I described how Spring Cloud Contract may be used to test contracts in two ways - Consumer Driven and Producer Driven.

I believe that after reading this article, you are able to start using Spring Cloud Contract in your project. In the next part, I’m going to explain how another contract testing framework, called Pact, works.