Contract Testing with Pact

This is the third article of the series about contract testing. In the first part, I explained contract testing and what benefits it gives. Then, I introduced one of the most popular frameworks for this type of test, especially in the JVM ecosystem - Spring Cloud Contract.

In this chapter, you can read about yet another, trendy tool, which supports developers in implementing contract tests. This framework is Pact. You will learn how it works, what benefits this tool provides, and what are the main differences comparing it to Spring Cloud Contract. After reading this series, you will have a good background of knowledge to test the communication of your systems integrated with REST API.

The examples presented in this article are centered around the communication between the order-service and payment-service, which is implemented through the REST API. The tests are written in Groovy using Spock. Projects use Gradle as a build tool. This code is available in this repository. To recap how the services communicate:

Defining the contract

If we want to test the communication using Spring Cloud Contract, we start with defining the contract. However, working with the Pact framework looks different. Instead of writing the contract, we start by describing our expectations about the interactions between services. What’s the difference? The only one is that what we write here is not exactly the document that we share between the communication parties. The Pact uses those expectations to generate the final contract. Let’s see how we can do that using Groovy and Spock.

class OrderCreatorPactSpec extends Specification {
    (...)

    def "should create order and execute online payment"() {
        def accountIdVal = randomUUID()
        def orderIdVal = randomUUID()
        def quotaVal = BigDecimal.valueOf(1040.42)

        given: 'payment-service executes online payment correctly'
        def paymentService = new PactBuilder()
        paymentService {
            serviceConsumer "order-service"
            hasPactWith "payment-service"
            port 8088

            uponReceiving('a request to execute online payment')
            withAttributes(
                    method: 'PUT',
                    path: regexp("/payment/$UUID_REGEXP", "/payment/$orderIdVal"),
                    headers: [
                            "Accept"      : APPLICATION_JSON_VALUE,
                            "Content-Type": APPLICATION_JSON_VALUE
                    ]
            )
            withBody {
                accountId uuid()
                paymentType string("TRANSFER_ONLINE")
                quota decimal()
                dueDate datetime()
            }
            willRespondWith(
                    status: 200,
                    headers: ['Content-Type': APPLICATION_JSON_VALUE],
                    body: {
                        paymentExternalId("ext-$orderIdVal")
                        status('FINISHED')
                    }
            )
        }

        when:
        PactVerificationResult result = paymentService.runTest {
            assert sut.createOrder(accountIdVal, orderIdVal, quotaVal, TRANSFER_ONLINE).paymentProcessed()
        }

        then:
        result == new PactVerificationResult.Ok()
    }
}

The first thing you can see here is that those expectations are written in form of the test of the order-service, which is a consumer. Pact supports only Consumer Driven Contract Testing. Thus, the expectations are usually located in the consumer.

The test above not only defines the interaction expectations but also validates that they are met by the consumer. Namely, in this test, we execute the logic of order creation, which internally uses an HTTP client. Finally, in the ‘then’ section, the test checks whether the client meets defined expectations.

The Pact framework, unlike Spring Cloud Contract¹, was created as a technology-agnostic tool from the beginning. Thus, it provides a bunch of APIs and DSLs in various languages - starting from JVM languages (Java, Groovy, etc.), by Ruby, and Rust to C++. Let’s review the expectations in this test, which are defined using Groovy DSL.

The expectations are defined in the ‘given’ section of this test. The class PactBuilder provides a fluent DSL which helps to define the communication aspects. In this case, it starts with defining the parties of the communication and port. This helps to identify the contracts to validate while running the producer’s tests.

serviceConsumer "order-service"
hasPactWith "payment-service"
port 8088

Request

The request is described using three methods:

• uponReceiving, which only marks the start of the interactions and allows to describe what triggers this interaction, e.g. uponReceiving(‘a request to execute a payment’)
• withAttributes, which describes the REST API endpoint’s method, path, and required headers
• withBody, which defines the fields that are required in the request body

Similar to Spring Cloud Contract, we don’t provide the expected values exactly if it’s possible. Instead, they are described by general patterns that those fields should meet.

For instance, in the request body, there is no specific accountId provided, but it’s defined as any uuid using uuid().

withBody {
    accountId uuid()
    paymentType string("TRANSFER_ONLINE")
    quota decimal()
    dueDate datetime()
}

Also, the path is defined using a regular expression:

path: regexp("/payment/$UUID_REGEXP", "/payment/$orderIdVal")

The value provided as a second argument is necessary because the response refers to this value. Thanks to this construct, the tests of the producer use provided value, instead of a randomly generated uuid.

Response

The order-service requires that for the explained request, the producer will generate a specific response. That response’s status should be 200. It must contain a Content-Type header with application/json and two fields - paymentExternalId and status. These expectations are defined in the test using the willRespondWith() method available in Groovy DSL. There are more convenient DSL methods that allow describing the contract. Please refer to the documentation for more details.

willRespondWith(
        status: 200,
        headers: ['Content-Type': APPLICATION_JSON_VALUE],
        body: {
            paymentExternalId("ext-$orderIdVal")
            status('FINISHED')
        }
)

Testing the consumer

Having the expectations of the API, we can test the consumer. The test should validate whether the client conforms to the communication requirements. To use the Pact to test consumers, we have to add the following dependency.

testImplementation 'au.com.dius.pact.consumer:groovy:4.4.4'

This includes not only the DSL explained above but also the API which allows executing the test in Spock. It executes the order creation logic, which under the hood uses an HTTP client, and validates that client correctly calls the REST API of payment-service.

when:
PactVerificationResult result = paymentService.runTest {
    assert sut.createOrder(accountIdVal, orderIdVal, quotaVal, TRANSFER_ONLINE).paymentProcessed()
}

then:
result == new PactVerificationResult.Ok()

If we build the order-service application, the tests should succeed. As a side effect, the Pact generates the contract in JSON format. This is the real contract, which is generated from the expectations defined in the test. The JSON is used to test the consumer, and as I’ll explain later, the producer. This file is located in the build/pact directory and for the expectations explained above it looks like the following (the snippet was shortened for readability).

{
  "consumer": {
    "name": "order-service"
  },
  "interactions": [
    {
      "description": "a request to execute online payment",
      "request": {
        "body": {
          "accountId": "e2490de5-5bd3-43d5-b7c4-526e33f71304",
          (…)
        },
        "headers": {
          "Accept": "application/json",
          "Content-Type": "application/json"
        },
        "matchingRules": {
          "body": {
            "$.accountId": {
              "combine": "AND",
              "matchers": [
                {
                  "match": "regex",
                  "regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
                }
              ]
            },
            (…)
           ]
          }
        },
        "method": "PUT",
        "path": "/payment/f4d13515-8ffd-424a-9c71-f0a52e1ffa31"
      },
      "response": {
        "body": {
          "paymentExternalId": "ext-f4d13515-8ffd-424a-9c71-f0a52e1ffa31",
          "status": "FINISHED"
        },
        "headers": {
          "Content-Type": "application/json"
        },
        "status": 200
      }
    }
  ],
  "provider": {
    "name": "payment-service"
  }
}

Testing the producer

The consumer has defined the expectations and generated the contract. Now we should ensure that the other side of communication conforms to those expectations. To do that, we need to somehow deliver the contract to the producer. How can we do that?

Basically, there are two options:

• expose the contract by ourselves - this may be done by providing the URL to the contract exposed on the server, or location on the local filesystem. This is quite an unwieldy solution.
• use a Pact Broker - a tool that stores, manages, and exposes contracts.

I recommend using the Pact Broker as it makes the testing and releasing process much easier.

Pact Broker

You can think about the Pact Broker as a centralized database that stores the contracts between the services. It provides a couple of features, which simplify the process:

• allows testing the services against the contract, which makes sure that the services in specific, tested versions may be deployed to production
• provides documentation and visualization explaining how the services interact with each other
• manages contract versioning and conflict management

You can deploy, configure and manage the instance of Pact Broker on your own using, for example, this Docker image. However, there is also a commercial, fully managed, and scalable version of this tool - Pact Flow.

Publishing contracts to Pact Broker

To publish the contracts to Pact Broker, you have to use a Gradle plugin. In the configuration below it’s defined that the broker was started on localhost on port 9292.

plugins {
      id "au.com.dius.pact" version "4.3.6"
}

(...)

pact {
    publish {
        pactBrokerUrl = 'http://localhost:9292/'
    }
}

This allows publishing the contracts using pactPublish task.

./gradlew pactPublish

Now we have the contracts between order-service and payment-service in Pact. It’s available in the broker’s GUI.

If you click the file icon, the details about the contract are displayed. The interactions that we defined as the expectations in the consumer’s tests are visible here as a JSON.

Executing the producer’s tests

The producer’s application must be configured to correctly integrate with Pact Broker and validate whether it meets the contract’s requirements. This service is implemented in Spring Boot, thus the tests must integrate with it. First, the dependency, which provides an API to execute the producer’s tests, must be added.

testImplementation 'au.com.dius.pact.provider:junit5spring:4.2.21'

The test engine must know the Pact Broker’s host. Thus, we configure it in the test application.properties file.

pactbroker:
  host: localhost
  port: 9292

Finally, the test class must be implemented to execute the producer’s validations against the contract published to Pact Broker.

@SpringBootTest(webEnvironment = RANDOM_PORT)
class ContractVerificationSpec extends Specification {

    @LocalServerPort
    private int port

    @Shared
    ProviderInfo serviceProvider
    ProviderVerifier verifier

    def setupSpec() {
        serviceProvider = new ProviderInfo("payment-service")
        serviceProvider.hasPactsFromPactBroker("http://localhost:9292")
    }

    def setup() {
        verifier = new ProviderVerifier()
        serviceProvider.port = port
    }

    def 'verify contract with #consumer'() {
        expect:
        verifyConsumerPact(consumer) instanceof VerificationResult.Ok

        where:
        consumer << serviceProvider.consumers
    }

    private VerificationResult verifyConsumerPact(ConsumerInfo consumer) {
        verifier.initialiseReporters(serviceProvider)
        def testResult = verifier.runVerificationForConsumer([:], serviceProvider, consumer)

        if (testResult instanceof VerificationResult.Failed) {
            verifier.displayFailures([testResult])
        }

        testResult
    }
}

This may look quite complicated. Let’s see step by step what happens there.

The class is annotated with the ‘@SpringBootTest’ annotation, which causes the server with test configuration to be started with a random port.

Then, two objects required to execute contract verifications are declared:

• ProviderInfo - defines the tested service configuration (payment-service) like name, host, port, request path, and so on.
• ProviderVerifier - verifies the producer against the defined consumers.

In Spock’s setup methods, we defined the provider configuration with the name payment-service, and port used by SpringBoot. Also, informed that the contracts are stored in the broker that is available on localhost:9292. This causes the consumers which have a contract with this producer, to be taken from the broker together with expected interactions. Thanks to that, we don’t have to update this test, when e.g. interaction changes or a new consumer wants to integrate with payment-service. Whenever the payment-service’s tests are executed, all the contracts from all the consumers are taken from the Pact and tested.

Now, if you run the build of this application, the test should pass. However, if the API is not implemented according to the expectations in the contract, the test will fail with information about the problem.

This is only one possible way to run the contract tests on the producer’s side. I chose this one because all the examples in this series are written in Spock. However, another recommended way is to use a Gradle plugin to verify the producer. This reduces the code that must be written to test the producer. However, this type of testing is out of the scope of this article.

When working with Pact and Pact Broker, there is a good practice to publish the result of testing to the broker. This provides insights into whether the specific version of the service conforms to the consumers’ expectations. In order to do that, we have to configure two system properties:

• pact.verifier.publishResults - if set to true, the results are published to the broker
• pact.provider.version - version of the service to be published

test {
    useJUnitPlatform()
    systemProperty 'pact.provider.version', project.version
    systemProperty 'pact.verifier.publishResults', 'true'
}

Such configuration causes the results to be published to the broker after running the tests. On the main page of the broker’s GUI, we can see that the contract between the services was recently validated and the tests passed.

For every integration, Pact Broker manages and exposes the matrix of the versions. It’s available after clicking the grid icon on the contracts list.

It shows the history of contact validation between the services. We can see that version 1.1.0 has a bug, so we shouldn’t deploy it. But how can the CD pipeline know whether a specific version was tested and is eligible to be deployed to production? This is what the tool can-i-deploy can help us with. Thanks to this CLI we can ‘ask’ the Pact Broker whether the service is compatible and conforms to the published contracts. For example, we can check whether the latest version of payment-service may be safely deployed to production.

pact-broker can-i-deploy --pacticipant payment-service --latest

‘can-i-deploy’ informs us that this version is fine.

However, if we query for version 1.1.0, which according to the matrix above is not compatible, the failure will be reported.

pact-broker can-i-deploy --pacticipant payment-service --version 1.1.0

We can use this tool in our CD pipeline and validate service compatibility before deploying them to production. This protects us against deploying incompatible versions of the services.

Summary

In this article, I explained how the Pact framework works. We implemented the consumer’s tests which confirm that the order service’s code conforms to the communication contracts. Then, I showed how to generate the contract in Pact’s JSON format to be shared with the producer.

Also, the Pact broker was introduced as a recommended solution to share the contracts between parties. This allows the execution of the producer’s tests using the contracts defined by all the consumers. The producer’s tests are not even aware of who those consumers are.

Finally, I showed how the Pact Broker and the ‘can-i-deploy’ tool may be used in the CD pipeline, to protect the production against the deployment of incompatible services.

Within this article, I mentioned the main differences between Spring Cloud Contract and Pact framework. The most significant differences are:

• Pact supports only a consumer-driven approach while Spring Cloud Contract additionally allows a producer-driven.
• Pact’s contracts are language agnostic (JSON) and generated from the consumer’s tests. Spring Cloud Contract requires writing the contract by hand in one of the available DSLs.
• In Spring Cloud Contract we store the contracts in e.g. Git repository and share them between the parties when need to be, for instance, updated. Pact has a broker which improves sharing and managing the contracts.
• Spring Cloud Contract requires providing the stub of the producer to implement the consumer’s test. This stub is built from the contracts, which usually are stored in the producer’s repository. In Pact we start implementation of the consumer’s tests which can be finished without external stub (integration expectations are included in the tests).
• Pact provides additional features which simplify deployment management, e.g. can-i-deploy.

I hope you already have a basic understanding of the contract testing concepts, together with knowledge about those two frameworks. This should allow you to start implementing contract tests for the REST APIs in your project, using Spring Cloud Contract or Pact.

In the next article, I will explain how to test event-driven communication with Kafka brokers.

¹Currently, Spring Cloud Contract also supports non-Spring and non-JVM languages.