Testing with Testcontainers and Jqwik in Docker

When testing complex software systems, it’s crucial to identify as many edge cases as possible to ensure optimal performance in real-world scenarios. The challenge, however, lies in efficiently generating hundreds or even thousands of meaningful tests that can uncover hidden bugs. This is where model-based testing (MBT) comes into play. MBT is a testing technique that automates the generation of test cases by creating a model that represents the expected behavior of the software.

In this article, we will explore the model-based testing approach by applying it to regression testing on a simple REST API. We’ll be using the jqwik test engine on JUnit 5 to run both property-based and model-based tests. Additionally, we’ll utilize Testcontainers to launch Docker containers with different versions of our application, allowing us to test various scenarios effortlessly.

Understanding Model-Based Testing

Model-based testing is a method used to test stateful software by comparing the software component being tested with a model that represents the expected behavior of the system. Instead of manually writing test cases, we use a testing tool to:

1. Identify a list of possible actions supported by the application.
2. Automatically generate test sequences from these actions, targeting potential edge cases.
3. Execute these tests on both the software and the model, and compare the results.

   In our demonstration, the actions are simply the endpoints exposed by the application’s API. We will use a basic service with a CRUD REST API that allows us to:

   • Find an employee by their unique employee number.
   • Update an employee’s name.
   • Retrieve a list of all employees from a specific department.
   • Register a new employee.

     The image below illustrates the process of finding an employee, updating their name, identifying their department, and registering a new employee.

     

     Once we have everything configured and ready, running the test will result in a rapid sequence of hundreds of requests being sent to the two stateful services.

     Leveraging Docker Compose
     

     Imagine we need to switch the database from Postgres to MySQL while ensuring that the application’s behavior remains consistent. To test this, we can run both versions of the application, send identical requests to each, and compare the responses. Docker Compose is a tool that can help us set up the environment by running two versions of the app:

4. Model (mbt-demo:postgres): The current live version and our source of truth.
5. Tested version (mbt-demo:mysql): The new feature branch under test.

   Below is a sample Docker Compose configuration for our services:

   “`yaml
   services:

   MODEL
   

   app-model:
   image: mbt-demo:postgres
   depends_on:

   • postgres
     postgres:
     image: postgres:16-alpine

     TESTED
     

     app-tested:
     image: mbt-demo:mysql
     depends_on:

     • mysql
       mysql:
       image: mysql:8.0
       “`

       Automating Tests with Testcontainers
       

       To streamline our testing process, we can use Testcontainers’ ComposeContainer to automate the setup with our Docker Compose file during the testing phase. By doing this, we avoid the tedious task of starting the application and databases manually.

       In our setup, we’ll use jqwik as our JUnit 5 test runner. First, we’ll need to add the necessary dependencies to our pom.xml file:

       “`xml

       
       net.jqwik
       jqwik
       1.9.0
       test
       
       
       net.jqwik
       jqwik-testcontainers
       0.5.2
       test
       
       
       org.testcontainers
       testcontainers
       1.20.1
       test
       
       “`

       With these dependencies in place, we can instantiate a ComposeContainer and pass our test Docker Compose file as an argument:

       “`java
       @Testcontainers
       class ModelBasedTest {

       @Container
       static ComposeContainer ENV = new ComposeContainer(new File(“src/test/resources/docker-compose-test.yml”))
       .withExposedService(“app-tested”, 8080, Wait.forHttp(“/api/employees”).forStatusCode(200))
       .withExposedService(“app-model”, 8080, Wait.forHttp(“/api/employees”).forStatusCode(200));

       // tests
       }
       “`

       ### Creating a Test HTTP Client

       Next, we’ll create a small utility to help us execute HTTP requests against our services:

       “`java
       class TestHttpClient {
       ApiResponse get(String employeeNo) { /* … */ }

       ApiResponse put(String employeeNo, String newName) { /* … */ }

       ApiResponse> getByDepartment(String department) { /* … */ }

       ApiResponse post(String employeeNo, String name) { /* … */ }

       record ApiResponse(int statusCode, @Nullable T body) { }

       record EmployeeDto(String employeeNo, String name) { }
       }
       “`

       Additionally, in the test class, we can declare another method that helps us create `TestHttpClient` instances for the two services started by the ComposeContainer:

       “`java
       static TestHttpClient testClient(String service) {
       int port = ENV.getServicePort(service, 8080);
       String url = “http://localhost:%s/api/employees”.formatted(port);
       return new TestHttpClient(service, url);
       }
       “`

       ### Introduction to jqwik

       Jqwik is a property-based testing framework for Java that integrates with JUnit 5. It automatically generates test cases to validate code properties across diverse inputs, thereby enhancing test coverage and uncovering edge cases. By using generators to create varied and random test inputs, jqwik can effectively test the boundaries and potential breakpoints of your code.

       For those new to jqwik, it’s worth exploring their API in detail through the [official user guide](https://jqwik.net/docs/current/user-guide.html). While this tutorial won’t cover all API specifics, it’s crucial to understand that jqwik allows us to define a set of actions we want to test.

       To start, we’ll use jqwik’s `@Property` annotation — instead of the traditional `@Test` — to define a test:

       “`java
       @Property
       void regressionTest() {
       TestHttpClient model = testClient(“app-model”);
       TestHttpClient tested = testClient(“app-tested”);
       // …
       }
       “`

       Next, we’ll define the actions, which are the HTTP calls to our APIs. These actions can also include assertions. For instance, the `GetOneEmployeeAction` will attempt to fetch a specific employee from both services and compare the responses:

       “`java
       record ModelVsTested(TestHttpClient model, TestHttpClient tested) {}

       record GetOneEmployeeAction(String empNo) implements Action {
       @Override
       public ModelVsTested run(ModelVsTested apps) {
       ApiResponse actual = apps.tested.get(empNo);
       ApiResponse expected = apps.model.get(empNo);

       assertThat(actual)
       .satisfies(hasStatusCode(expected.statusCode()))
       .satisfies(hasBody(expected.body()));
       return apps;
       }
       }
       “`

       Additionally, we need to wrap these actions within Arbitrary objects. Arbitraries are akin to factories that can generate a wide variety of instances of a type, based on a set of configured rules.

       For example, the Arbitrary returned by `employeeNos()` can generate employee numbers by choosing a random department from the configured list and concatenating a number between 0 and 200:

       “`java
       static Arbitrary employeeNos() {
       Arbitrary departments = Arbitraries.of(“Frontend”, “Backend”, “HR”, “Creative”, “DevOps”);
       Arbitrary ids = Arbitraries.longs().between(1, 200);
       return Combinators.combine(departments, ids).as(“%s-%s”::formatted);
       }
       “`

       Similarly, `getOneEmployeeAction()` returns an Arbitrary action based on a given Arbitrary employee number:

       “`java
       static Arbitrary getOneEmployeeAction() {
       return employeeNos().map(GetOneEmployeeAction::new);
       }
       “`

       After declaring all the other Actions and Arbitraries, we’ll create an ActionSequence:

       “`java
       @Provide
       Arbitrary> mbtJqwikActions() {
       return Arbitraries.sequences(
       Arbitraries.oneOf(
       MbtJqwikActions.getOneEmployeeAction(),
       MbtJqwikActions.getEmployeesByDepartmentAction(),
       MbtJqwikActions.createEmployeeAction(),
       MbtJqwikActions.updateEmployeeNameAction()
       ));
       }
       “`

       “`java
       static Arbitrary> getOneEmployeeAction() { /* … */ }
       static Arbitrary> getEmployeesByDepartmentAction() { /* … */ }
       // same for the other actions
       “`

       Now, we can write our test and leverage jqwik to use the provided actions to test various sequences. Let’s create the `ModelVsTested` tuple and use it to execute the sequence of actions against it:

       “`java
       @Property
       void regressionTest(@ForAll(“mbtJqwikActions”) ActionSequence actions) {
       ModelVsTested testVsModel = new ModelVsTested(
       testClient(“app-model”),
       testClient(“app-tested”)
       );
       actions.run(testVsModel);
       }
       “`

       That’s it — we can finally run the test! The test will generate a sequence of thousands of requests trying to find inconsistencies between the model and the tested service.

       ### Identifying and Fixing Errors

       If we run the test and examine the logs, we might quickly spot a failure. For instance, when searching for employees by department with a specific argument, the model might produce an internal server error, whereas the test version returns a 200 OK status.

       Upon investigation, we might find that the issue arises from a native SQL query using Postgres-specific syntax to retrieve data. Although this was a simple issue in our small application, model-based testing can help uncover unexpected behaviors that might only surface after specific sequences of repetitive steps push the system into a particular state.

       

       In conclusion, this article provided hands-on examples of applying model-based testing in practice. From defining models to generating test cases, we’ve seen how this powerful approach can improve test coverage and reduce the manual effort involved in testing. Now that you’ve seen the potential of model-based testing to enhance software quality, it’s time to dive deeper and tailor it to your own projects.

       For those interested in experimenting further, you can clone the [GitHub repository](https://github.com/etrandafir93/model-based-testing-practice) to customize the models and integrate this methodology into your testing strategy. Start building more resilient software today!

       Thank you to Emanuel Trandafir for contributing this post. For more information and resources, be sure to explore the references provided.

For more Information, Refer to this article.