Recording APIs with WireMock

JavaWireMockSpringBootTesting
Recording the API

When an application depends on a third party API, you always seem to face similar challenges. Do you integrate against it, even in development? What about the tests, won’t they be flaky if the API needs to be there?

Meanwhile, if you decide to use stubs, how sure are you that the application is, well, working? I can’t remember how many times I have had perfectly green tests based on mocks that were not testing anything, because the API had changed in some way.

Recently I’ve been thinking about this, and I have tried to address this through API recordings. I know this idea from the Ruby world, concretely VCR. This time, however, I wanted to use it in a Java project, so I looked at WireMock.

Why?

Following the Testing Pyramid, we want to have most of our tests at the lowest possible level. Unit tests should not be doing network requests. On the other hand, if we use mocks, they should be close to the source, ideally based on actual requests.

If we have these mocks already, why not use them for our local development? I want my app to boot and quickly show something, again without the need to connect to the outside.

And, I want automation. Editing .json files by hand is a recipe for errors and outdated data. I want to avoid friction.

Thanks to WireMock, this process is very convenient and almost transparent to the app. A bit of plumbing is required, but it is real easy to do.

Mock me!

I have this repository in Github with all the details. It can be summarized in one picture:

Recording the API

For the example API, I took this very useful sample API. There are four pieces to take into account in this setup.

Production

The app in production mode goes directly to the API, just like that. The request is something like this:

public List<Todo> todos() {
    ResponseEntity<List<Todo>> response = template.exchange(
            "/todos",
            HttpMethod.GET,
            null,
            new ParameterizedTypeReference<List<Todo>>() {});
    return response.getBody();
}

Development

For the development server, we rely on certain mocks being present, that we will serve transparently by starting a WireMock server on startup. To do that in a Spring Boot project, we use this initializer:

@Component
public class MockServerInitializer {
    @Value("${mock.active}")
    boolean active;

    @Value("${mock.port}")
    int port;

    @EventListener
    public void onApplicationEvent(ContextRefreshedEvent event) {
        if(active) {
            WireMockServer server = new WireMockServer(port);
            server.start();
        }
    }
}

It is important to set mock.active to false for any non-development environment. Other than that, the configuration points to the local server instead of the real API:

serverUrl: http://localhost:${mock.port}
mock:
  active: true
  port: 8080

Unit tests

The unit tests that need to use this mock data will use the same mocks from WireMock that the development server uses. Activating the mocks is as easy as inheriting from the RecordedTest class, which looks like this:

public abstract class RecordedTest {
    @Value("${record.port}")
    int port;

    WireMockServer server;

    @Before
    public void useMocks() throws Exception {
        server = new WireMockServer(port);
        server.start();
    }

    @After
    public void stopUsingMocks() throws Exception {
        server.stop();
    }
}

Inheritance and @Before and @After annotations is perhaps not a best practice, but it makes the tests really succint. Defining additional behavior is easy by just defining extra WireMock rules. This way you can test more specific scenarios like timeouts while keeping most of the regular code quite clean.

Integration tests

This is where everything comes together. Our integration tests make sure that the connection to the real API works. In the process, the interactions get recorded so that we can store that as the mocks that the rest of the app uses. To enable a test to become a RecordingTest, you need to inherit the class:

public abstract class RecordingTest {
    @Value("${record.port}")
    int port;

    abstract String recordingServerUrl();

    @Value("${record.persist}")
    private boolean persistRecordings;

    @Value("${record.extractBody}")
    private int extractBody;

    private WireMockServer server;

    @Before
    public void setUp() throws Exception {
        server = new WireMockServer(port);
        server.start();
        server.startRecording(config());
    }

    @After
    public void tearDown() throws Exception {
        server.stopRecording();
        server.stop();
    }

    private RecordSpec config() {
        return recordSpec()
                .forTarget(recordingServerUrl())
                .makeStubsPersistent(persistRecordings)
                .extractTextBodiesOver(extractBody)
                .build();
    }
}

This class proxies all the requests to the real API, and stores them in the default folder when configured. The URL of the server that we want to integrate with is defined in the abstract method recordingServerUrl. This way each test can define a different endpoint, and we are not limited to a single external endpoint.

Why not every time? We don’t want new files being created each time a test runs. Instead, this is a conscious decision, triggered with this script target:

goal_refresh-recordings() {
  RECORD_PERSIST=true RECORD_EXTRACTBODY=0 ./gradlew clean integration
}

After that the new recordings are present and ready to be used. This can be done manually or integrated into a pipeline, to make sure that it happens often.

NOTE: If the Third Party API is secured through mTLS, you can still make this setup work by making WireMock aware of the keystore that your code uses to connect to it, by creating a custom config for the WireMock server:

server = new WireMockServer(options()
        .trustStorePath(System.getProperty("javax.net.ssl.keyStore"))
        .trustStorePassword("changeit")
        .port(port));

Summary

The setup contains a bit of magic, but the result is quite simple to use. This way, you can achieve a great testing coverage without compromising the development experience, all while keeping your core unit tests slim and fast. If you have fought with out-of-sync mocks before, you will see the advantage!

EDIT 05/08/2019: Added support for multiple external APIs

Related Posts

Keep Reading