20 May 2020

Http Calls in Your Code and How to Test Them

9 minutes

This is a problem I always encountered whenever I try to test code that calls an external HTTP api. As web application developers, we call external APIs all the time.

And whenever that comes up, I entered into a dilemma of how to test that API call. First, let us consider our options:

Option 1: We can actually send the API call and observe the response.

This is obviously not pragmatic. If it’s not an idempotent API call (i.e the result is different if you call it more than once), it is not reliably reproducible. And, not all APIs have public sandbox endpoints that you can just hit. And another more important issue is the performance of the test suite. If you just keep hitting every external endpoint that you test, your unit test suite is going to be a lot slower. An HTTP call is definitely more expensive than a method call or even a file read operation!

So, that leaves us options that don’t actually hit the API. So, how do we test an API call if we don’t actually hit an API? Here comes the next option:

Option 2: Either mock, stub or fake the HTTP client code.

This is a very pragmatic option. Especially if the test framework you are using supports those stubs and mocks. That also means Option 2 has 3 sub-options.

The first sub-option is mocking the code that calls the API. That means you actually replace the part of the code that calls the API with something else at Runtime during test run. These days most unit test frameworks more or less support mocking of some level. The easiest point to mock is the HTTP client that you are using. As an example in Javascript, suppose you use Jest testing library. In Jest you can mock modules by having __mocks__ directory in your test file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42


// Production code mutateSomeRemoteResult.js
const axios = require('axios');

module.exports = {
  mutateSomeRemoteResult(apiKey, params) {
    return axios
      .post(`https://externalapi.com/stuff?apiKey=${apiKey}`, {
        params,
      })
      .then((res) => res.data)
      .catch((error) => console.log(error));
  },
};

// Test code (__mocks__/axios.js)
const axios = {
  post: jest.fn(() =>
    Promise.resolve({
      data: {
        /*..stuff..*/
      },
    })
  ),
};
module.exports = axios;

// Test code (getStuffFromAPI.js)
const mockAxios = require('axios');
const mutateSomeRemoteResult = require('mutateSomeRemoteResult');

describe('getStuffFromAPI', () => {
  it('calls api with the right params', async (done) => {
    await mutateSomeRemoteResult('apikey', { data: 'some data' });

    expect(mockAxios.get).toHaveBeenCalledWith(
      '<https://externalapi.com/stuff?apiKey=apikey>',
      { data: 'some data' }
    );

    done();
  });
});

So, the gist is the mocking code(i.e mockAxios module) is replacing the actual call and recorded what parameters have been passed in or how many times it has been called. And then in the test, you assert the http call was made correctly. In this approach, you don’t actually make the HTTP call, it’s just a shell module method. So, there is virtually no performance hit. You can repeat it as many time as you want. It is consistent. And in unit testing, you want consistency above everything else.

How about the sub-option 2 then? Stubs are a lesser form of mock actually. They are basically pre-made object that replaces the actual implementation and just gives you a canned value. The bottom line is you want to use stub when you care only about the result from something you want to mock. And use a mock when you want to make sure the wiring is downright correct. For example, we can directly stub the axios’s get function by using Jest’s mock functions.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40


// users.js
const axios = require('axios')

module.exports = {
  getStuffFromAPI(apiKey) {
    return axios
      .post(`https://externalapi.com/stuff?apiKey=${apiKey}`, {
        params: {
          apiKey: 'apikey',
        }
      })
      .then(res => res.data)
      .catch(error => console.log(error));
  },

  doStuffWithAPIData() {
    /* pre-api call code */
    const data = await getStuffFromAPI('apikey')

    /* do stuff with data */
    return result
  }
};

//Test code users.test.js
const axios = require('axios')
const doStuffWithAPIData = require('doStuffWithAPIData')

jest.mock('axios');

test('should fetch users', () => {
  const users = [{name: 'Bob'}];
  const resp = {data: users};
  axios.get.mockResolvedValue('some raw data to compute');

  const result = await doStuffWithAPIData()

  expect(result).toEqual('some legit result')
});

In the above example, all we care about is we get something from the API call. And we don’t particularly care about how it’s made. So, we stub the API call code with a canned result and the rest of the code uses that data as if it was directly from the API.

The third option is using a fake, a fake server to be precise. A fake is a simpler working implementation that will actually behave similarly, albeit in a simpler way, as the actual API. For example, we can implement a simple http server that will just return results based on some pre-configuration as we want.

A fake server is a more involved and the most tedious option of all three sub-options because you have to maintain that server and configuration code together with your test suite. And the server must be run before you run your test suite somehow. And it also suffers from the performance problem as the first option although a server running on the same machine will have the minimum amount of latency compared to the real one. Even still, it is still a lot slower than a method call.

Alright, all of those seems like pretty viable options. How about the third option? Well, the third one is combination of the first 2 options. It is more sophisticated and requires great tooling.

Option 3: Recording actual API calls and playing them back.

In Ruby programming language, there is a gem (a gem is a ruby library package) called VCR. The way it works is it hooks into several supported http clients (pretty much most http clients in ruby) and sets up some magical configuration around it to do the following:

For the first time you call an API in your test, it will actually call the API and record the response (in a predetermined file structure).
For subsequent calls, it will read the recorded result from the file strucutre and reconstructs the exact http response object and stubs it in-place of the API call.

So, overall, you get the best of both world. Your implementation code deal with real http response and you only pays the performance price once. And you can turn it on and off via configuration to renew the recorded response. The main gotcha is you actually need to call the API once. This means the API you call may need to return the data that is just right for your test cases you want to do. So, if you don’t have control over the API you call, it is a bit tricky.

And it can be less flexible compared to the second option.

Which option do I choose?

Now that we have these options, how should we choose? The answer, as always, is it depends.

My personal strategy is to go for second option most of the time because they are the easiest and the least daunting to set up. But, if an existing project has already set up something like VCR, I would happily use it. And, if the HTTP call is actually super important and we need to get it right 100%, I might actually write a fake server that can be configured for various scenarios that I am interested to get it tested. Or if it’s something like a payment API, I will use the sandbox version of the API to smoke test sucessful and failure scenario and run them only just before deployment as final check.

So, I would prefer to just isolate those http calls and stub them or mock them depending on what I want to accomplish. Again, command query distinction is the key when you decide whether to mock or stub. Use a stub for queries and a mock for commands.

API calls under several layers

So far, I have only used the HTTP client as the mocking point because most of the time it is the layer that you want to mock and isolate. But, sometimes, the API call is not from the code you implement but rather from a dependency that you can’t change or easily mock. In those cases, here are a few tips that I collected from experience.

I would first pinpoint the place where the http call is being made.
Then I would consider if that call is important for the test that I am trying to make (i.e if the actual API call needs to happen in order for the subsequent logic to work or do we just care about the result)
Often, it is not necessary, in that case I would just do the minimum set up that I can do to by-pass the HTTP call. Be it a stub at the boundary of the dependency or a mock directly at the underlying HTTP client implementation.
If it’s actually a core part of the logic, I would isolate the piece that interact with the dependency out into a separate adapter class and test that class in isloation using mocks and stubs for http call. And mock/stub that class again in my core logic so that I am mocking the code I own.

Often, it is tempting to say that you want your tests to verify things similar to production environment as much as possible. I mean that’s what testing is supposed to be right?

Yes, it is, ideally. However, the reality is you can never actually mimic the production environment. So, there is always a threshold that you allow to diverge from production. And I think those http calls are something that we cannot actually mimic in a real sense because we want other contrasting features like repeatability, consistency and performance in a test suite. And you can’t realistically simulate everything that can go wrong with the network anyway.