Achieve true Test Coverage by understanding the system’s behavior.
There is no doubt about the importance of Test Coverage in software engineering, and today, with the increasing popularity of microservice architecture and distributed systems, it has become more critical. In this article, we’ll show how to increase test coverage by using Tracing, one of the integral components of distributed systems. We will also examine best practices and tools in a sample project implemented using Java and Spring Boot.
What is Test Coverage?
It is better to start by knowing what Test Coverage is exactly before discussing its conjunction with Tracing to improve Test Coverage. In short
The term Test Coverage refers to the extent to which a software product’s functionality has been tested.
It assesses how well the tests cover various aspects of the product functionality, such as different use cases, user scenarios, or edge cases. Test coverage can be measured at various levels, including:
- Functional Coverage
- Requirement Coverage
- Risk Coverage
- And more
Higher test coverage means more aspects of the software product have been tested, ensuring better quality.
Test Coverage vs Code Coverage
Two similar terms in software testing are Test Coverage and Code Coverage. However, they are not the same. Knowing the difference between the two is important and helpful for a better understanding of Test Coverage.
Code coverage is a measure that shows how many percentage percentages of lines, branches, functions, or conditions in the source code have been executed during testing.
Code Coverage focuses on the code rather than the functionality. Higher code coverage indicates that the tests have executed most of the code and will reduce the bugs or issues in the codebase.
Consequently, Test Coverage measures the degree to which the software’s functionality has been tested, while Code Coverage determines the proportion of the codebase the tests have executed.
What is the gap to have a true test coverage?
As mentioned in the previous section, implementing a distributed system is complex, and having acceptable test coverage for it is even more difficult.
A distributed system consists of multiple services that communicate with each other over the network.
This distributed nature introduces additional complexities in different aspects, such as communication, deployment, data consistency, and more.
To address these challenges, development teams need to adopt testing strategies such as:
- End-to-end testing
- Contract testing
- Leveraging tracing data
- Service virtualization
How can tracing data improve test coverage?
By leveraging tracing, we can improve test coverage by better understanding the system’s behavior and interactions between its components.
Without tracing, you may have tests in place to verify the functionality of each individual microservice. However, you may not have complete visibility into how these services interact with each other and whether the end-to-end flow.
Relation between end-to-end tests and Tracing data
To have even better coverage, end-to-end tests can help us a lot, instead of mocking interaction with other services, we try to run a real version of those services and interact with them, either by running them locally from code or in a docker container.
End-to-end testing enables us to ensure better test coverage, but it can be more challenging and resource-intensive to set up and maintain.
Since we interact with the real version of the system in end-to-end testing, we can analyze the system’s runtime behavior using tracing data.
Let’s get our hands dirty with real code
caller-service interaction diagram
Let’s continue with an imaginary distributed application with three services (caller-service, decorate-service, and echo-service). These services are implemented using Java and Spring Boot (find the source code here).
As you can see in the diagram, the caller-service needs to call decorate-service and echo-service to build a given message from a REST API.
If we run all three projects and then call the /build API from the caller-service we will see the decorated result!
http :8080/caller/build/deli -> Echo:**deli**
How Digma can help us to have better Test Coverage
Digma is an IDE plugin that provides many handy observability features during development and allows us to trace our code quickly.
Digma can help developers identify areas in the code that have not been properly tested, allowing developers or testers to improve the effectiveness of the test suite.
They allow the developer to find the code test usage, which can be challenging in a distributed application due to several applications, various programming languages, and more.
Digma pinpoints the tests that indirectly referenced the modified method, allowing the developer to select test execution and significantly reduce the testing time.
After installing the Digma plugin in your IDE, we need to re-run all three projects again. When we run the end-to-end test again. We will see the results in the Observability view in IntelliJ like this:
Digma’s Observability view in IntelliJ
To see tracing data, we can click on the Trace button on the right.
Digma’s Tracing view in IntelliJ
In addition, Digma gives us a lot of helpful insights about services such as:
- Scaling Issues
- Performance Impact
- Bottlenecks
- SQL Query Issues
- Chatty Microservices
- And more …
With Digma, we can also run all the tests that are going through the code you want to check.
Now that everything is running let’s start writing tests and using Digma to trace data to see how the code runs.
Write integration test using MockWebServer
One common practice for writing integration tests for this type of application is to mock service interactions. In our case, we will mock the echo-service and the decorator-service using the MockWebServer API.
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
class CallerControllerIntegTest {
private static int MOCK_SERVER_PORT;
static {
try (var serverSocket = new ServerSocket(0)) {
MOCK_SERVER_PORT = serverSocket.getLocalPort();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
@DynamicPropertySource
static void trackerProperties(DynamicPropertyRegistry registry) {
String mockServerUrl = "http://localhost:" + MOCK_SERVER_PORT;
registry.add("clients.decorate-service.url", () -> mockServerUrl);
registry.add("clients.echo-service.url", () -> mockServerUrl);
}
private MockWebServer mockWebServer;
@Autowired
WebTestClient webTestClient;
@BeforeEach
void beforeEach() throws IOException {
mockWebServer = new MockWebServer();
mockWebServer.start(MOCK_SERVER_PORT);
}
@AfterEach
void afterEach() throws IOException {
mockWebServer.shutdown();
}
@Test
void buildApi_validRequest_buildDecoratedMessage() {
var decorateJsonResponse = new MockResponse()
.setResponseCode(OK.value())
.setBody("**test**")
.addHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
mockWebServer.enqueue(decorateJsonResponse);
var echoJsonResponse = new MockResponse()
.setResponseCode(OK.value())
.setBody("""
{
"message": "Echo:**test**"
}
""")
.addHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
mockWebServer.enqueue(echoJsonResponse);
webTestClient
.get()
.uri("/caller/build/test")
.exchange()
.expectStatus()
.isOk()
.expectBody(String.class)
.isEqualTo("Echo:**test**");
}
}
Testing services individually might not encompass all scenarios since it could overlook the dependencies and interactions between services.
If you have installed the Digma plugin in IntelliJ, when you run your integration or end-to-end tests in the IDE, Digma will automatically detect this and add observability to each test context, and you can see related tests to an asset in the Digma view in the test tab.
Digma’s Test view in IntelliJ
As you can see in the image above, Digma allows us to see the Test Coverage for each asset (in our case, assets are endpoints, but in general, they can be database queries or code locations). The main benefit is that we can see which tests are covering our endpoints and check the Test Coverage practically. We can also re-run tests as needed after making changes from the Digma view.
By clicking on the Open Trace button, you can see this trace view inside the IDE. As you can see, we can not see any real interaction between the caller-service and other services since we mocked the dependencies.
Digma’s tracing view for tests in IntelliJ
Write end-to-end tests without mocking interactions
Let’s go one step further and write end-to-end tests, for the sake of simplicity, We don’t want to containerize services and use the Spring Boot ServiceConnection features to test them. If you are interested in more information about ServiceConnection, read this article.
This way, our end-to-end tests will be very simple:
class CallerServiceEndToEndTests {
@Test
void buildApi_validRequest_buildDecoratedMessage() {
WebTestClient.bindToServer().baseUrl("http://localhost:8080").build()
.get()
.uri("/caller/build/deli")
.exchange()
.expectStatus()
.isOk()
.expectBody(String.class)
.isEqualTo("Echo:**deli**");
}
}
The only thing that you need to consider is we need to run all three services before running our end-to-end test.
If we follow the same process in the integration test section in the Digma view, we will see real interaction between the services in the Digma’s trace view.
Digma’s tracing view for tests in IntelliJ
It can be a great source of observability for us during development. We will be able to study the application in a controlled experiment, which is a huge opportunity to uncover issues and study how the system behaves to increase the Test Coverage.
Final Thought
Proper test Coverage is challenging in distributed systems, but the article shows that we can significantly improve it by writing end-to-end tests and leveraging tracing data provided by tools like Digma.
Download Digma: Here
