Test Reporting API

The Test Reporting API allows you to capture test execution events programmatically and produce results similar to Gradle’s built-in Test task. It’s particularly useful to integrate non-JVM test results into Gradle’s testing infrastructure and publish them in Gradle’s HTML test reports.

The API is primarily targeted at plugin developers and platform providers, with the Javadoc serving as the main reference.

Let’s take a look at a quick example that defines a custom Gradle task, CustomTest, which demonstrates how to capture and report test events:

src/main/java/com/example/CustomTest.java

package com.example;

import org.gradle.api.DefaultTask;
import org.gradle.api.file.ProjectLayout;
import org.gradle.api.tasks.TaskAction;
import org.gradle.api.tasks.testing.GroupTestEventReporter;
import org.gradle.api.tasks.testing.TestEventReporter;
import org.gradle.api.tasks.testing.TestEventReporterFactory;
import org.junit.platform.launcher.Launcher;
import org.junit.platform.launcher.LauncherDiscoveryRequest;
import org.junit.platform.launcher.core.LauncherDiscoveryRequestBuilder;
import org.junit.platform.launcher.core.LauncherFactory;
import org.junit.platform.launcher.listeners.SummaryGeneratingListener;
import org.junit.platform.launcher.listeners.TestExecutionSummary;

import javax.inject.Inject;
import java.time.Instant;
import java.util.ArrayList;
import java.util.List;

import static org.junit.platform.engine.discovery.DiscoverySelectors.selectClass;

/**
 * A custom task that demonstrates the {@code TestEventReporter} API.
 */
public abstract class CustomTest extends DefaultTask {

    @Inject
    public abstract ProjectLayout getLayout();

    @Inject
    protected abstract TestEventReporterFactory getTestEventReporterFactory();


    @TaskAction
    void runTests() {
        try (GroupTestEventReporter root = getTestEventReporterFactory().createTestEventReporter(
            "root",
            getLayout().getBuildDirectory().dir("test-results/custom-test").get(),
            getLayout().getBuildDirectory().dir("reports/tests/custom-test").get()
        )) {
            root.started(Instant.now());

            List<String> failedTests = new ArrayList<>();

            try (GroupTestEventReporter junittest = root.reportTestGroup("CustomJUnitTestSuite")) {
                junittest.started(Instant.now());

                LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
                    .selectors(
                        selectClass(MyTest1.class)
                    )
                    .build();

                Launcher launcher = LauncherFactory.create();
                TestExecutionSummary summary = executeTests(launcher, request);

                summary.getFailures().forEach(result -> {
                    try (TestEventReporter test = junittest.reportTest(result.getTestIdentifier().getDisplayName(),
                        result.getTestIdentifier().getLegacyReportingName())) {
                        test.started(Instant.now());
                        String testName = String.valueOf(result.getTestIdentifier().getParentIdObject());
                        failedTests.add(testName);
                        test.metadata(Instant.now(),"Parent class:", String.valueOf(result.getTestIdentifier().getParentId().get()));
                        test.failed(Instant.now(), String.valueOf(result.getException()));
                    }
                });

                String failedTestsList = String.join(", ", failedTests);
                junittest.metadata(Instant.now(), "Tests that failed:", failedTestsList);

                if (summary.getTestsFailedCount() > 0) {
                    junittest.failed(Instant.now());
                } else {
                    junittest.succeeded(Instant.now());
                }
            }

            if (!failedTests.isEmpty()) {
                root.failed(Instant.now());
            } else {
                root.succeeded(Instant.now());
            }
        }
    }

    private TestExecutionSummary executeTests(Launcher launcher, LauncherDiscoveryRequest request) {
        // Use SummaryGeneratingListener to collect test execution data
        SummaryGeneratingListener listener = new SummaryGeneratingListener();
        launcher.registerTestExecutionListeners(listener);

        // Execute the tests
        launcher.execute(request);

        // Return the summary generated by the listener
        return listener.getSummary();
    }
}

The GroupTestEventReporter is initialized to act as the root event reporter.

It creates directories for test results and reports under the project’s build directory:

This establishes the reporting hierarchy. Nesting is supported.
The task uses JUnit Platform to dynamically discover and execute tests, this would be replaced with your custom test system/platform.

It selects the class MyTest1 for testing.

JUnit’s Launcher is used to execute the tests and collect a summary of the results.
The TestEventReporter is used to record detailed events for each test.

For each failure, the test is reported with metadata and marked as failed.
Each test group (junittest) and the root (root) are finalized with success or failure based on the test results.

By using the Test Reporting API:

Integration: Custom tests (even those outside JVM frameworks) can generate events compatible with Gradle’s test reporting infrastructure.
Rich Metadata: Developers can attach additional context (e.g., error messages or parent class details) to test events.
Consistency: Results integrate seamlessly with Gradle’s HTML reports, making it easier to debug and understand test outcomes.

For more details on the HTML report itself, see the JVM Testing Documentation.

For a downloadable example, refer to the Custom Test Task Sample.

Was this page helpful?