Built-In Cucumber Reports

What Are Built-In Cucumber Reports?

Built-in Cucumber reports are the reports generated by Cucumber after feature files, scenarios, and steps are executed. They summarize what ran, what passed, what failed, what was skipped, how long execution took, and where failures occurred. These reports help testers and developers understand automation results without first building a custom reporting system.

In simple terms, built-in Cucumber reports summarize the execution results of feature files, scenarios, and steps after a test run. They provide the first layer of visibility for a Cucumber framework. When a suite finishes, the team should not be left guessing whether all scenarios passed, which feature failed, which step stopped execution, or what information should be shared with CI/CD tools. Reports answer those questions.

Cucumber supports multiple built-in reporting formats. Some formats are meant for humans, such as console and HTML reports. Other formats are meant for tools, such as JSON, JUnit XML, and message reports. A mature framework usually generates more than one report format from the same execution because different consumers need different outputs.

Why Cucumber Reports Are Important

After executing automation tests, simply knowing that execution finished is not enough. A test run must produce information that can be analyzed. Reports tell the team which scenarios passed, which scenarios failed, which step failed, how long execution took, what percentage of tests passed, and which feature file contains failures. Without reporting, automation results become difficult to trust and difficult to act on.

Execution finished
  -> Generate report
  -> Analyze results
  -> Fix failures or confirm quality

Reports also make automation useful outside a developer's local machine. A CI server such as Jenkins, GitHub Actions, Azure DevOps, GitLab CI, Bamboo, or TeamCity needs structured output to mark builds as passed or failed. Developers need failure details to debug quickly. QA leads need summaries to understand test health. Product teams may need readable evidence that business scenarios were executed.

Good reports reduce communication friction. Instead of sending a message that says "some Cucumber tests failed," the team can inspect the report, identify the feature, scenario, step, error, and timing, and decide the next action. Reporting turns raw execution into useful feedback.

Reporting Flow

The reporting flow starts with feature files and a runner. The runner triggers Cucumber execution. During execution, Cucumber processes feature files, matches steps to step definitions, runs hooks, executes automation code, and records events. Reporting plugins consume those events and write report files in configured formats. After execution, humans and tools read those report files.

Feature Files
  -> Runner
  -> Cucumber Execution
  -> Plugins
  -> Reports Generated
  -> Analysis

This plugin-based design is useful because the same execution can generate multiple outputs. The console can show readable Pretty output for local debugging. HTML can provide a quick visual summary. JSON can feed advanced reporting libraries. JUnit XML can feed CI dashboards. Message reports can feed event-based integrations.

Types of Built-In Reports

Cucumber supports several built-in report formats. The most common are Console output, Pretty output, HTML report, JSON report, JUnit XML report, and Message report. Each format serves a different purpose. Choosing the right combination depends on whether the report is for local development, human review, CI/CD integration, third-party tools, or advanced processing.

Built-in reports
  Console Report
  Pretty Report
  HTML Report
  JSON Report
  JUnit XML Report
  Message Report

Do not think of these as competing options. In real frameworks, they are often generated together. A developer may look at Pretty output while running tests locally. A tester may open the HTML report. Jenkins may consume the JUnit XML file. A custom report generator may consume the JSON file. One run can satisfy all these needs.

Report Generation with Plugins

Reports are generated using the plugin option inside Cucumber runner configuration. In a Cucumber JVM project using annotations, the @CucumberOptions annotation commonly defines report plugins. Each plugin entry tells Cucumber what format to generate and where to place the output file.

@CucumberOptions(
  plugin = {
    "pretty",
    "html:target/cucumber-report.html",
    "json:target/cucumber.json",
    "junit:target/cucumber.xml"
  }
)

This configuration generates readable console output, an HTML report, a JSON report, and a JUnit XML report in the target folder. The exact runner style may vary based on JUnit, TestNG, Maven, or command-line execution, but the principle is the same. Plugins control report output.

Pretty Report

The Pretty plugin makes console output easier to read. Without readable formatting, console output can be difficult to scan during local development. With Pretty output, feature names, scenario names, steps, and statuses are displayed in a structured format. This is helpful when a developer or tester runs a small set of scenarios locally and wants immediate feedback in the terminal.

plugin = {"pretty"}

Pretty output is not usually the final reporting solution for enterprise frameworks. It is primarily useful during development and troubleshooting. It does not provide rich dashboards, historical reports, or CI-friendly structured results. Still, it is worth enabling because it improves day-to-day local execution.

Console Output

Console output is the first report many testers see. Without Pretty formatting, the output may show steps and failures in a less readable form. With Pretty formatting, the console follows the feature, scenario, step, and status flow. This makes it easier to identify where execution currently is and which step failed.

Feature
  -> Scenario
  -> Steps
  -> Status

Console output is temporary and local by nature. It is not ideal for sharing with the team unless logs are captured by CI. For permanent review, generate file-based reports such as HTML, JSON, and JUnit XML. Console output is best treated as a developer convenience, not the only reporting artifact.

HTML Report

The HTML report provides a user-friendly web page containing execution details. It is the most commonly viewed built-in report because it is easy to open in a browser and scan. A typical HTML report shows feature names, scenario names, step results, status, and execution details. It gives a quick human-readable summary of the run.

plugin = {
  "html:target/cucumber-report.html"
}

The output file is usually created inside the target directory.

target/cucumber-report.html

HTML reports are useful for local review and basic team sharing. However, built-in HTML reports are relatively simple compared with advanced tools such as Extent Reports, Allure, or ReportPortal. They are good for execution summaries, but they may not provide rich charts, history, trend analysis, or screenshot galleries without additional integration.

HTML Report Contents

The HTML report typically includes feature, scenario, steps, pass or fail status, and execution time. If a scenario fails, the report helps identify the failed step. This is valuable because Cucumber scenarios often contain several Given, When, Then, and And steps. Knowing the exact failed step helps separate setup failures from action failures and assertion failures.

Feature
  -> Scenario
  -> Steps
  -> Pass or Fail
  -> Execution Time

For readable reports, scenario names and step text matter. A report is only as useful as the scenario language. If scenarios are named "TC_001" or "Verify API," the report gives little business insight. If scenarios are named "Create customer successfully" or "Reject login with invalid password," the report immediately communicates behavior.

JSON Report

The JSON report is machine-readable. It contains structured execution information about features, scenarios, steps, tags, statuses, durations, errors, and metadata. Humans may not read the raw JSON directly, but tools can consume it. JSON reports are commonly used for custom dashboards, third-party reporting tools, report merging, and integrations.

plugin = {
  "json:target/cucumber.json"
}

A simplified JSON idea may look like this, though real output is more detailed.

{
  "feature": "Login",
  "scenario": "Valid Login",
  "status": "passed"
}

Many teams generate JSON even when they mainly view HTML because JSON can be used later by advanced tools. If the project grows and needs richer reporting, historical trends, or merged reports, having JSON output already available makes integration easier.

JUnit XML Report

JUnit XML is one of the most important report formats for CI/CD integration. Most CI tools understand JUnit XML. Jenkins, Azure DevOps, GitHub Actions, Bamboo, TeamCity, and GitLab CI can consume XML test results and show build dashboards, failed tests, timing, and trends. This makes JUnit XML useful even when the project is not using JUnit as the main test framework.

plugin = {
  "junit:target/cucumber.xml"
}

The XML report usually represents the test execution as test suites and test cases. CI tools can parse it automatically and mark builds as passed or failed. This is why enterprise projects should usually generate JUnit XML in every run. HTML is useful for people; XML is useful for pipelines.

XML Report Structure

The XML report contains information such as test suite, test cases, pass status, failure status, skipped status, and execution time. It is not designed to be pleasant for humans to read directly. Its value is that CI/CD systems understand it. Once published, CI dashboards can show which tests failed and how long they took.

Test Suite
  -> Test Cases
  -> Pass
  -> Fail
  -> Execution Time

Because CI systems depend on these files, report paths should be stable. If the pipeline expects target/cucumber.xml, avoid changing that path casually. Consistent report names make build scripts simpler and more reliable.

Message Report

The Message report uses newline-delimited JSON, often called NDJSON. It contains structured execution events and is intended mainly for tooling and advanced integrations. This format is more event-oriented than a simple summary report. It can represent the flow of Cucumber execution in detail.

plugin = {
  "message:target/messages.ndjson"
}

Most beginners do not need to use Message reports directly. They are useful when building advanced reporting tools, custom integrations, or event-based processing. For day-to-day framework usage, Pretty, HTML, JSON, and JUnit XML are more common.

Multiple Reports Together

Enterprise projects usually generate multiple report formats in the same execution. This is not duplication for its own sake. It serves different audiences. Developers get readable console output. Testers get HTML. CI gets JUnit XML. Reporting tools get JSON. Advanced integrations may get message reports.

plugin = {
  "pretty",
  "html:target/cucumber-report.html",
  "json:target/cucumber.json",
  "junit:target/cucumber.xml"
}

Each plugin runs independently. If configured correctly, one Cucumber execution can produce all required outputs. This is better than rerunning the same tests just to generate different reports. The report strategy should be part of framework design, not an afterthought.

Report Location

In Maven projects, reports are commonly stored under the target directory. This folder is generated during the build and can be cleaned before each execution. Typical output files include target/cucumber-report.html, target/cucumber.json, target/cucumber.xml, and target/messages.ndjson.

Project
  -> target
     -> cucumber-report.html
     -> cucumber.json
     -> cucumber.xml

Using target keeps generated artifacts separate from source files. Reports should generally not be committed to Git unless the project has a special reason. CI/CD pipelines can archive the generated files after each run. Local reports can be regenerated whenever tests run.

CI/CD Reporting Flow

CI/CD reporting connects automation execution to build feedback. A developer commits code. Jenkins or another CI tool runs Maven tests. Cucumber executes feature files. Plugins generate report files. The CI server publishes JUnit XML, archives HTML and JSON reports, and shows results in the dashboard. Notifications may be sent to email, Slack, Teams, or another channel.

Git Commit
  -> Jenkins
  -> Maven Test
  -> Cucumber
  -> JUnit XML
  -> Jenkins Dashboard

For CI/CD, JUnit XML is especially important because build servers understand it automatically. HTML reports are usually archived for manual inspection. JSON reports may feed advanced report generation. A good pipeline keeps these artifacts available after the build finishes so failures can be investigated later.

HTML Report Example

A simple HTML report may show that the Login feature ran, the Valid Login scenario passed, and execution took 1.25 seconds. If a scenario fails, the report identifies the failed step and the error message. This immediately narrows the investigation. The failure may be in setup, user action, API call, UI interaction, or validation.

Feature: Login
Scenario: Valid Login
Status: Passed
Execution Time: 1.25 sec

The report is more valuable when feature and scenario names are meaningful. "Valid Login" tells the reader the behavior. "Scenario 1" does not. Reporting quality depends on Gherkin quality.

Failed Step Information

When a scenario fails, Cucumber reports show where execution stopped. The failed step usually includes the exception or assertion error. This helps identify whether the problem is a test issue, data issue, environment issue, or product defect. A failed Given step often points to setup. A failed When step may point to action or request execution. A failed Then step often points to assertion or business validation.

Scenario
  -> Step
  -> Expected result
  -> Actual result
  -> Failure details

Clear failure messages improve reports. If an assertion says only "expected true but was false," the report is weak. If the assertion says "Expected customer status ACTIVE but found INACTIVE," the report is much better. Framework validators should produce meaningful assertion messages where possible.

Screenshot Integration

Built-in Cucumber reports do not automatically include screenshots in every framework. Screenshot capture usually requires additional code, often in hooks. In Selenium frameworks, an After hook can detect scenario failure, capture a screenshot, and attach it to the scenario report. This makes UI failures easier to debug.

Failure
  -> Hook
  -> Capture Screenshot
  -> Attach
  -> Enhanced Report

Screenshot attachment depends on the Cucumber version, runner, and reporting plugin behavior. Some advanced reporting tools handle screenshots more richly than built-in reports. Still, built-in reports can be enhanced with attachments when the framework is configured correctly.

Limitations of Built-In Reports

Built-in reports are useful, but they are not full enterprise dashboards. They generally do not provide rich analytics, historical trends, pie charts, defect grouping, team-level insights, screenshot galleries, flaky test tracking, advanced filtering, or release-level quality dashboards. They are designed primarily to summarize execution results.

For richer reporting, teams commonly use Extent Reports, Allure Reports, ReportPortal, custom dashboards, or CI analytics. These tools may consume Cucumber JSON, JUnit XML, or message reports and produce enhanced visualization. Built-in reports are a strong starting point, but large teams often outgrow them.

Built-In Reports vs Advanced Reports

Use built-in reports first because they are simple and reliable. Add advanced reports when the team needs richer analysis, historical reporting, management dashboards, or better visual evidence. Do not add complexity before there is a clear reporting need.

Common Mistakes

One common mistake is generating only an HTML report. HTML is useful for humans, but CI tools usually need XML, and advanced reporting tools often need JSON. Another mistake is forgetting the JSON report. Even if the team does not use JSON immediately, it is often needed later for report merging, dashboards, or integrations.

Teams also overwrite reports without archiving important results. For local runs this may be fine, but CI builds should archive reports so failures can be investigated after the build. Another mistake is changing report paths without updating pipeline configuration. If Jenkins expects one path and Cucumber writes another, reports will not be published.

Another common mistake is assuming built-in reports automatically include screenshots. Screenshot capture requires explicit framework logic. Finally, poor scenario names make reports hard to understand. Reporting starts with good Gherkin.

Best Practices

Generate HTML, JSON, and JUnit XML reports together in regular executions. Use Pretty output during local development for readable console feedback. Store generated reports in the target directory. Archive reports in CI/CD pipelines. Attach screenshots on UI failures using hooks when appropriate. Keep report names and paths consistent across environments.

Clean old reports before new executions when appropriate. Use JSON reports as input for advanced reporting tools. Use JUnit XML for CI dashboards. Use HTML for human review. Use advanced reporting tools when dashboards, screenshots, historical trends, and analytics are required. Keep reports safe by avoiding sensitive data exposure in logs or attachments.

Enterprise Reporting Architecture

In enterprise projects, reporting is part of the automation architecture. Feature files are executed by runners. Cucumber plugins generate Pretty, HTML, JSON, JUnit XML, and sometimes Message reports. CI/CD tools publish XML results, archive HTML and JSON artifacts, and send notifications. Advanced reporting tools may consume JSON or message files to create dashboards.

Feature File
  -> Runner
  -> Cucumber
  -> Plugin
  -> Pretty
  -> HTML
  -> JSON
  -> JUnit XML
  -> CI/CD
  -> Dashboard

This architecture allows one execution to feed multiple reporting consumers. Developers, testers, managers, and automated tools each get the format they need. That is the practical value of generating multiple report types.

Built-In Report Comparison

This comparison helps decide which reports to generate. For most frameworks, Pretty, HTML, JSON, and JUnit XML are a practical default. Message reports can be added when advanced tooling requires event data.

Report Archiving

Report archiving is important in CI/CD. If a build fails and reports are not archived, the team may lose the evidence needed to troubleshoot. CI jobs should publish JUnit XML and archive HTML, JSON, screenshots, logs, and any other useful artifacts. Archived reports should be tied to the build number, branch, commit, and execution time.

Historical archives help compare runs. If a scenario started failing after a specific commit, archived reports can show the timeline. If execution time increased, old reports can help identify when the slowdown started. Built-in reports do not provide full trend analysis by themselves, but archived artifacts provide the raw history needed for investigation.

Report Naming Strategy

Report names should be consistent and meaningful. A simple project can use target/cucumber-report.html, target/cucumber.json, and target/cucumber.xml. A larger project may include suite names, browser names, environment names, or timestamps when needed. For example, smoke and regression suites may write separate report files.

Be careful with timestamped paths in CI. They can be useful for archiving, but the pipeline still needs a predictable pattern to find them. A good strategy is to write reports to a known folder and let the CI server archive the whole folder. This keeps build configuration simple.

Reporting for Parallel Execution

Parallel execution can complicate reporting. When multiple runners or threads execute scenarios, reports may be generated separately or need merging. JSON reports are commonly used as inputs for merged reports. JUnit XML files can also be published as multiple test result files. The framework should avoid multiple processes writing to the same report file at the same time unless the tool supports it.

For parallel suites, define report paths carefully. Each runner can write to a unique JSON or XML file, and the CI pipeline can collect all matching files. This prevents corrupted report output and preserves scenario-level results. Reporting design should be reviewed before enabling parallel execution at scale.

Using Reports for Debugging

Reports should help debugging, not just summarize pass and fail counts. A useful report shows the failed feature, scenario, step, exception, execution time, and relevant attachments. In API automation, reports may include safe request and response details. In UI automation, reports may include screenshots. In both cases, sensitive data should be masked.

When reports are designed well, developers can often understand the failure without rerunning locally. This saves time and improves collaboration. When reports are weak, every CI failure becomes a manual reproduction task. Reporting quality directly affects team productivity.

Using Reports for Stakeholder Communication

Cucumber reports can also support stakeholder communication when scenarios are written in business language. A product owner or business analyst may not care about Java classes, locators, REST Assured syntax, or runner configuration, but they can understand a report that says "Customer creates order successfully" or "Payment is rejected for expired card." Reports become more valuable when the scenario names describe real behavior.

This does not mean every report should be sent directly to business stakeholders. Built-in reports are still technical execution artifacts. However, readable scenarios make it easier for QA leads and automation engineers to summarize results. A report with meaningful feature names, scenario names, and tags can support release discussions, defect triage, and regression sign-off.

Reports and Tags

Tags make reports easier to filter and understand. If scenarios are tagged with @Smoke, @Regression, @API, @UI, @Payment, or @Critical, the report output can help identify which category failed. Even when the built-in HTML report is simple, tags still appear in structured outputs such as JSON and can be used by advanced report processors.

Good tag strategy improves reporting strategy. A failed smoke test has a different urgency from a failed low-priority regression case. A failed payment test may need a different team than a failed customer-profile test. Tags should classify scenarios by execution type, module, risk, or technology so reports can be interpreted faster.

Reports for API Automation

In API automation, reports should show more than scenario names. Useful API report details may include endpoint, method, status code, response time, request body, response body, correlation ID, and validation failure. Built-in reports may not provide all of this automatically, but step logs, attachments, or enhanced reporting integrations can include these details when needed.

Be careful with sensitive data. API requests may contain tokens, passwords, API keys, session cookies, personally identifiable information, or payment-like data. Reports should mask or omit sensitive values. A report that helps debugging but leaks credentials creates a new problem. Safe reporting is part of responsible automation design.

Reports for UI Automation

In UI automation, reports are most useful when they identify the failed screen action and include screenshots for failed scenarios. Built-in reports can show the failed step, but screenshots require framework support. A Cucumber hook can capture the browser screenshot when a scenario fails and attach it to the report if the reporting setup supports attachments.

UI failures are often caused by timing, locator changes, browser behavior, data issues, or real application defects. A screenshot gives immediate context. It can show whether the page was still loading, whether an error message appeared, whether an element was hidden, or whether the browser was on the wrong page. Reports without screenshots often require rerunning the test to understand the failure.

Cleaning Old Reports

Old reports can confuse analysis if they remain in the output directory. A tester may open a stale report and think it belongs to the latest execution. Build tools often clean the target directory before running tests, but local runs and custom scripts may not. A framework should have a clear cleanup strategy for generated reports.

Cleaning reports does not mean losing important history. In CI/CD, reports should be archived before the workspace is cleaned. Locally, overwriting the latest report is usually fine. In pipelines, each build should preserve its own artifacts. This gives both clean current output and historical traceability.

Report Failures Caused by Configuration

Sometimes tests run correctly but reports are missing or incomplete because of configuration mistakes. The plugin path may be wrong. The output folder may not exist. The CI pipeline may look for XML files in the wrong directory. Parallel runners may overwrite each other's report files. A build cleanup step may delete reports before publishing them.

When reports are missing, verify the plugin configuration, target paths, runner execution, build logs, and CI artifact settings. Also check whether tests failed before Cucumber initialized report plugins. Reporting problems should be treated as framework issues because they reduce visibility into automation results.

Choosing a Reporting Strategy

A reporting strategy should answer who needs the report and what they need from it. Developers need fast failure details. Testers need scenario results and evidence. CI tools need structured pass/fail files. Leads may need summaries and trends. Advanced dashboards may need JSON or message data. A single HTML report rarely satisfies every need in a mature project.

A practical starting strategy is to generate Pretty, HTML, JSON, and JUnit XML for every normal run. Archive HTML and JSON. Publish JUnit XML to CI. Add screenshots for UI failures. Add advanced reporting only when the team needs richer dashboards, historical analytics, or better visual summaries. This keeps the setup useful without unnecessary complexity.

Maintaining Report Quality

Report quality depends on more than plugin configuration. It depends on scenario naming, step wording, assertion messages, tag strategy, logging, attachments, and artifact archiving. A technically generated report can still be poor if scenario names are vague and failures lack context. A good report tells a clear story about what behavior was tested and what went wrong.

Review reports periodically. Look at failed scenarios and ask whether the report gives enough information to act. If the answer is no, improve assertion messages, attach useful evidence, rename unclear scenarios, or add safe logs. Reports should evolve with the framework. They are part of the product quality feedback loop.

Interview-Ready Summary

Built-in Cucumber reports provide execution summaries without requiring third-party reporting libraries. Cucumber supports multiple report formats, including Pretty, HTML, JSON, JUnit XML, and Message reports. Reports are configured using the plugin option in @CucumberOptions or equivalent runner configuration.

Enterprise frameworks commonly generate HTML for human review, JSON for reporting tools, and JUnit XML for CI/CD platforms. Pretty output helps during local development. Message reports support advanced event-based integrations. Built-in reports are suitable for execution summaries, while advanced reporting tools are preferred for dashboards, screenshots, historical trends, and deeper analytics.

Golden Rules

Generate HTML, JSON, and JUnit XML reports in every important execution. Use the Pretty plugin for readable console output during development. Store reports in the target directory and archive them in CI/CD pipelines. Use JSON reports as the source for advanced reporting tools and integrations. Use built-in reports for execution summaries and advanced tools for richer analytics and visualization.

The practical takeaway is simple: Cucumber reports turn automation execution into understandable feedback. A strong framework does not stop at running tests; it produces clear evidence that people and tools can use.