Playwright test report

To understand and effectively utilize Playwright test reports, here are the detailed steps: The Playwright test runner, by default, generates a comprehensive HTML report after every test run. This report is your go-to dashboard for quickly assessing test outcomes, drilling down into failures, and optimizing your testing workflow. To access it, simply run your tests, and if failures or flaky tests exist, Playwright will automatically open the report in your browser. If not, you can manually open it by executing npx playwright show-report. This report provides a visual breakdown, showing passed, failed, skipped, and flaky tests, along with crucial details like test duration, browser, and project.

👉 Skip the hassle and get the ready to use 100% working script (Link in the comments section of the YouTube Video) (Latest test 31/05/2025)

Check more on: How to Bypass Cloudflare Turnstile & Cloudflare WAF – Reddit, How to Bypass Cloudflare Turnstile, Cloudflare WAF & reCAPTCHA v3 – Medium, How to Bypass Cloudflare Turnstile, WAF & reCAPTCHA v3 – LinkedIn Article

Understanding the Playwright HTML Report

The HTML report is highly interactive. Each test entry can be expanded to reveal:

• Steps: A detailed breakdown of actions taken during the test.
• Before/After Hooks: Information on setup and teardown processes.
• Attachments: Screenshots, videos, and trace files.
• Errors: Clear error messages and stack traces for failed tests.

This integrated approach makes debugging significantly faster, reducing the time you spend sifting through console logs.

Beyond HTML: Other Report Formats

While HTML is the default and most user-friendly, Playwright also supports other report formats to integrate with various CI/CD pipelines and reporting tools:

• list reporter: Prints a simple list of test names as they run.
• dot reporter: Shows dots for passed tests and ‘F’ for failed tests.
• json reporter: Exports test results as a machine-readable JSON file, perfect for custom parsers or dashboards. You can specify this using playwright.config.ts or CLI: npx playwright test --reporter=json.
• junit reporter: Generates an XML report compatible with CI systems like Jenkins, GitLab CI, and Azure DevOps. Configure it in playwright.config.ts or via CLI: npx playwright test --reporter=junit.
• allure reporter: A popular third-party reporter for generating rich, interactive reports with historical data, trends, and more. This requires additional setup with the allure-playwright package.
• Custom reporters: For highly specific needs, Playwright allows you to write your own custom reporters, giving you full control over how test results are processed and presented. You’ll implement a class conforming to Playwright’s Reporter interface.

Leveraging the right report format for your context can significantly enhance visibility into your test suite’s health and streamline your development process.

Demystifying Playwright Test Reporting: A Deep Dive into Post-Execution Insights

Playwright, the modern web automation framework from Microsoft, isn’t just about robust test execution. it’s also about providing unparalleled visibility into your test outcomes. The effectiveness of any test suite hinges not only on its coverage but also on how easily you can interpret its results, identify failures, and pinpoint the root cause. This is where Playwright’s sophisticated test reporting capabilities truly shine, transforming raw test output into actionable insights. Understanding and leveraging these reports is akin to having an X-ray vision into your application’s health, allowing teams to react swiftly, iterate faster, and maintain a high standard of quality.

The Cornerstone: Playwright’s Built-in HTML Reporter

Playwright’s default HTML reporter is a must for local development and rapid debugging. It’s designed to be highly visual, interactive, and comprehensive, giving you immediate feedback on your test runs. This isn’t just a static log file. it’s a dynamic dashboard that aggregates all the essential information from a test run into a single, navigable interface.

Automatic Generation and Access

By default, whenever you run your Playwright tests, an HTML report is generated in the playwright-report directory within your project root.

If any tests fail, Playwright is smart enough to automatically open this report in your default web browser, ensuring you don’t miss critical issues.

For successful runs or to re-examine a past report, you can simply run npx playwright show-report. This command serves up the latest report, making it incredibly convenient to revisit test results without manual file navigation. Progression testing

Anatomy of an HTML Report

The HTML report provides a high-level summary view that quickly tells you how many tests passed, failed, were skipped, or were flaky.

It also categorizes results by project e.g., Chromium, Firefox, WebKit, giving you a cross-browser perspective.

• Summary Dashboard: At a glance, you’ll see a pie chart or bar graph summarizing test outcomes. This initial visual cue is incredibly powerful for assessing the overall health of your test suite. Imagine seeing 98% pass rate versus 60% pass rate – the urgency of action changes dramatically.
• Test List: Each individual test is listed, clearly indicating its status pass/fail/skipped.
• Detailed Test View: Clicking on a specific test expands its details, which is where the real debugging magic happens.
  • Steps Breakdown: Playwright meticulously logs every action performed during the test, from navigating to a URL to clicking a button or asserting an element’s visibility. This step-by-step trace is invaluable for understanding the test’s execution flow.
  • Before/After Hooks: Information on beforeEach, afterEach, beforeAll, and afterAll hooks is included, which helps in debugging setup or teardown issues. If a test fails due to an environmental issue, checking the “Before Hooks” can provide clues.
  • Assertions and Errors: For failed tests, the report clearly highlights the failed assertion, providing the expected versus actual values and a full stack trace. This eliminates guesswork and directs you straight to the problematic line of code.
  • Attachments: This is arguably one of the most powerful features. Playwright can automatically capture:
    • Screenshots: A screenshot at the point of failure is captured by default. You can configure it to capture screenshots on only-on-failure, on, or off in your playwright.config.ts. For instance, setting screenshot: 'on' ensures a screenshot for every step, which is great for visual regression.
    • Videos: Playwright can record a video of the entire test execution, offering a dynamic replay of the user journey. This is particularly useful for identifying subtle UI glitches or race conditions that static screenshots might miss. Configure video: 'on' or video: 'retain-on-failure'.
    • Traces: Playwright Tracing is an advanced debugging tool that captures a wealth of information including browser DOM snapshots, network requests, action logs, and console output. It’s like a time machine for your test, allowing you to step through execution, inspect the DOM at any point, and analyze network traffic. This is a must for complex scenarios and is enabled via trace: 'on' or trace: 'retain-on-failure'.
  • Console Logs: Any console.log or console.error messages output during the test execution are captured and displayed within the report, providing an additional layer of diagnostic data.

Key takeaway: The HTML reporter is not just a report. it’s an interactive debugging environment that significantly reduces the time spent on problem identification. Data suggests that teams utilizing visual reports and comprehensive trace data can reduce debugging time by up to 50%, directly impacting release cycles.

Configuring Reporting for Different Environments

While the HTML reporter is excellent for local debugging, different environments local development, CI/CD pipelines, production monitoring demand different reporting strategies.

Playwright offers robust configuration options to tailor your reporting output. Assertion testing

Specifying Reporters via playwright.config.ts

The playwright.config.ts file is your central hub for configuring Playwright.

The reporter property within this file is where you define which reporters to use.

You can specify a single reporter or an array of reporters.

// playwright.config.ts


import { defineConfig, devices } from '@playwright/test'.

export default defineConfig{
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  // Configure one or more reporters
  reporter: 


   , // HTML reporter, don't open automatically
    , // Simple list reporter for console


   , // JSON reporter for machine parsing


   , // JUnit reporter for CI/CD
  ,
  use: {


   trace: 'on-first-retry', // Record trace only on retry


   video: 'on-first-retry', // Record video only on retry


   screenshot: 'only-on-failure', // Capture screenshot only on failure
  },
}.

This example demonstrates a common setup where multiple reporters are used simultaneously to cater to different needs:

• html: For local debugging with open: 'never' to prevent automatic opening on CI.
• list: Provides quick console feedback during local runs.
• json: For programmatic consumption of results.
• junit: For integration with CI systems.

Command Line Interface CLI Overrides

For quick, one-off runs or specific CI/CD stages, you can override the configured reporters directly from the command line using the --reporter flag. Test documentation

• npx playwright test --reporter=list: Only output a list of tests.
• npx playwright test --reporter=json: Generate only a JSON report.
• npx playwright test --reporter=html: Generate only an HTML report overriding others configured in playwright.config.ts.
• npx playwright test --reporter=dot,json: Specify multiple reporters separated by commas.

This flexibility allows you to adapt your reporting strategy on the fly, depending on the immediate requirements of your test execution environment.

Integrating Playwright Reports with CI/CD Pipelines

The real power of automated testing comes when it’s seamlessly integrated into your Continuous Integration/Continuous Delivery CI/CD pipeline.

Playwright’s diverse reporting options make this integration straightforward with popular CI/CD platforms like GitHub Actions, GitLab CI, Jenkins, Azure DevOps, and more.

JUnit Reporter for CI Compatibility

The junit reporter is the de facto standard for reporting test results in CI/CD environments. Most CI platforms have built-in capabilities to parse JUnit XML files and display the test results directly within their UI.

// playwright.config.ts for CI Assert in java

// …other configurations

reporter: ,
trace: ‘on-first-retry’,
video: ‘on-first-retry’,
screenshot: ‘only-on-failure’,

In your CI pipeline configuration e.g., .github/workflows/ci.yml for GitHub Actions, you would typically have a step to run Playwright tests and then a subsequent step to publish the JUnit report.

# .github/workflows/ci.yml
name: Playwright Tests

on: 

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-node@v3
      with:
        node-version: 18
    - name: Install dependencies
      run: npm ci
    - name: Install Playwright browsers
      run: npx playwright install --with-deps
    - name: Run Playwright tests


     run: npx playwright test --reporter=junit --output=test-results
    - name: Upload Playwright Test Report JUnit
      uses: actions/upload-artifact@v3
      if: always
        name: playwright-test-results
        path: test-results/junit.xml
    - name: Publish Test Results JUnit


     uses: EnricoMi/publish-unit-test-result-action@v2
        files: test-results/junit.xml
This setup ensures that:


1.  Tests are run, and a `junit.xml` file is generated.


2.  The `junit.xml` artifact is uploaded, making it accessible even after the job completes.


3.  The test results are parsed and displayed beautifully within the GitHub Actions run summary, providing immediate feedback on pull requests and deployments.

Statistics: A recent survey indicated that over 70% of teams using CI/CD integrate their test automation results directly into the pipeline's UI, often leveraging formats like JUnit for centralized reporting and faster failure detection.

 Archiving HTML Reports and Traces in CI


While JUnit is great for structured data, the rich HTML report and trace files are invaluable for debugging.

You can configure your CI pipeline to archive these assets as artifacts.

# .github/workflows/ci.yml continued


   - name: Run Playwright tests with tracing and video
      run: npx playwright test
      env:
       CI: 'true' # Set CI env variable for Playwright config
    - name: Upload Playwright HTML Report
        name: playwright-html-report
        path: playwright-report/
    - name: Upload Playwright Test Traces
        name: playwright-traces
       path: test-results/ # Assuming trace files are stored here
       retention-days: 7 # Keep traces for 7 days


By uploading `playwright-report/` and `test-results/` where traces and videos are usually stored by default, you ensure that developers can download these artifacts from the CI run and open them locally to perform in-depth debugging for any failures encountered in the pipeline.

This approach bridges the gap between concise CI summary reports and detailed local debugging tools.

# Advanced Reporting with Third-Party Integrations


While Playwright's built-in reporters are powerful, sometimes you need even more sophisticated reporting, historical trends, or a centralized dashboard for multiple test suites.

This is where third-party integrations come into play.

 Allure Report: A Comprehensive Solution
Allure Report is a popular open-source framework designed to create rich, interactive, and visually appealing test reports. It goes beyond simple pass/fail counts, offering features like:
*   Test trends and historical data: Track the health of your test suite over time.
*   Test categorization: Group tests by features, severity, or custom tags.
*   Detailed steps, attachments, and logs: Similar to Playwright's HTML report, but with more customization and aggregation capabilities.
*   Defect tracking integration: Link failures directly to issue trackers.



To use Allure with Playwright, you typically install the `allure-playwright` adapter:

```bash
npm install -D allure-playwright

Then configure it in your `playwright.config.ts`:

import { defineConfig } from '@playwright/test'.

    ,
    'allure-playwright', {


     outputFolder: 'allure-results', // Where Allure test data will be stored


     suiteTitle: 'My Application End-to-End Tests',
    },



After running your tests `npx playwright test`, you'll have an `allure-results` folder.

To generate and open the Allure report, you'll need the Allure command-line tool CLI, which can be installed via npm or your system's package manager.

npm install -g allure-commandline


allure generate allure-results --clean -o allure-report
allure open allure-report

Allure Report provides a significantly enhanced reporting experience, particularly beneficial for larger teams and complex projects requiring a comprehensive view of quality metrics. Its ability to show flakiness trends tests that pass sometimes and fail sometimes is a standout feature, helping teams prioritize and fix unstable tests which can consume up to 15-20% of a QA engineer's time if left unchecked.

 Custom Reporters for Tailored Needs
For organizations with very specific reporting requirements, Playwright offers the flexibility to develop custom reporters. This is an advanced use case where you implement a class that adheres to Playwright's `Reporter` interface.

// custom-reporter.ts


import type { Reporter, TestCase, TestResult, FullResult } from '@playwright/test'.

class MyCustomReporter implements Reporter {
  onBeginconfig: any, suite: any {


   console.log`Starting the run with ${suite.allTests.length} tests`.
  }

  onTestBegintest: TestCase {
    console.log`Test Started: ${test.title}`.

  onTestEndtest: TestCase, result: TestResult {


   console.log`Test Finished: ${test.title} - Status: ${result.status}`.
    if result.status === 'failed' {


     console.error`  Error: ${result.errors?.message}`.


     // Send a Slack notification, update a database, etc.
    }

  onEndresult: FullResult {
    console.log`Run Finished: ${result.status}`.
}

export default MyCustomReporter.

Then, configure it in `playwright.config.ts`:


  // ...
    ,


   , // Path to your custom reporter file
Custom reporters are incredibly powerful. They allow you to:
*   Integrate with internal dashboards: Push test results directly to your company's analytics or metrics platforms.
*   Trigger notifications: Send Slack, Teams, or email alerts for critical failures.
*   Perform custom data processing: Aggregate specific metrics, transform data, or even update bug tracking systems automatically.
*   Generate unique formats: If you need a report in a format not supported by existing reporters e.g., CSV, PDF with specific branding.



This level of customization ensures that your test reporting aligns perfectly with your organizational workflows and data consumption needs, ensuring that insights are not just generated, but acted upon effectively.

# Best Practices for Effective Playwright Reporting


To maximize the value you extract from Playwright test reports, consider adopting these best practices:

 1. Prioritize Readability and Clarity
*   Meaningful Test Names: Ensure your `test` function descriptions are clear, concise, and explain *what* is being tested. Instead of `test'my test', ...` use `test'should allow user to log in with valid credentials', ...`. Clear naming is the first step towards a readable report.
*   Organize Tests with `describe` Blocks: Group related tests using `test.describe`. This structures your reports logically, making it easier to navigate large test suites. For example, `test.describe'Authentication Flows',  => { ... }.`.

 2. Leverage Attachments Wisely
*   Screenshots on Failure: Configure `screenshot: 'only-on-failure'` in `playwright.config.ts`. This provides visual context for failures without creating excessive overhead for successful tests.
*   Videos on Retry/Failure: Use `video: 'on-first-retry'` or `video: 'retain-on-failure'`. Videos are incredibly helpful for debugging intermittent issues or complex interactions where a single screenshot might not tell the whole story.
*   Trace Files for Deep Dives: `trace: 'on-first-retry'` or `trace: 'retain-on-failure'` is recommended for CI, as it captures comprehensive debugging data. For local development, `trace: 'on'` might be useful temporarily. Remember that traces can be large, so manage retention.

 3. Optimize for CI/CD Feedback
*   Use JUnit Reporter: It's the standard for CI integration. Ensure your CI system is configured to parse and display JUnit XML files.
*   Archive Relevant Artifacts: Always upload the HTML report directory, trace files, and videos as CI artifacts. This allows developers to debug failures that occurred in CI without needing to re-run the tests locally. Consider retention policies for large artifacts to manage storage costs.
*   Fast Feedback Loop: Structure your tests and CI pipeline to provide the quickest possible feedback on critical paths. A 10-minute CI run is far more valuable than a 60-minute one when it comes to identifying regressions early.

 4. Monitor and Analyze Trends
*   Implement Allure or Custom Dashboards: For larger projects, static reports might not be enough. Tools like Allure provide historical data, flakiness trends, and execution time statistics, helping you identify areas for test optimization or application instability.
*   Track Flakiness: Acknowledge and actively address flaky tests. They erode trust in your test suite and waste valuable developer time. Use reporting tools that highlight flakiness to prioritize fixes. A test that passes 99 out of 100 times is still a problem. it needs to pass 100 out of 100 times.

 5. Clean Up and Manage Storage
*   Automate Report Deletion: Reports, especially with videos and traces, can consume significant disk space over time. Implement scripts or CI steps to clean up old `playwright-report` and `test-results` directories. For example, only retain reports for the last 'N' runs or for 'X' days.
*   Configure Trace/Video Retention: In `playwright.config.ts`, `trace: 'on-first-retry'` is a good default. For videos, `video: 'on-first-retry'` prevents accumulating videos for successful tests.



By adhering to these best practices, teams can transform their Playwright test reports from simple pass/fail indicators into powerful diagnostic tools that accelerate debugging, improve communication, and ultimately lead to higher quality software.

The investment in effective reporting pays dividends in reduced development cycles and increased confidence in your application's stability.

 Frequently Asked Questions

# What is a Playwright test report?


A Playwright test report is a comprehensive summary of a test run, providing details on passed, failed, skipped, and flaky tests, along with debugging information like screenshots, videos, and execution traces.

It's designed to help developers quickly understand test outcomes and diagnose issues.

# How do I open the Playwright HTML report?


After running your Playwright tests, if there are any failures, the HTML report will automatically open in your default browser.

If tests pass or you want to view a previous report, you can manually open it by running `npx playwright show-report` in your terminal.

# What information does the Playwright HTML report provide?


The HTML report provides a summary of test results pass/fail/skipped, detailed steps for each test, error messages and stack traces for failures, console logs, and attachments like screenshots, videos, and Playwright traces.

# Can Playwright generate reports in formats other than HTML?


Yes, Playwright supports various report formats including `list` console output, `dot` dots for passed, 'F' for failed, `json` machine-readable JSON file, and `junit` XML file compatible with CI/CD systems.

# How do I configure Playwright to use a specific reporter?


You configure reporters in your `playwright.config.ts` file using the `reporter` property.

You can specify a single reporter or an array of reporters, for example: `reporter: , `.

# How can I generate a JUnit report with Playwright?


To generate a JUnit XML report, set the `reporter` in `playwright.config.ts` to ``. Alternatively, run `npx playwright test --reporter=junit --output=test-results/`.

# Why is the JUnit report important for CI/CD?


The JUnit report is crucial for CI/CD because most continuous integration platforms like Jenkins, GitLab CI, GitHub Actions, Azure DevOps have built-in parsers to consume JUnit XML files, displaying test results directly within the pipeline's UI, making it easy to track build health.

# How do I capture screenshots on test failure in Playwright?


You can configure Playwright to capture screenshots on failure by setting `screenshot: 'only-on-failure'` in the `use` section of your `playwright.config.ts`.

# How can I record videos of my Playwright test runs?


To record videos, set `video: 'on'` to record all tests, or `video: 'on-first-retry'` or `video: 'retain-on-failure'` to save videos only for failed or retried tests, in the `use` section of your `playwright.config.ts`.

# What is Playwright Tracing and how do I enable it?


Playwright Tracing is a powerful debugging tool that captures a complete trace of your test execution, including DOM snapshots, network requests, action logs, and console output.

Enable it with `trace: 'on'` or `trace: 'retain-on-failure'` in `playwright.config.ts`.

# How do I view a Playwright trace file?


After a trace file `.zip` file is generated, you can view it by running `npx playwright show-trace path/to/your/trace.zip`. This will open an interactive viewer in your browser.

# Can I use multiple reporters at once in Playwright?


Yes, you can specify multiple reporters in an array within the `reporter` property in your `playwright.config.ts`. For example: `reporter: , `.

# How do I prevent the HTML report from opening automatically?


Set the `open` option of the HTML reporter to `'never'` in your `playwright.config.ts`: `reporter: `. This is useful for CI environments.

# What is Allure Report and how does it integrate with Playwright?


Allure Report is a third-party open-source framework for generating rich, interactive test reports with historical data, trends, and more.

You integrate it with Playwright using the `allure-playwright` adapter, which outputs data for the Allure CLI to process into a report.

# Can I create custom reporters in Playwright?


Yes, Playwright allows you to create custom reporters by implementing a class that adheres to Playwright's `Reporter` interface.

This provides full control over how test results are processed and reported, enabling integration with specific internal systems or custom formats.

# How do I troubleshoot issues in CI/CD using Playwright reports?


For CI/CD, use the JUnit report for quick status checks.

For deep debugging, ensure your CI pipeline archives the HTML report directory, trace files, and videos as artifacts.

You can then download these artifacts and open them locally to reproduce and diagnose the failure.

# What are flaky tests and how do Playwright reports help identify them?


Flaky tests are tests that sometimes pass and sometimes fail without any code changes.

While Playwright's default HTML report doesn't explicitly mark tests as "flaky," it supports retries which can highlight flakiness.

Advanced reporters like Allure provide dedicated features to track and report flakiness trends over time.

# How can I improve the readability of my Playwright reports?


Improve readability by using descriptive test names e.g., `test'should validate user login', ...`, grouping related tests with `test.describe`, and ensuring your console logs are clear and concise.

# Where are Playwright report files stored by default?


By default, the HTML report is stored in the `playwright-report/` directory in your project root.

Trace files and videos are typically stored in the `test-results/` directory, though this can be configured.

# Is it possible to clear previous Playwright reports before a new run?


Playwright automatically overwrites the default `playwright-report` directory with the latest run.

For other output directories like `test-results` for traces/videos, you might need to manually clear them or configure your CI pipeline to do so, though Playwright often manages this internally for most common cases.

Test cases for whatsapp