Skip to main content

Debugging Failing Doc Generation

Sometimes your tests might be passing, but failing when generating documentation. Here are some tips to help debug what is happening.

Disabling exit on failure​

To prevent doc generation from becoming corrupted, the default behavior is to exit the process if a test fails.

To disable this behavior you can set the IGNORE_TEST_FAILURES environment variable by adding IGNORE_TEST_FAILURES=true to your script command.

package.json
  "scripts": {
    "build": "vite build",
    "dev": "vite dev",
    "test": "playwright test --ui",
    "doc": "TEST2DOC=true playwright test --config=playwright-test2doc.config.ts",
    "doc:debug": "IGNORE_TEST_FAILURES=true TEST2DOC=true playwright test --config=playwright-test2doc.config.ts"
  },

Run with Playwright UI​

Adding the --ui flag to the script or passing it in on the command line will start up the Playwright test suite with the UI. This will allow for more information and a visual inspection of what the problem might be.

Passing the flag from the command line​

pnpm doc:debug --ui

Adding the flag in the script commands​

package.json
  "scripts": {
    "build": "vite build",
    "dev": "vite dev",
    "test": "playwright test --ui",
    "doc": "TEST2DOC=true playwright test --config=playwright-test2doc.config.ts",
    "doc:debug": "IGNORE_TEST_FAILURES=true TEST2DOC=true playwright test --config=playwright-test2doc.config.ts --ui"
  },

Flaky tests​

If a test is flaky (passes when running the test normally but fails often during doc generation), the most pragmatic solution is unfortunately to skip the test for doc generation and create a manual doc to capture that functionality.

To skip a test you can use Playwright's tagging feature and filter out the test by tags. See the Filtering Tests guide for more details.

This should only be a temporary solution, as you should probably still investigate the root cause for the flakiness and resolve it.

Read More​

For more comprehensive debugging techniques, check out these official Playwright resources:

Interactive Debugging​

Visual Debugging​

Common Issues​

  • Test Timeouts - Configuring and understanding timeout errors
  • Test Isolation - Understanding test isolation with browser contexts
  • Locators - Best practices for reliable element selection

Advanced​