The challenge

A legal technology platform relies on complex browser workflows, asynchronous updates and shared test data. The challenge was to make test results meaningful: a green test should provide evidence that the intended behaviour worked.

Where AI fits into the workflow

AI assistants support test development, refactoring, review and investigation of CI failures. Repository-specific instructions capture conventions and UI pitfalls, with separate workflows for test changes, code review and analysis of test run results. Engineers provide context, examine results and retain responsibility for decisions.

Diagnosing unreliable and false-positive tests

One investigation uncovered a test that compared fallback values after a timer could not be read, producing a misleading pass. The correction made the helper fail when a real value was unavailable. Another problem involved an asynchronous form update: tracking the network request provided a better signal than guessing from the visible form state.

Architecture and validation

Playwright and TypeScript tests use fixtures, page objects and reusable helpers. The team examines traces and application behaviour, checks the resulting application state and validates fixes for flaky tests through repeated runs. Tests run in Azure DevOps pipelines.

Outcome

In these cases, the test now fails when it cannot read the actual timer, and the form diagnostics distinguish an unsent request from a pending response. These changes show how to check application behaviour and expose misleading test results.

Discuss a similar challenge

[email protected]

← Back to selected work