An export test needs to verify the file's contents and intended meaning, not just that a download completed. The dashboard, underlying data and exported report can disagree even when the export button works.
Start with an export contract: which records, filters, labels, permissions and presentation rules must survive in the file? This defines the expected result and distinguishes defects from intentional differences.
Start with a representative report
Consider this illustrative scenario: a customer filters a usage dashboard by region and reporting period, then downloads a report to share with colleagues. The page shows the selected region correctly, but the export uses a cached filter and includes records from a different region.
The download check passes and the dashboard looks right. The shared artifact is still wrong.
Create a small synthetic dataset whose records are easy to distinguish. Include an excluded region, an empty category, a long label and a value that requires the product's documented rounding rule. Define the expected result independently of the export implementation.
Write the export acceptance contract
Record the business rules before choosing parsers or screenshot tools:
- Which filters, time boundaries and permissions apply?
- Which records and fields should appear or remain excluded?
- Which units, labels and rounding rules are required?
- Which totals must reconcile, and what intentional differences are allowed?
- Which titles, footers, pages or sheets must be present?
- Which file formats and supported viewers are in scope?
Avoid requiring raw equality where the product deliberately transforms data. A rounded chart label and a full-precision spreadsheet value may both be correct. The rule describing that transformation must be explicit.
Verify the artifact before inspecting its contents
Tie the downloaded file to the action that produced it. Capture the candidate build, fixture version, selected filters, user role and export request or job identifier where available.
In Playwright, download handling can wait for the download event and persist the completed artifact. Save the evidence before closing its browser context, because the temporary download belongs to that context. Playwright download handling.
Then establish that the file can be opened by the expected parser or viewer. An extension alone does not establish that a response is the intended report. An error document downloaded under a report filename must not pass.
Check data and presentation independently
For structured output, inspect fields using a parser appropriate to the format. Check record identity, inclusion and exclusion, totals, types and the expected transformations. A successful parse does not establish business correctness.
Compare the UI, API response and export against the agreed fixture and calculation rules. Agreement between outputs is not sufficient if they all repeat the same incorrect backend value.
For presentation, render the report in the supported environment and inspect clipping, missing labels, page breaks, unreadable contrast and misplaced content. Keep the rendering environment stable when using image comparisons. Playwright documents how environment differences can affect browser screenshot baselines; exported documents likewise need a defined renderer and comparison setup. Playwright visual comparison guidance.
Keep content assertions separate from image differences. A document may look nearly identical while containing the wrong reporting period. Conversely, a benign rendering difference may require review without indicating incorrect data.
Include a deliberate failure
Prove that the checks can detect the mistake you care about. In an isolated fixture, make the export ignore the selected region or omit a required heading. The corresponding check should fail with evidence that explains the mismatch.
Use a role-restricted fixture as well. An export must not include records the test user is not permitted to view. Keep this exercise inside an authorized test environment with synthetic data.
Do not automatically replace an expected file because the new file differs. A baseline update is a change to the acceptance record and needs a reason and review.
Use explicit release verdicts
- Pass: the required artifact checks completed and matched the approved contract for the tested candidate.
- Fail: the artifact contradicted a required data, access or presentation rule.
- Unknown: the artifact could not be obtained, opened or evaluated, or the expected result was not defined.
Review failures by impact. A missing permitted row, leaked restricted row and small layout shift are different problems. Do not hide a critical data or permission defect inside an average visual score.
When evaluating an automation approach, have your own team maintain a fixture and investigate a failed export. Record setup effort, diagnosis effort and remaining manual review. The desired outcome is dependable report evidence your team can operate independently.
Frequently asked questions
Is checking the downloaded filename enough?
No. Check that the artifact opens and that its contents satisfy the export contract. Filename and download completion checks cover only the delivery step.
Should every file use visual comparison?
Choose checks according to the contract. Structured data needs semantic checks; presentation-sensitive documents may also need rendered inspection. Neither method replaces the other.
How should changing data be handled?
Use controlled fixtures for regression checks and define permitted variability explicitly. Preserve enough context to explain why a changed value was expected.
Bring a representative export and its expected behavior to a ContextQA workflow review to scope an evaluation around the artifact your customers actually use.
