[Bug]: mask for toHaveScreenshot masking also not visible element

[dimkin-eu]

Version

1.50.0

Steps to reproduce

test.describe("describe", () => {
    test("short visual spec", async ({page}) => {
        await page.goto("https://github.com/microsoft/playwright")
        await expect(page).toHaveScreenshot(`foobar.png`, {
            scale: "css",
            caret: "hide",
            mask: [page.locator(".HeaderMenu-dropdown-link"), page.locator("[aria-label=Homepage]")]
        })
    })
})

Expected behavior

hidden elements shouldn't be masked

Actual behavior

also hidden/not visible elements are being masked

Additional context

hidden but masked elements add unneeded noise to the image and/or make it unusable in image comparison
as an extra example - changing mask to mask: [page.locator("[aria-hidden=true]")] makes this

Environment

System:
    OS: macOS 15.3
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Memory: 95.25 MB / 32.00 GB
  Binaries:
    Node: 22.12.0 - /usr/local/bin/node
    npm: 10.9.0 - /usr/local/bin/npm
    pnpm: 10.3.0 - /usr/local/bin/pnpm
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: 1.50.0 => 1.50.0

[Skn0tt]

I agree this is surprising. I'll discuss with the team whether this is on purpose, but most likely we can't change this without a breaking change. To unblock yourself, try masking page.locator(".HeaderMenu-dropdown-link").locator("visible=true") (see https://playwright.dev/docs/locators#matching-only-visible-elements).

[dimkin-eu]

wow, this is a nice workaround
but not sure in which way should this be applied to mask Array<Locator> in a nice way :) ( luckily we need only 1 element s far :) )

[Skn0tt]

You could .map(l => l.locator("visible=true")) on the array :)

[dimkin-eu]

there is one more edge case - when a masked element is covered with a semi-transparent overlay

I hope such a small cut will make sense (a semi-transparent menu is over a coverable component )

should be

as is

[Skn0tt]

it's impossible for us to identify if a semi-transparent overlay is visible or not. your best bet is to hide those semi-transparent elements, if you absolutely need to screenshot an element behind it.

[Skn0tt]

We discussed this issue yesterday, and concluded that we won't make any breaking changes to this API without a strong reasoning. One thing that came up though is that you might be better off using stylePath instead of mask: https://playwright.dev/docs/api/class-pageassertions#page-assertions-to-have-screenshot-1-option-style-path

That would allow you to override styles for the elements you don't want to see, and for example set display: hidden.

I'm assuming that your microsoft/playwright screenshot is purely for demo purposes and your real usecase is a different site. We've been thinking about injected a classname into the site DOM on screenshot, that you can use to hide elements - similar to https://argos-ci.com/docs/injecting-css#add-css-in-your-code. It's a feature we've been thinking about, would it help your usecase?

[dimkin-eu]

thank you for your efforts

we won't make any breaking changes to this API without a strong reasoning.

I think it would be beneficial adding info about .locator("visible=true") to docs

I'm assuming that your microsoft/playwright screenshot is purely for demo purposes and your real usecase is a different site.

yes. our use case - many (2K+) storybook tests.
2 things we faced

• masking purely invisible elements - solvable
• masking elements covered with semi-transparent elements like menus will pop mask over that element - we solved this with our code, which masks elements by ourselves, taking into account z-indexes - quite specific, and looks like this will not be a native playwright solution.

With masking, we wanted to remove core/generic elements from non-related tests ( like 1 core element could exist on the majority of screenshots ) to not update many unrelated pages - faster, and less noise for related teams. So we mask elements conditionally before toHaveScreenshot

will check stylePath