Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Large test suites sometimes provide limited benefit to incremental development #40546

Open
mbrodesser-Igalia opened this issue Jun 14, 2023 · 0 comments
Labels

Comments

@mbrodesser-Igalia
Copy link
Contributor

If a test suite contains many (e.g. 100) tests and the first test fails in a way that the following tests aren't executed, incremental development can't make use them.

A recent example of this is https://searchfox.org/mozilla-central/source/testing/web-platform/tests/html/semantics/popovers/popover-light-dismiss.html.

One mitigation for this would be to split such files into multiple files, containing, ideally, just one test per file. If this is feasible, it should be explicitly mentioned at https://web-platform-tests.org/test-suite-design.html and reviewers should pay attention to it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants