Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add additional mutation to > (resp. <) #501

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

nfrankel
Copy link

Most tests miss testing the boundary. Mutation testing can help a lot with this. The current code misses the change from > to >= and < and <=.

@nfrankel
Copy link
Author

The failing test checks for a typo on a source file I didn't touch.

@sourcefrog
Copy link
Owner

Hi, thanks for the patch.

I think this is worth considering, but we ought to check whether it generates too many low-value hits. Have you tried it on a large tree.

@nfrankel
Copy link
Author

I leave it up to you, but this mutation is part of established mutation testing tools.

https://pitest.org/quickstart/mutators/#CONDITIONALS_BOUNDARY

@sourcefrog
Copy link
Owner

I know, but it would also be useful to know that you tried it on some tree and it found a certain fraction of interesting or uninteresting misses.

@nfrankel
Copy link
Author

I didn't try it on a regular codebase. I just wanted to test this specific use-case, which is the one where Mutation Testing shines, and found it lacking.

@sourcefrog sourcefrog self-assigned this Feb 16, 2025
@sourcefrog
Copy link
Owner

Next steps:

  • Aside from the typo warning, which I fixed in main, there are some tests failing due to this generating new mutants that are not expected by the tests
  • After the tests pass, mutants test the mutants tree and see if anything is caught, and whether they seem like interesting findings
  • book/src/mutants.md should be updated to describe this
  • Add it to NEWS

@nfrankel nfrankel force-pushed the main branch 2 times, most recently from 18de762 to be96ac3 Compare February 17, 2025 10:28
@nfrankel
Copy link
Author

I did my best, but I'm stuck on one of the tests in error. I can't understand where I should set the expectation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants