Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli): execute one test from a test file #338

Merged
merged 24 commits into from
Feb 28, 2025

Conversation

khalatevarun
Copy link
Contributor

@khalatevarun khalatevarun commented Feb 12, 2025

fix: #281

Affected regions:

  • runTests()
  • executeTestFile()
  • main()
  • package.json - new packages added to parse the file - acorn and acorn-walk

Copy link

vercel bot commented Feb 12, 2025

@khalatevarun is attempting to deploy a commit to the Antiwork Team on Vercel.

A member of the Team first needs to authorize it.

@khalatevarun khalatevarun marked this pull request as draft February 12, 2025 02:10
@khalatevarun khalatevarun changed the title feat: add ability to run test in a file with a line no feat: execute one test from a test file Feb 12, 2025
@khalatevarun khalatevarun changed the title feat: execute one test from a test file feat(cli): execute one test from a test file Feb 12, 2025
@khalatevarun khalatevarun marked this pull request as ready for review February 13, 2025 05:30
Copy link
Member

@rmarescu rmarescu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll be able to take a closer look on this during the next milestone (next week or so). Generally, I think it should use a parser like @babel/parser as it is more reliable than regex.

Here is a sample AI implementation, for reference/inspiration (haven't looked into details nor tested the code)

@khalatevarun
Copy link
Contributor Author

@rmarescu I request you to please take a look at this PR as its ready and synced with latest development
@gladyshcodes would like to know your thoughts on this too

Copy link
Member

@rmarescu rmarescu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears that Babel would be a better option for us.

  • TypeScript/JSX Support: Since Shortest is written in TypeScript, Babel's native TypeScript parsing will more accurately handle test files.
  • Parent-Child Relationship Tracking: The parentPath feature in Babel's traversal is ideal for following method chains - essential for our use case where shortest() calls have chained expect() methods.
  • Scope Awareness: Babel can identify related test assertions even when they span multiple lines or appear in different expressions but the same scope.
  • Visitor Pattern: Babel's visitor pattern is more efficient than current nested AST walks.
  • Type Safety: Better TypeScript types for AST nodes will make your code more maintainable.

Although Acorn generally has better performance metrics:

  • The performance difference is minimal for our specific use case
  • We're parsing individual test files, not entire codebases
  • The improved accuracy in test detection far outweighs any slight performance difference

For example, the current logic doesn't parse the updated api-assert-bearer.test.ts correctly:

Test Discovery: No tests found at line 7 in examples/api-assert-bearer.test.ts

CleanShot 2025-02-27 at 10 37 24@2x


Will work on replacing with Babel in this PR, and will add TestCase in a separate one.


if (lineNumber) {
testsToRun = this.filterTestsByLineNumber(
registry.currentFileTests,
Copy link
Member

@rmarescu rmarescu Feb 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think each TestFunction should have the info about startLineNumber and endLineNumber so that, together with the filePath, should generate a more unique hash for the test.
Right now, filePath on TestFunction is not set, which means the hash is generated mostly from the name

const test: TestFunction = {
name,
filePath: "",
expectations: [],
};

That can cause conflicts when 2 tests have the same name. The combo of name + filePath + startLineNumber + endLineNumber should create a unique hash.

With that in place, the logic here would only need to filter tests that fit the criteria.

I'll implement in a follow-up PR.

@rmarescu rmarescu self-assigned this Feb 27, 2025
@@ -281,16 +285,78 @@ export class TestRunner {
};
}

private async executeTestFile(file: string) {
private async filterTestsByLineNumber(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that straight-forward to add test coverate for this at this moment.

Comment on lines +303 to +317
if (!testLocation) {
testLocation = testLocations.find((location) => {
const TEMP_TOKEN = "##PLACEHOLDER##";
let pattern = location.testName.replace(
new RegExp(escapeRegex(EXPRESSION_PLACEHOLDER), "g"),
TEMP_TOKEN,
);

pattern = escapeRegex(pattern);
pattern = pattern.replace(new RegExp(TEMP_TOKEN, "g"), ".*");
const regex = new RegExp(`^${pattern}$`);

return regex.test(testNameNormalized);
});
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parseShortestTestFile is used on TS test files, while the name stored on TestFunction object is generated from the compiled file. Because of that, the names may not match.

E.g. in TS file:

Test the API POST endpoint ${TESTING_API_BASE_URI}/assert-bearer with body { "flagged": "true" } and the bearer token ${ALLOWED_TEST_BEARER}

Compiled file (expressions evaluated):

Test the API POST endpoint api/assert-bearer with body { "flagged": "true" } and the bearer token Bearer 123

@rmarescu rmarescu added this to the v0.4.5 milestone Feb 28, 2025
@rmarescu rmarescu merged commit 6218f66 into anti-work:main Feb 28, 2025
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Execute one test from a test file
2 participants