Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add contributing task #605

Open
wants to merge 43 commits into
base: master
Choose a base branch
from
Open

Conversation

dnestoro
Copy link
Member

@dnestoro dnestoro commented Mar 6, 2025

What does this PR do?

Adds interactive task that helps users to contribute to the metadata repository. The user only needs to provide path to tests and answer few questions in order to make a fully usable pull request.

For reviewers:

@TaskAction
void run() throws IOException {
InteractiveTaskUtils.printUserInfo("Hello! This task will help you contributing to metadata repository." +
" Please answer the following contributingQuestions. In case you don't know the answer on the question, type \"help\" for more information");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please answer the following contributingQuestions

I would reword that to: "Please answer the following questions".

@mhalbritter
Copy link
Collaborator

Okay, after cleaning up some empty directories it now continues, but then crashes with:

[INFO] Generating stubs for: de.mkammerer:argon2-jvm:2.12...
[INFO] Updating allowed packages in: metadata/index.json...
[INFO] Removing dummy test stubs...
[INFO] Copying tests from: /home/mhalbritter/tmp/argon2-jvm/argon2-jvm/src/test/java to /home/mhalbritter/Projects/graalvm-reachability-metadata/tests/src/de.mkammerer/argon2-jvm/2.12/src/test/java...
[INFO] Generating user-code-filter.json...
[INFO] Adding following dependencies to build.gradle file: []...
[INFO] Configuring agent block in: build.gradle...
[INFO] Generating metadata...[51s]
jdk.graal.compiler.util.json.JsonParserException: line 721 column 79 Expected , or } but found :
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.error(JsonParser.java:535)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.expectedError(JsonParser.java:555)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:293)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parse(JsonParser.java:127)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ConfigurationParser.parseAndRegister(ConfigurationParser.java:90)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.command.ConfigurationGenerateConditionalsCommand.apply(ConfigurationGenerateConditionalsCommand.java:119)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.ConfigurationTool.main(ConfigurationTool.java:84)
        at org.graalvm.nativeimage.enterprise.configure/com.oracle.svm.enterprise.configure.ConfigurationEnterpriseTool.main(ConfigurationEnterpriseTool.java:24)

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> Process 'command '/home/mhalbritter/.sdkman/candidates/java/current/bin/native-image-configure'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 6s

> Task :contribute FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':contribute'.
> Process 'command 'gradle'' finished with non-zero exit value 1

I'm using

> java -version
java version "23.0.2" 2025-01-21
Java(TM) SE Runtime Environment Oracle GraalVM 23.0.2+7.1 (build 23.0.2+7-jvmci-b01)
Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 23.0.2+7.1 (build 23.0.2+7-jvmci-b01, mixed mode, sharing)

@mhalbritter
Copy link
Collaborator

Okay, i retried with a GraalVM 17, and now it looks better.

One thing to note: I first added metadata for 2.12, and then for 2.11. Now the metadata index.json looks like this:

[ {
  "metadata-version" : "2.12",
  "tested-versions" : [ "2.12" ],
  "module" : "de.mkammerer:argon2-jvm"
}, {
  "metadata-version" : "2.11",
  "tested-versions" : [ "2.11" ],
  "latest" : true,
  "module" : "de.mkammerer:argon2-jvm"
} ]

I don't think the latest: true should be on the 2.11 version.

@mhalbritter
Copy link
Collaborator

I also noticed in the metadata/index.json file:

{
  "allowed-packages" : [ "de.mkammerer.argon2", "de.mkammerer" ],
  "directory" : "de.mkammerer/argon2-jvm",
  "module" : "de.mkammerer:argon2-jvm"
}

I get where the de.mkammerer.argon2 package is coming from, but where does de.mkammerer comes from?

@mhalbritter
Copy link
Collaborator

Now I've tried to add tests which not only need JUnit, but also some more dependencies (in this case org.awaitility:awaitility:4.3.0).

In the first run, I've omitted Awaitility just to see what happens, and it breaks (which I guess is expected):

[INFO] Generating metadata...[1m 14s]
/home/mhalbritter/Projects/graalvm-reachability-metadata/tests/src/de.mkammerer/argon2-jvm/2.12/src/test/java/de/mkammerer/argon2/SomeAwailitilityTest.java:5: error: package org.awaitility does not exist
import org.awaitility.Awaitility;
                     ^
/home/mhalbritter/Projects/graalvm-reachability-metadata/tests/src/de.mkammerer/argon2-jvm/2.12/src/test/java/de/mkammerer/argon2/SomeAwailitilityTest.java:14: error: cannot find symbol
        Awaitility.await().until(() -> true);
        ^
  symbol:   variable Awaitility
  location: class SomeAwailitilityTest
2 errors

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileTestJava'.
> Compilation failed; see the compiler output below.
  /home/mhalbritter/Projects/graalvm-reachability-metadata/tests/src/de.mkammerer/argon2-jvm/2.12/src/test/java/de/mkammerer/argon2/SomeAwailitilityTest.java:5: error: package org.awaitility does not exist
  import org.awaitility.Awaitility;
                       ^
  /home/mhalbritter/Projects/graalvm-reachability-metadata/tests/src/de.mkammerer/argon2-jvm/2.12/src/test/java/de/mkammerer/argon2/SomeAwailitilityTest.java:14: error: cannot find symbol
          Awaitility.await().until(() -> true);
          ^
    symbol:   variable Awaitility
    location: class SomeAwailitilityTest
  2 errors

* Try:
> Check your code and dependencies to fix the compilation error(s)
> Run with --scan to get full insights.

BUILD FAILED in 607ms

> Task :contribute FAILED

Then i reran ./gradlew contribute again, this time including the Awaitility dependency. After answering all the questions again (and specifying Awaitility this time), it generated this metadata index.json:

[ {
  "metadata-version" : "2.12",
  "tested-versions" : [ "2.12" ],
  "module" : "de.mkammerer:argon2-jvm"
}, {
  "metadata-version" : "2.12",
  "tested-versions" : [ "2.12" ],
  "latest" : true,
  "module" : "de.mkammerer:argon2-jvm"
} ]

The first 2.12 is from the first failed run, the second one is from the second run. I think there needs to be more cleverness by omitting versions which are already in there.

private void createStubs(boolean shouldUpdate){
InteractiveTaskUtils.printUserInfo("Generating stubs for: " + coordinates );
if (shouldUpdate) {
invokeCommand("gradle scaffold --coordinates " + coordinates + " --update", "Cannot generate stubs for: " + coordinates);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd use the gradle wrapper here, as using the global gradle can lead to all sort of problems if the versions don't match.

Path allTests = originalTestsLocation.resolve(".");

InteractiveTaskUtils.printUserInfo("Removing dummy test stubs");
invokeCommand("rm -r " + destination, "Cannot delete files from: " + destination);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This won't work under Windows. And please make sure that the destination are really the tests (maybe by double-checking that the destination is under the project dir), as this could be dangerous.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added additional check that the absolute path (of the file we want to delete) starts with the absolute path of the project (see this). The check is now used in the following places: here and here.

Besides that, the task now also asks user for explicit confirmation that it can delete certain file. Check this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced with:

ensureFileBelongsToProject(destination);
boolean shouldDelete = InteractiveTaskUtils.askForDeletePermission(destination);
if (!shouldDelete) {
        throw new RuntimeException("The task didn't get permission to delete dummy stubs. Cannot proceed with the task execution");
}

InteractiveTaskUtils.printUserInfo("Removing dummy test stubs");
getFileSystemOperations().delete(deleteSpec -> deleteSpec.delete(destination));

Which first checks that the destination path begins with the project path and then asks user for explicit confirmation that a file on the given location can be removed (@mhalbritter do you think that it makes sense, or you have a suggestion for some additional check?)

For Windows support, I am now using built in gradle FileSystemOperations. cc @melix

invokeCommand("rm -r " + destination, "Cannot delete files from: " + destination);

InteractiveTaskUtils.printUserInfo("Copying tests from: " + originalTestsLocation + " to " + destination);
invokeCommand("cp -a " + allTests + " " + destination, "Cannot copy files to: " + destination);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't work under windows.

Copy link
Member Author

@dnestoro dnestoro Mar 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced with Gradle's FileSystemOperations:

getFileSystemOperations().copy(copySpec -> {
        copySpec.from(allTests);
        copySpec.into(destination);
})

Path destination = testsDirectory.resolve("src").resolve("test");
InteractiveTaskUtils.printUserInfo("Copying resources from: " + originalResourcesDirectory + " to " + destination);

invokeCommand("cp -r " + originalResourcesDirectory + " " + destination, "Cannot copy files to: " + destination);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't work under Windows.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replaced with Gradle's FileSystemOperations:

getFileSystemOperations().copy(copySpec -> {
      copySpec.from(originalResourcesDirectory);
      copySpec.into(destination);
});

private void addUserCodeFilterFile(List<String> packages) throws IOException {
InteractiveTaskUtils.printUserInfo("Generating " + USER_CODE_FILTER_FILE);

ConfigurationStringBuilder sb = new ConfigurationStringBuilder();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't be an object which is then written to JSON via the ObjectMapper better here?

invokeCommand("git", List.of("commit", "-m", "Add metadata for " + coordinates), "Cannot commit changes", null);

InteractiveTaskUtils.printUserInfo("Pushing changes");
invokeCommand("git push origin " + branch, "Cannot push to origin");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes you are working on a fork, right? Because that would fail if a user just cloned the reachability repo and doesn't have write privileges to the repo.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I think that you don't need any privileges to push branch directly to metadata repo. We have many branches open directly on the repo.

public class FilesUtils {

public static void findJavaFiles(Path root, List<Path> result) {
if (Files.exists(root) && Files.isRegularFile(root) && root.toString().endsWith(".java")) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could also be implemented with java.nio.file.Files#walkFileTree().

},
{
"question-key": "testsLocation",
"question": "Where are your tests implemented? Absolute path to the directory containing java packages where you implemented tests: ",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd reword that. It wasn't clear to me that those are the existing tests for the library I want to add.

{
"question-key": "additionalDependencies",
"question": "What additional testImplementation dependencies you want to include? Enter the next dependency (to stop type \"-\")",
"help": "Enter the testImplementation dependencies (pres enter after each dependency) you want to include use in tests. Provide dependencies in form of Maven coordinates. Maven coordinates consist of three parts in the following format \"groupId:artifactId:version\". For more information visit: https://maven.apache.org/repositories/artifacts.html When you finish adding dependencies, type \"-\" to terminate the inclusion process"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: "pres" instead of "press".

{
"question-key": "shouldCreatePullRequest",
"question": "Do you want to create a pull request to the reachability metadata repository [Y/n]:",
"help": "In case you want to open a pull request to Reachability metadata repository automatically, type \"y\". This way the task will create a new branch for you, commit changes with a proper message, and push the branch remotly. At the end, the task will give you a link to the Github repository where you should fill in a pull request checkslist and complete the process."
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: "remotly".

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: "checkslist".

@dnestoro
Copy link
Member Author

Okay, after cleaning up some empty directories it now continues, but then crashes with:

[INFO] Generating stubs for: de.mkammerer:argon2-jvm:2.12...
[INFO] Updating allowed packages in: metadata/index.json...
[INFO] Removing dummy test stubs...
[INFO] Copying tests from: /home/mhalbritter/tmp/argon2-jvm/argon2-jvm/src/test/java to /home/mhalbritter/Projects/graalvm-reachability-metadata/tests/src/de.mkammerer/argon2-jvm/2.12/src/test/java...
[INFO] Generating user-code-filter.json...
[INFO] Adding following dependencies to build.gradle file: []...
[INFO] Configuring agent block in: build.gradle...
[INFO] Generating metadata...[51s]
jdk.graal.compiler.util.json.JsonParserException: line 721 column 79 Expected , or } but found :
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.error(JsonParser.java:535)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.expectedError(JsonParser.java:555)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:293)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseArray(JsonParser.java:337)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:235)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseObject(JsonParser.java:274)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parseLiteral(JsonParser.java:233)
        at jdk.graal.compiler/jdk.graal.compiler.util.json.JsonParser.parse(JsonParser.java:127)
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.configure.ConfigurationParser.parseAndRegister(ConfigurationParser.java:90)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.command.ConfigurationGenerateConditionalsCommand.apply(ConfigurationGenerateConditionalsCommand.java:119)
        at org.graalvm.nativeimage.configure/com.oracle.svm.configure.ConfigurationTool.main(ConfigurationTool.java:84)
        at org.graalvm.nativeimage.enterprise.configure/com.oracle.svm.enterprise.configure.ConfigurationEnterpriseTool.main(ConfigurationEnterpriseTool.java:24)

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> Process 'command '/home/mhalbritter/.sdkman/candidates/java/current/bin/native-image-configure'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 6s

> Task :contribute FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':contribute'.
> Process 'command 'gradle'' finished with non-zero exit value 1

I'm using

> java -version
java version "23.0.2" 2025-01-21
Java(TM) SE Runtime Environment Oracle GraalVM 23.0.2+7.1 (build 23.0.2+7-jvmci-b01)
Java HotSpot(TM) 64-Bit Server VM Oracle GraalVM 23.0.2+7.1 (build 23.0.2+7-jvmci-b01, mixed mode, sharing)

this issue is already reported here

@dnestoro
Copy link
Member Author

I also noticed in the metadata/index.json file:

{
  "allowed-packages" : [ "de.mkammerer.argon2", "de.mkammerer" ],
  "directory" : "de.mkammerer/argon2-jvm",
  "module" : "de.mkammerer:argon2-jvm"
}

I get where the de.mkammerer.argon2 package is coming from, but where does de.mkammerer comes from?

It came from the scaffold task which adds default allowed package (which was in your casede.mkammerer).
I fixed that now (my task removes that default package first)

@dnestoro
Copy link
Member Author

The first 2.12 is from the first failed run, the second one is from the second run. I think there needs to be more cleverness by omitting versions which are already in there.

@mhalbritter I am now throwing an exception in case you want to provide support for the library + same version of it, if it already exists. See this

Copy link
Member

@melix melix left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't take a look yet, just something that was obvious in the first place: you are invoking CLI in many places, but you shouldn't do this. You are running as a Gradle plugin, so you should use the Gradle constructs which are cross platform, e.g FileSystemOperations.

@dnestoro
Copy link
Member Author

dnestoro commented Mar 24, 2025

@melix @mhalbritter thanks for the feedback. I will refactor the PR to avoid using CLI, and ping you again when the PR is ready for review again. Until then, I will move this PR into a draft state, so please ignore incoming changes and don't test it until I finish refactoring

@dnestoro dnestoro marked this pull request as draft March 24, 2025 16:45
@dnestoro dnestoro marked this pull request as ready for review March 27, 2025 12:58
@dnestoro dnestoro requested review from melix and mhalbritter March 27, 2025 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants