Running tests from a custom compilation uses wrong Gradle task for tests

[kirillzh]

I have a custom integrationTest compilation in KMP project and a JVM target. I’ve got some tests in jvmIntegrationTest source set. When trying to run these tests from IDE using Kotest plugin, it runs cleanJvmTest and jvmTest tasks, I would expect it to run cleanJvmIntegrationTest and jvmIntegrationTest tasks.

Is it something that Kotest plugin need to somehow pick up on, or are there additional steps I need to take when configuring my compilation?

Here's how I configure integrationTest compilation for my JVM target:

jvm {
  val integrationTest by compilations.creating {
    val main by compilations.getting
    associateWith(main)

    defaultSourceSet {
      dependencies {
        // Compile against the main compilation's compile classpath and outputs:
        implementation(main.compileDependencyFiles + main.output.classesDirs)
        implementation(kotlin("test-junit"))
      }
    }

    // Define a Gradle task to run the integration tests
    project.tasks.register<Test>("jvmIntegrationTest") {
      description = "Run JVM integration tests"
      group = VERIFICATION_GROUP
      classpath = compileDependencyFiles + runtimeDependencyFiles + output.allOutputs
      testClassesDirs = output.classesDirs
    }
  }
}

[kirillzh]

This is where plugin looks for JVM test tasks:

kotest-intellij-plugin/src/main/kotlin/io/kotest/plugin/intellij/toolwindow/treeModel.kt

Line 59 in 8b7fdab

.filter { it.name.endsWith("jvmTest") || it.name.endsWith("test") }

What would be the most sensible approach here to make it pick up test tasks from other compilations?

[kirillzh]

As a workaround, created this task that automatically generates a run configuration for each integration test file:

/**
 * Generate runner configuration for JVM Integration tests manually, until Kotest plugin
 * supports running tests from custom test compilations: https://github.com/kotest/kotest-intellij-plugin/issues/266.
 */
private fun Project.generateJvmIntegrationTestRunConfiguration() {
  val runConfigurationsDir = rootDir.resolve(".idea/runConfigurations")
  runConfigurationsDir.mkdirs()

  val integrationTestDir = projectDir.resolve("src/jvmIntegrationTest/kotlin")

  if (integrationTestDir.exists()) {
    // Generate run configuration for each integration test file, if any:
    val testFiles = integrationTestDir.walk().filter {
      it.isFile && it.name.endsWith("Tests.kt")
    }
    testFiles.forEach { file ->
      println("Generating run configuration for $file")
      val configName = file.nameWithoutExtension
      val fullyQualifiedName =
        file.toRelativeString(integrationTestDir).replace("/", ".").removeSuffix(".kt")

      val runConfigurationXml = """
<component name="ProjectRunConfigurationManager">
  <configuration default="false" name="$configName" type="GradleRunConfiguration" factoryName="Gradle">
    <ExternalSystemSettings>
      <option name="executionName" />
      <option name="externalProjectPath" value="${"$"}PROJECT_DIR$" />
      <option name="externalSystemIdString" value="GRADLE" />
      <option name="scriptParameters" value="" />
      <option name="taskDescriptions">
        <list />
      </option>
      <option name="taskNames">
        <list>
          <option value="${project.path}:jvmIntegrationTest" />
          <option value="--tests" />
          <option value="&quot;$fullyQualifiedName&quot;" />
        </list>
      </option>
      <option name="vmOptions" />
    </ExternalSystemSettings>
    <ExternalSystemDebugServerProcess>false</ExternalSystemDebugServerProcess>
    <ExternalSystemReattachDebugProcess>true</ExternalSystemReattachDebugProcess>
    <DebugAllEnabled>false</DebugAllEnabled>
    <ForceTestExec>true</ForceTestExec>
    <method v="2" />
  </configuration>
</component>
""".trimIndent()

      val runConfigurationFile = runConfigurationsDir.resolve("$configName.xml")
      println("Generating into $runConfigurationFile")
      runConfigurationFile.writeText(runConfigurationXml)
    }
  }
}

[sksamuel]

This is where plugin looks for JVM test tasks:

kotest-intellij-plugin/src/main/kotlin/io/kotest/plugin/intellij/toolwindow/treeModel.kt

Line 59 in 8b7fdab

.filter { it.name.endsWith("jvmTest") || it.name.endsWith("test") }

What would be the most sensible approach here to make it pick up test tasks from other compilations?

We need a way to determine if a module is a test module or not, programatically, rather than using the string names.