Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-3365

Improve Sqoop tests to be able to execute them in a single JVM

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.4.7
    • None
    • None
    • None

    Description

      Gradle currently creates a new JVM for every test case it executes (the forkEvery parameter is set to 1, see: https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.Test.html#org.gradle.api.tasks.testing.Test:forkEvery). This provides better isolation for the tests but it has huge performance overhead as well since creating a JVM is an expensive operation. It would be great if we could execute all our tests in a single JVM so we could save that big cost but unfortunately it does not work out of the box because:

      • There are test classes which unnecessarily change the state of the JVM (for example by setting static fields and system properties) and can cause other tests to fail if they are run in the same JVM.
      • There are test classes which rely on a "clean JVM state" and they fail if the state is different than implicitly expected by them.
      • There are test classes which create a Kerberos KDC which sets a lot of static fields in the JVM which are really hard if not impossible to restore.

      This JIRA addresses the first two of the above issues.
      I expect the third category to be addressed by SQOOP-3104. It should introduce a category for kerberized tests and Gradle should execute this category with the forkEvery parameter set to 1.
      However Gradle should be able to execute the rest of the tests with forkEvery 0.

      Please note that the test execution order with Gradle is not strictly defined but platform dependent so it is possible that running the tests on a new platform will produce an execution order which will unveil a test interdependency which is not resolved by the subtasks of the JIRA.
      If you encounter such a situation please feel free to create a new subtasks here.

      Attachments

        Activity

          People

            Unassigned Unassigned
            vasas Vasas Szabolcs
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: