Uploaded image for project: 'Lucene.Net'
  1. Lucene.Net
  2. LUCENENET-627

Port RandomizedTesting Test Runner

    XMLWordPrintableJSON

Details

    Description

      While the test framework can run with the standard NUnit test runner and we have managed to make it work, it is missing the ability to reproduce a random test failure, which is crucial for debugging some of the more complicated random tests. Lucene uses a custom JUnit test runner called RandomizedTesting to accomplish this.

      It also includes some other nice features

      1. Ensure when a test fails, the random seeds are included in error messages and logs
      2. Code analysis to ensure the tests are setup properly
      3. Run tests in a random order to ensure they are not dependent upon each other

      Preliminary analysis shows that the API for NUnit allows building custom test runners and it is a close enough match to implement the functionality. Most likely, this will require a custom adapter as well, so the test runner can integrate into Visual Studio and dotnet test/mstest.

      Do note that without doing some pretty heavy refactoring on the Codec, Directory, and LuceneTestCase classes, it is not possible to run tests in parallel within the same AppDomain because the codecs use a static variable to turn codec impersonation on/off. For now, it would probably be best to run tests serially.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              nightowl888 Shad Storhaug
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 150h
                  150h
                  Remaining:
                  Remaining Estimate - 150h
                  150h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified