Uploaded image for project: 'Lucene.Net'
  1. Lucene.Net
  2. LUCENENET-640

Sequential IndexWriter performance in concurrent environments.

    XMLWordPrintableJSON

Details

    • Important

    Description

      When creating Lucene.Net indices in parallel, sequential-like performance is experienced. Profiling 8 concurrent IndexWriter instances writing in parallel shows that WeakIdentityMap::IdentityWeakReference::Equals spends most time garbage collecting (94.91%) and TokenStream::AssertFinal (87.09% garbage collecting) in my preliminary tests (see screenshots).

      The WeakIdentityMap implementation uses an IdentityWeakReference as key, which is implemented as a class. By inspection of this class, it is merely a System.Runtime.InteropServices.GCHandle wrapper as can be seen in the mono project, manually wrapping of this struct in a struct rather than a class - will eliminate some of the immense amounts of garbage collection.

      Attachments

        1. Program.cs
          2 kB
          Mathias Henriksen
        2. overviewBug.jpg
          107 kB
          Mathias Henriksen
        3. IdentityWeakReferenceBug.jpg
          18 kB
          Mathias Henriksen
        4. AssertFinalBug.jpg
          29 kB
          Mathias Henriksen

        Issue Links

          Is contained by

          Improvement - An improvement or enhancement to an existing feature or task. LUCENENET-630 Identify/Fix Bottlenecks

          • Major - Major loss of function.
          • Closed
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. LUCENENET-616 Make Collections from Lucene.Net.Support into a 1st Class Feature

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. LUCENENET-610 Reduce locking in FieldCacheImpl::Cache::Get

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Task - A task that needs to be done. LUCENENET-636 Port ConditionalWeakTable from .NET Core 3.x to .NET Standard 2.0

          • Major - Major loss of function.
          • Closed
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. LUCENENET-644 The design and implementation of a better ThreadLocal<T>

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Task - A task that needs to be done. LUCENENET-642 Eliminate Runtime Reflection Code from TokenStream

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Web Link GitHub Pull Request #236

          Activity

            People

              Unassigned Unassigned
              matt2843 Mathias Henriksen
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m