Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-11400

Kudu scan bottleneck due to sharing a single Kudu client for multiple tablet scans

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • Impala 4.1.0
    • None
    • Backend
    • Patch, Important
    • ghx-label-3

    Description

      This issue was observed when impala queries large datasets resides in Kudu. Even single ImpalaD is scanning multiple kudu tablets, it shows a slowness to retrive data eventhough ImpalaD makes parrellel scans. Reason for this is ImpalaD only uses a single Kudu client for multiple scans but KuduScanner::NextBatch runs on a single thread. So it's rpc reactor thread utilizes upto a single core and bottlenecks all parrelel scans. 

      This behaviour makes Impala clusters that scans kudu cannot be vertically scales to the maximum performance/cores of a node.

      Please refer the screenshots from Kudu slack channel for more information.

       

      Attachments

        1. 2-1.jpeg
          199 kB
          Sameera Wijerathne
        2. 4.JPG
          147 kB
          Sameera Wijerathne
        3. 3.JPG
          138 kB
          Sameera Wijerathne
        4. 5.JPG
          121 kB
          Sameera Wijerathne
        5. 2.JPG
          126 kB
          Sameera Wijerathne
        6. 1.JPG
          155 kB
          Sameera Wijerathne
        7. 0.JPG
          114 kB
          Sameera Wijerathne
        8. 2.jpeg
          199 kB
          Sameera Wijerathne
        9. WhatsApp Image 2022-06-07 at 10.39.27 PM.jpeg
          208 kB
          Sameera Wijerathne
        10. Kudu_1.png
          187 kB
          Sameera Wijerathne
        11. Impala_1.png
          180 kB
          Sameera Wijerathne
        12. Kudu_2.png
          188 kB
          Sameera Wijerathne
        13. Impala_2.png
          179 kB
          Sameera Wijerathne

        Activity

          People

            Unassigned Unassigned
            sameerawi Sameera Wijerathne
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: