Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13741

TxnHandler.enqueueLockWithRetry() - optimize sql

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.0.0
    • None
    • Metastore, Transactions
    • None

    Description

      TxnHandler.enqueueLockWithRetry() does SQL insert into 2 tables using (possibly) multiple statements for each. Could easily generate 1 statement for each table.

      TxnHandler.addDynamicPartitions() - the insert stmt here should combing multiple rows into single SQL stmt (but with a limit for extreme cases)

      https://issues.apache.org/jira/browse/HIVE-13395?focusedCommentId=15271712&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15271712

      In TxnHandler.commitTxn, would it make sense to rearrange this so that the check is made whether there are any operations that could conflict before the mutex is obtained and the transaction id checked? If there's nothing to record in the write sets I don't see why you need to hold the mutex or even record a commit txn id.

      Note that Oracle doesn't support "insert into T values(1,2), (3,4)"

      Attachments

        Issue Links

          Activity

            People

              ekoifman Eugene Koifman
              ekoifman Eugene Koifman
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 3h
                  3h
                  Remaining:
                  Remaining Estimate - 3h
                  3h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified