Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8829

Upgrade to Thrift 0.9.2

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.15.0
    • 1.1.0
    • None

    Description

      Apache Thrift 0.9.2 was released recently (https://thrift.apache.org/download). It has a fix for THRIFT-2660 which can cause HS2 (tcp mode) and Metastore processes to go OOM on getting a non-thrift request when they use SASL transport. The reason (thrift code):

        protected SaslResponse receiveSaslMessage() throws TTransportException {
    underlyingTransport.readAll(messageHeader, 0, messageHeader.length);

    byte statusByte = messageHeader[0];
    byte[] payload = new byte[EncodingUtils.decodeBigEndian(messageHeader, STATUS_BYTES)];
    underlyingTransport.readAll(payload, 0, payload.length);

    NegotiationStatus status = NegotiationStatus.byValue(statusByte);
    if (status == null) {
      sendAndThrowMessage(NegotiationStatus.ERROR, "Invalid status " + statusByte);
    } else if (status == NegotiationStatus.BAD || status == NegotiationStatus.ERROR) {
      try {
        String remoteMessage = new String(payload, "UTF-8");
        throw new TTransportException("Peer indicated failure: " + remoteMessage);
      } catch (UnsupportedEncodingException e) {
        throw new TTransportException(e);
      }
    }

      Basically since there are no message format checks / size checks before creating the byte array, on getting a non-SASL message this creates a huge
      byte array from some garbage size.

      For HS2, an attempt was made to fix it here: HIVE-6468, which never went in. I think for 0.15.0 it's best to upgarde to Thrift 0.9.2.

      Attachments

        1. HIVE-8829.1.patch
          3 kB
          Prasad Suresh Mujumdar
        2. HIVE-8829.1.patch
          3 kB
          Brock Noland

        Issue Links

          breaks

          Bug - A problem which impairs or prevents the functions of the product. HIVE-8931 Test TestAccumuloCliDriver is not completing

          • Major - Major loss of function.
          • Resolved
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-2444 Add TCP keepalive option for the hive jdbc driver

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-6468 HS2 & Metastore using SASL out of memory error when curl sends a get request

          • Major - Major loss of function.
          • Closed
          supercedes

          Sub-task - The sub-task of the issue HIVE-4224 Upgrade to Thrift 1.0 when available

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HIVE-6964 Upgrade to Thrift 0.9.1

          • Major - Major loss of function.
          • Resolved

          Task - A task that needs to be done. HIVE-8418 Upgrade to Thrift 0.9.1

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              prasadm Prasad Suresh Mujumdar
              vgumashta Vaibhav Gumashta
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: