[IMPALA-12373] Implement Small String Optimization for StringValue - ASF JIRA

XML

Word

Printable

JSON

Implement Small String Optimization for StringValue.

Current memory layout of StringValue is:

  char* ptr;  // 8 byte
  int len;    // 4 byte

For small strings with size up to 8 we could store the string contents in the bytes of the 'ptr'. Something like that:

  union {
    char* ptr;
    char small_buf[sizeof(ptr)];
  };
  int len;

Many C++ string implementations use the Small String Optimization to speed up work with small strings. For example:

Microsoft STL, libstdc++, libc++, Boost, Folly.

causes

IMPALA-12606 Sporadic failures around query_test.test_queries.TestQueries.test_intersect

IMPALA-13138 Never smallify existing StringValue objects, only new ones during DeepCopy

1.	Adjust estimations for small strings	Open	Zoltán Borók-Nagy
2.	Harden long string testing	Open	Unassigned
3.	Add backend flag to disable small string optimization	Open	Zoltán Borók-Nagy