Build Global Dictionary in no time

Backgroud

Currently, realtime OLAP do not support COUNT_DISTINCT(bitmap) for non interger type.
Because of the lack the ability of encoding string at once, so I want to use RocksDB & HBase as implementation of streaming distributed dictionary. 

Design

1. each receiver will own a local dict cache
2. all receiver will share a remote dict storage
3. we choose to use RocksDB as local dict cache
4. we choose to use HBase as remote dict storage
    
5. for each cube, we will create a local dict and a hbase table
6. we will create column family both in RocksDB and HBase for each column which occur in COUNT_DISTINCT

Design Diagram