Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-17384

Support reading hbase conf dir from flink-conf.yaml

    XMLWordPrintableJSON

Details

    • Hide
      After FLINK-17384 hbase connector supports reading hbase configuration directory from flink-conf.yaml through the `env.hbase.conf.dir` property when the `HBASE_CONF_DIR` environment variable is not set.
      And If the above strategy fails ,the client is using the HBASE_HOME environment variable. If it is set, the client tries to access $HBASE_HOME /conf .
      Show
      After FLINK-17384 hbase connector supports reading hbase configuration directory from flink-conf.yaml through the `env.hbase.conf.dir` property when the `HBASE_CONF_DIR` environment variable is not set. And If the above strategy fails ,the client is using the HBASE_HOME environment variable. If it is set, the client tries to access $HBASE_HOME /conf .

    Description

      Currently when using hbase connector with Flink SQL, below manual steps are required:

      1. export HBASE_CONF_DIR
      2. add hbase libs to flink_lib(because the hbase connnector doesn't have client's( and others) jar)

      And we should improve this.

      For 1) we could support read hbase conf dir from flink-conf.yaml just like hadoop/yarn does.

      For 2) we should support HBASE_CLASSPATH in config.sh. In case of jar conflicts such as guava, we also should support flink-hbase-shaded just like hadoop does

      In this JIRA we focus on implementing the 1st proposal.

      Attachments

        Issue Links

          relates to

          Improvement - An improvement or enhancement to an existing feature or task. FLINK-17678 Create flink-sql-connector-hbase module to shade HBase

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #12144

          Activity

            People

              jackylau Jacky Lau
              jackylau Jacky Lau
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: