Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24276

hbase spark connector doesn't support writing to table not in default namespace

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • connector-1.0.0
    • hbase-connectors-1.0.1
    • hbase-connectors, spark
    • None
      • HBase 2.2.4
      • Hadoop 2.10.0
      • Spark 2.4.5

    Description

      Defining the following table catalog:

      val catalog = """{
  |"table": {"namespace": "ns1", "name": "test1"},
  |"rowkey": "id",
  |"columns": {
    |"id": {"cf": "rowkey", "col": "id", "type": "string"}, 
    |"x": {"cf": "d", "col": "xxx", "type": "int"}
  |}
|}""".stripMargin

      Try to write some test data to ns1:test1 table using spark:

      val df = Seq(("abc", 1), ("def", 2)).toDF("id", "x")
df.write.options(Map(HBaseTableCatalog.tableCatalog -> catalog, "hbase.spark.use.hbasecontext" -> "false", HBaseTableCatalog.newTable-> "5")).format("org.apache.hadoop.hbase.spark").save()

      After executing the code above, I found out that the test data was written to default:test1, rather than ns1:test1.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HBASE-27624 Cannot Specify Namespace via the hbase.table Option in Spark Connector

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub PR: support writing to table in arbitrary namespace

          Activity

            People

              naitree Naitree Zhu
              naitree Naitree Zhu
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: