Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7712

Impala read from and write to GCS

    XMLWordPrintableJSON

Details

    • ghx-label-4

    Description

      Can Impala read from and write to google cloud storage GCS like the way it does with amazon s3

      I have tested the use case with S3, but when talking to GCS impala errors out with:
      Query: create table gcs_impala2 (title string) location 'gs://mybucket-gcs/some_data/' ERROR: AnalysisException: null CAUSED BY: RuntimeException: java.lang.ClassNotFoundException: Class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem not found CAUSED BY: ClassNotFoundException: Class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem not found
       On the same cluster i have Hive talking to GCS using the GCS connector jar provided by google form :
      https://cloud.google.com/dataproc/docs/concepts/connectors/install-storage-connector
       
      Also, HDFS reads and writes from/to GCS.
       
      Made sure java version matches and appropriate values are in classpath.
       
      Appreciate your time and effort.
      Thanks

      Attachments

        Issue Links

          Activity

            People

              stigahuang Quanlong Huang
              Haaris Haaris
              Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: