Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-10858

beam.io.jdbc not working on Google Cloud Dataflow

Details

    Description

      Using the JDBC python wrapper to connect to an external server is failing on Cloud Dataflow.

      The error produced is;

      RuntimeError: /root/apache-beam-custom/packages/beam/sdks/java/extensions/schemaio-expansion-service/build/libs/beam-sdks-java-extensions-schemaio-expansion-service-2.25.0-SNAPSHOT.jar not found. Please build the server with cd /root/apache-beam-custom/packages/beam; ./gradlew :sdks:java:extensions:schemaio-expansion-service:shadowJar

      You can recreate the issue by running the below

      from apache_beam.io import jdbc
      jdbc.ReadFromJdbc(
       table_name='[tablename]',
       driver_class_name='com.mysql.cj.jdbc.Driver',
       jdbc_url='jdbc:mysql://[host]:3306/[databasename]',
       username='[username]',
       password='[password]')

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            EhabElBadrawi Ehab El Badrawi
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: