Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-5758

BigQuery hits socket timeout before reaching "wait_time" setting

    XMLWordPrintableJSON

Details

    Description

      The zeppelin.bigquery.wait_time BigQuery interpreter parameter is only useful up to a value of 30 seconds. Anything beyond that exceeds the underlying HTTP client's default read timeout and will result in a java.net.SocketTimeoutException: Read timed out exception being thrown. (A full stack trace is attached.)

      Google's Java API guide suggests overriding the HttpRequestInitializer to set the desired connect and read timeouts: https://developers.google.com/api-client-library/java/google-api-java-client/errors#timeouts

      This exact approach isn't feasible because the BigQuery interpreter's createAuthorizedClient method is static. Instead, we can modify the solution to use an approach similar to this StackOverflow answer which uses the builder's setHttpRequestInitializer: https://stackoverflow.com/a/32894630

      It should be noted that setting the read timeout too large likely won't provide any value.  Regardless of the timeoutMs value, BigQuery will always return a response within ~200 seconds regardless if the job has actually completed or not: 
      https://cloud.google.com/bigquery/docs/reference/rest/v2/jobs/getQueryResults#query-parameters
      Given that the BigQuery interpreter doesn't handle jobComplete being false, there's no reason to set the read timeout much larger than 200 seconds.
       
      I've attached a diff of the changes I applied to fix this issue.  It should be noted that I am not a Java developer, so I apologize if the solution is a bit crude.
       

      Attachments

        1. stacktrace.log
          4 kB
          Marcus Truscello
        2. bigquery-timeout.patch
          3 kB
          Marcus Truscello

        Activity

          People

            Unassigned Unassigned
            mtruscello Marcus Truscello
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: