Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-5516

Regression Testing:when yarn-application model, some functions cannot be work normally

    XMLWordPrintableJSON

Details

    Description

      after change to yarn-application mode, some functions cannot work normally, even effect other mudules, eg:

      • Q1、when use default config of 'zeppelin.flink.uiWebUrl', it will show a strange Flink Web URL in zeppelin paragraph running UI. eg: `zeppelin.domain`/false/#/job/jobId. or else localhost display.

       

      private def getDisplayedJMWebUrl(yarnAppId: String): String = {
        // `zeppelin.flink.uiWebUrl` is flink jm url template, {{applicationId}} will be replaced
        // with real yarn app id.
        val flinkUIWebUrl = properties.getProperty("zeppelin.flink.uiWebUrl")
        if (StringUtils.isNotBlank(flinkUIWebUrl)) {
          flinkUIWebUrl.replace("{{applicationId}}", yarnAppId)
        } else {
          this.jmWebUrl
        }
      }
      
      // 2
      flink/flink-scala-parent/src/main/resources/interpreter-setting.json
      "zeppelin.flink.uiWebUrl": {
        "envName": null,
        "propertyName": null,
        "defaultValue": false,
        "description": "User specified Flink JobManager url, it could be used in remote mode where Flink cluster is already started, or could be used as url template, e.g. https://knox-server:8443/gateway/cluster-topo/yarn/proxy/{{applicationId}}/ where {{applicationId}} would be replaced with yarn app id",
        "type": "string"
      },

       

       

      • Q2、when recovery is enabled, some functions different even cannot work after zeppelin restarted.
        • S1、it will report NPE or RuntimeException, Cannot recover unless restarted (But some times it's work ok)
        • S2、cannot rerun conf, env it not changed, it will report :java.io.IOException: Can not change interpreter properties when interpreter process has already been launched
        • S3、Flink Web UI URL button lost,see follow screenshot:

       

      • Q3、yarn model `hadoop classpath` need only, but yarn-application mode we must config HADOOP_CLASSPATH env

       

      If there are problems that have not be clearly described, you can contact me at any time:

      email: aib628@dingtalk.com

       

      Attachments

        1. image-2021-09-13-14-12-55-914.png
          212 kB
          vmaster.cc
        2. image-2021-09-13-14-31-38-494.png
          961 kB
          vmaster.cc
        3. image-2021-09-13-15-03-41-258.png
          1023 kB
          vmaster.cc
        4. image-2021-09-13-15-25-39-031.png
          406 kB
          vmaster.cc

        Activity

          People

            Unassigned Unassigned
            vmaster vmaster.cc
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: