Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-43366

Spark Driver Bind Address is off-by-one

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.3.3
    • None
    • Block Manager
    • None

    Description

      I have the following environment variable set in my driver pod configuration:

      SPARK_DRIVER_BIND_ADDRESS=10.244.0.53

      However, I see an off-by-one IP address being referred to in the Spark logs:

      23/05/04 02:37:03 INFO KubernetesClusterSchedulerBackend$KubernetesDriverEndpoint: Registered executor NettyRpcEndpointRef(spark-client://Executor) (10.244.0.54:53140) with ID 1,  ResourceProfileId 0
23/05/04 02:37:03 INFO BlockManagerMasterEndpoint: Registering block manager 10.244.0.54:32805 with 413.9 MiB RAM, BlockManagerId(1, 10.244.0.54, 32805, None)

      I am not sure why this might be the case.

      Attachments

        Activity

          People

            Unassigned Unassigned
            derektbrown Derek Brown
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: