Description
When shuffle jobs are launched by Mesos in a virtual network, Mesos scheduler sets executor --hostname parameter to 0.0.0.0 in the case when spark.mesos.network.name is provided. This makes executors use 0.0.0.0 as their advertised address and, in the presence of shuffle, executors fail to fetch shuffle blocks from each other using 0.0.0.0 as the origin. When a virtual network is used the hostname or IP address is not known upfront and assigned to a container at its start time so the executor process needs to advertise the correct dynamically assigned address to be reachable by other executors.
The bug described above prevents Mesos users from running any jobs which involve shuffle due to the inability of executors to fetch shuffle blocks because of incorrect advertised address when virtual network is used.