Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5507

Millions of "Failure finding Drillbit running on host" info messages in foreman logs

    XMLWordPrintableJSON

Details

    Description

      When foreman tries to execute a query with data files that reside on nodes other than the nodes that are running drillbits, there are millions of messages in the log files like this:

      2017-05-10 13:22:10,916 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp0535.mydomain.com. Skipping affinity to that host.

      After 200 MB of these messages, 960K of them and 7 seconds of planning, it continues to execute this query. Is there a way to disable / suppress these messages? What's causing this? Should this be printed with INFO logging as we're seeing or only with DEBUG logging? Is there a way to turn off this check? Below is a snippet from foreman logs:

      2017-05-10 13:22:10,916 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp0535.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp1402.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp0929.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp1098.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp1230.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp1388.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp0535.mydomain.com. Skipping affinity to that host.
      2017-05-10 13:22:10,923 [26eca5b2-a847-b7fe-adff-1685de29bb7a:foreman] INFO o.a.d.e.s.schedule.BlockMapBuilder - Failure finding Drillbit running on host hostlp1291.mydomain.com. Skipping affinity to that host.

      The hostname and domain are scrubbed.

      Attachments

        Issue Links

          Activity

            People

              timothyfarkas Timothy Farkas
              veeran Veera Naranammalpuram
              Paul Rogers Paul Rogers
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: