Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-4394

Netty4 ShuffleHandler: should use 1 boss thread

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.10.3
    • None
    • None

    Description

      https://github.com/apache/tez/commit/01847814d32aac6b77a6810ea25d6780fb94b257#diff-5c5dffcaf349fe521937812b49849c59fba3e67fc53d74fb990b7e0ef7ff5e59R182

          bossGroup = new NioEventLoopGroup(maxShuffleThreads, new ThreadFactory() {
            @Override
            public Thread newThread(Runnable r) {
              return new Thread(r, BOSS_THREAD_NAME_PREFIX + bossThreadCounter.incrementAndGet());
            }
          });
      

      With netty3, we initialized NioServerSocketChannelFactory with below constructor:

          selector = new NioServerSocketChannelFactory(
              Executors.newCachedThreadPool(bossFactory),
              Executors.newCachedThreadPool(workerFactory),
              maxShuffleThreads);
      

      this always creates 1 boss and n maxShuffleThreads threads:

          public NioServerSocketChannelFactory(
                  Executor bossExecutor, Executor workerExecutor,
                  int workerCount) {
              this(bossExecutor, 1, workerExecutor, workerCount);
          }
      

      In Nio model, bossThread take care all bounded socket(listen socket), workerThread take care Accepted-socket (included IO and call event method such as messageReceived).
      In case of the boss thread pool, actually the thread pool is unneeded because current implementation acquires only a single thread from it. That thread sleeps on the selector for server socket most of the time, after accepting connection that connection is registered with a worker. From that moment on worker is responsible for serving that connection.

      Attachments

        Issue Links

          Activity

            People

              abstractdog László Bodor
              abstractdog László Bodor
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 0.5h
                  0.5h