High CPU usage with log4j2

Kafka brokers make use of the Logging trait which instantiates a Logger object for every instantiation of the class using the trait by default.

When using log4j2 as the logging implementation, the instantiation of a Logger object requires a stack traversal [1] and [2].

While LOG4J2-2940 ensures stack is not traversed unless required, the default ContextSelector, ClassLoaderContextSelector causes a stack traversal.

These stack traversals are frequent and quite CPU intensive and profiling suggests they consume ~5% CPU time (of the CPU stacks we have profiled on a sample of clusters). While log4j2 users can potentially avoid this by changing the context selector in their configuration, it is easy to overlook and the default configurations results in high CPU usage inadvertently.

An easy fix would be to instantiate the loggers statically for some commonly instantiated classes in Kafka which make use of the Logging trait.