Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-3396

AuthOozieClient leaks memory

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 5.0.0
    • None
    • None
    • None

    Description

      Phenomenon:
      Recently we upgraded the AuthOozieClient and added the OOZIE-2447. When we were running online, the server memory usage continued to increase. Through the jmap and jhat command analysis, it was found that the deleteOnExist method in the writeAuthToken method under the AuthOozieClient class occupies a large amount of memory, which is caused by OOZIE-2447. Also, because I have applied the OOZIE-3379 in the production environment. I still find that the .oozie-auth-token-xxxx file update time has changed, but the file content has not changed.

      After reviewing the code, I found that currentToken.equals(readToken) in the AuthOozieClient class is always false because the AuthenticatedURL.token does not have an equals method. The reason is because currentToken.equals(readToken) is always false, causing the tmp file to be created continuously, but there is a memory leak due to the deleteOnExist method (https://stackoverflow.com/questions/40119188/memory-leak-on-deleteonexithook)

      Solution:
      1. In AuthOozieClient, change to currentToken.toString().equals(readToken.toString())
      2. In addition, the deleteOnExist method should be changed to the delete() in AuthOozieClient class.

      Doubt:
      Why delete the equals method in the AuthenticatedURL.Token class in Hadoop? See: https://issues.apache.org/jira/browse/HADOOP-10771

      Attachments

        Issue Links

          Activity

            People

              zuston Junfan Zhang
              zuston Junfan Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: