Uploaded image for project: 'Geode'
  1. Geode
  2. GEODE-733

Cluster Config service can throw a spurious exception

    XMLWordPrintableJSON

Details

    Description

      Occasionally this exception has been seen:

      info 2015/04/07 19:08:34.129 UTC locator0 <locator request thread[44]> tid=0x135a0] Attempting to unlock  <DLockService@55d6956e named __CLUSTER_CONFIG_LS de
      stroyed=false grantorId=[LockGrantorId: lockGrantorMember=10.138.46.164(locator0:1522:locator)<ec><v0>:55053, lockGrantorVersion=1, lockGrantorSerialNumber=3]
       grantor=<DLockGrantor@41c8af04 state=READY name=__CLUSTER_CONFIG_LS version=1>>  :  __CLUSTER_CONFIG_LOCK , but this thread does not own the lock.
      com.gemstone.gemfire.distributed.LockNotHeldException: Attempting to unlock  <DLockService@55d6956e named __CLUSTER_CONFIG_LS destroyed=false grantorId=[LockG
      rantorId: lockGrantorMember=10.138.46.164(locator0:1522:locator)<ec><v0>:55053, lockGrantorVersion=1, lockGrantorSerialNumber=3] grantor=<DLockGrantor@41c8af0
      4 state=READY name=__CLUSTER_CONFIG_LS version=1>>  :  __CLUSTER_CONFIG_LOCK , but this thread does not own the lock.
              at com.gemstone.gemfire.distributed.internal.locks.DLockService.unlock(DLockService.java:1941)
              at com.gemstone.gemfire.distributed.internal.SharedConfiguration.createConfigurationReponse(SharedConfiguration.java:376)
              at com.gemstone.gemfire.management.internal.configuration.handlers.ConfigurationRequestHandler.processRequest(ConfigurationRequestHandler.java:44)
              at com.gemstone.gemfire.distributed.internal.InternalLocator$LocatorHandler.processRequest(InternalLocator.java:1394)
              at com.gemstone.org.jgroups.stack.tcpserver.TcpServer$3.run(TcpServer.java:373)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      

      The problem is that there is an unlock in a finally block but no check to determine if the lock is actually being held.

      This was looked at previously and the proposed solution was:

      --- a/src/com/gemstone/gemfire/distributed/internal/SharedConfiguration.java
      +++ b/src/com/gemstone/gemfire/distributed/internal/SharedConfiguration.java
      @@ -362,8 +362,9 @@ public class SharedConfiguration {
       
           for (int i=0; i<configRequest.getNumAttempts(); i++) {
             boolean isLocked = sharedConfigLockingService.lock(SHARED_CONFIG_LOCK_NAME, 5000, 5000);
      -      try {
      -        if (isLocked) {
      +     
      +      if (isLocked) {
      +        try {
                 logger.info("Building up configuration response with following configurations");
                 Set<String> groups = configRequest.getGroups();
                 groups.add(SharedConfiguration.CLUSTER_CONFIG);
      @@ -381,10 +382,10 @@ public class SharedConfiguration {
                 }
                 configResponse.setFailedToGetSharedConfig(false);
                 return configResponse;
      +        }finally {
      +            sharedConfigLockingService.unlock(SHARED_CONFIG_LOCK_NAME);
               }
      -      } finally {
      -        sharedConfigLockingService.unlock(SHARED_CONFIG_LOCK_NAME);
      -      }
      +      } 
      

      Attachments

        Issue Links

          Activity

            People

              echobravo Ernest Burghardt
              jens.deppe Jens Deppe
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m