HARegionQueue over throttles when multiple threads attempt concurrent adds

HARegionQueue.checkQueueSizeConstraint has some code that implements a "throttle" on adds to a queue that is full. It is supposed to wait "eventEnqueueWaitTime" before doing an add. But because this code does two syncs (putGuard and permitMon) and only waits on one of them, it holds the other sync for the duration of this threads throttle. Any other concurrent thread trying to add to the queue gets stuck on the putGuard sync that is held by the first thread that is doing the timed wait. So it ends up waiting "eventEnqueueWaitTime" to acquire the first sync and then ends up waiting again "eventEnqueueWaitTime" when it does its own timed wait. If you have 10 concurrent threads trying to add one of them will end up waiting 10 *  "eventEnqueueWaitTime".

A couple ideas of how to fix this. Get rid of the putGuard and just use permitMon. Then as soon as the first thread goes into its timed wait another thread is allowed to sync on permitMon. But if this is done then we need to think carefully about the code inside this sync block since it can not be executed while one or more other threads are waiting in permitMon.

The other solution would be to compute the elapsed time it took to get into the first sync and subtract that from the time we wait on permitMon. This seems like a simple solution but does introduce at least one call of get time (the second call is only needed if the queue is full).