Invalidate replied calls in retry cache

The duration for which a client request entry is kept inside the retry cache should be long enough to cover all the retries made by the client. Retry cache would need to keep the metadata for all client requests until that duration is over. It can therefore require a large amount of heap in order to handle client requests. Since active heap usage is high it can also increase the frequency of JVM garbage collection.