Details
Description
We are having an issue where SSLIOSession::decryptData will effectively become an endless loop when the size of the inEncryptedBuf buffer is larger than the size of the inPlainBuf.
In this scenario the doUnwrap completely fills up the inPlainBuf. This causes the
if (inPlainBuf.hasRemaining())
to return false and never clear anything out of the inPlainBuf buffer.
From what we can tell the
if (inPlainBuf.hasRemaining()) {
should be removed, as it is in error. There is no reason that this buffer being full should prevent it from being emptied.
We verified that removing this code from 5.1.2 resolved the issue we were facing, along with all tests continuing to pass. There does not appear to be any change to this code in 5.2 alpha.
This issue shows up when we use BouncyCastle for FIPS validated TLS, as it creates a larger inEncryptedBuf then the SUN stack. This issue is completely reproducible when we get a large response from our endpoint.