Replication can sometimes wait until shutdown or core unload until removing some tmp directories.

We want to be aggressive in dir removal where it makes sense - like in this case. I've added some additional testing to ensure this.