Byte array different each time I generate a PDF

I have been using the FopFactory & FOUserAgent to render a PDF from an xsl & xml. This is working correctly however I'm running into an issue when writing my unit test.

I save the output PDF and in my unit test I am doing a render and then doing a byte[] compare. What I noticed is that the byte array is different each time I do a render.

When I compare the PDF's I am not able to see anything visually different. Does anyone know why it is changing each render? 

I tried setting the FOUserAgent creation date, but it still shows differences each render