Uploaded image for project: 'Commons Compress'
  1. Commons Compress
  2. COMPRESS-381

performance issue when using default Wiki/docs bzip2 compression Factory methods

    XMLWordPrintableJSON

Details

    Description

      Hello

      We are going to use this project's bzip2 implementation as it performed best for our use case (tested using https://github.com/ning/jvm-compressor-benchmark).

      However, when following the default examples using the wiki/example/javadoc pages , we were hitting a serious performance bottleneck.

      The reason: the default "compress" operation on a file which is suggested, is very slow, maybe because of disk I/O and lack of caching.

      For a 2 MB tiff file, bzip2 compression takes about 3 seconds with code (A), whereas code (B) takes only about 0.5 seconds!
      So it would be good to adapt documentation or take a look at bottle neck.

      Kind regards

      Dominique

      >>>
      FileInputStream fin = new FileInputStream(infile);
      BufferedInputStream bufferin = new BufferedInputStream(fin);
      final FileOutputStream outStream = new FileOutputStream(outfile);
      CompressorOutputStream cos = new CompressorStreamFactory() .createCompressorOutputStream(CompressorStreamFactory.BZIP2, outStream);
      IOUtils.copy(fin, cos);
      cos.close();
      >>>

      B:
      <<<<<
      final byte[] uncompressed = Files.readAllBytes(infile.toPath());
      ByteArrayOutputStream rawOut = new ByteArrayOutputStream(uncompressed.length);

      BZip2CompressorOutputStream out = new BZip2CompressorOutputStream(rawOut, COMPRESSION_LEVEL);
      out.write(uncompressed);
      out.close();
      FileOutputStream fos = new FileOutputStream(outfile);
      rawOut.writeTo(fos);
      fos.close();
      >>>>


      Pages with documentation:

      https://wiki.apache.org/commons/Compress
      https://commons.apache.org/proper/commons-compress/examples.html
      https://commons.apache.org/proper/commons-compress/javadocs/api-release/index.html

      Attachments

        Activity

          People

            Unassigned Unassigned
            dominique.demunck Dominique De Munck
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: