Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-19206

Hadoop release contains a 530MB bundle-2.23.19.jar

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • build
    • None

    Description

      The size of Hadoop binary release (v3.4.0) is 1.7 GB.

      hadoop-3.4.0$du -h -d 1
$du -h -d 1 .
2.0M	./bin
260K	./libexec
 72K	./include
212K	./sbin
184K	./etc
232K	./licenses-binary
316M	./lib
1.4G	./share
1.7G	.

      A large component is bundle-2.23.19.jar, which is AWS Java SDK :: Bundle

      hadoop-3.4.0$ls -lh share/hadoop/tools/lib/bundle-2.23.19.jar  
-rw-r--r--@ 1 szetszwo  staff   530M Mar  4 15:41 share/hadoop/tools/lib/bundle-2.23.19.jar

      We should revisit if such a large jar is really needed to be included in the release.

      Attachments

        Issue Links

          duplicates

          Sub-task - The sub-task of the issue HADOOP-19083 provide hadoop binary tarball without aws v2 sdk

          • Major - Major loss of function.
          • Open

          Activity

            People

              Unassigned Unassigned
              szetszwo Tsz-wo Sze
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: