Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-4361 Upgrade protobuf to 3.x
  3. TEZ-4300

Download protoc automatically compile/development time

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.9.3, 0.10.2
    • None
    • None

    Description

      Currently, Tez cannot be built out of the box because tez-api needs a protobuf compiler (protoc executable), which was also a problematic dependency in a sense that downloading/installing it is a burden on developers. Even if we have tons of gists for that, it's still uncomfortable. Not to mention if we want to support more versions at the same time.
      Shipping the typically used versions (2.5.0, 3.x for later) of the protoc executable could make sense, but I'm not sure at the moment if it has any license issues, need to check, also it's not general, instead an intelligent script could do it for developers.
      For 2.5, it should be done from source: https://github.com/protocolbuffers/protobuf/releases/tag/v2.5.0
      But for newer versions, the protoc executable can be downloaded from the releases page: https://github.com/protocolbuffers/protobuf/releases/tag/v3.7.1

      https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protoc-3.7.1-osx-x86_64.zip
      https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protoc-3.7.1-linux-x86_64.zip

      Attachments

        1. tez_build_protobuf_3_7_1.log
          16 kB
          László Bodor
        2. TestDAGRecovery-tez-tests.tgz
          1.47 MB
          Martin Tzvetanov Grigorov
        3. org.apache.tez.dag.history.events.TestHistoryEventsProtoConversion-output.txt
          0.4 kB
          Martin Tzvetanov Grigorov
        4. org.apache.tez.dag.app.TestRecoveryParser-output.txt
          34 kB
          Martin Tzvetanov Grigorov
        5. org.apache.tez.dag.app.dag.impl.TestDAGRecovery-output.txt
          848 kB
          Martin Tzvetanov Grigorov

        Issue Links

          Activity

            People

              abstractdog László Bodor
              abstractdog László Bodor
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h
                  4h