Uploaded image for project: 'Apache Apex Malhar'
  1. Apache Apex Malhar
  2. APEXMALHAR-1415

Clean-up Hdfs Output operators in library and incorporate the features of fault-tolerant Writer created for a POC

    XMLWordPrintableJSON

Details

    • Task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • None
    • None

    Description

      There are multiple Hdfs Output Operator in Malhar lib. A few of them are inefficient because they immediately flush after they write a tuple to output stream. This makes the operator very slow.
      For a POC we created a high-throughput fault-tolerant hdfs output operator. We need the capabilities of that operator in library and refactor the existing implementations to avoid duplicating functionality.

      Attachments

        Activity

          People

            timothyfarkas Timothy Farkas
            csingh Chandni Singh
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: