Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-5801

Evaluating Expression Language can in many cases be made much more efficient

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • Core Framework
    • None

    Description

      When a StandardPropertyValue is obtained and evaluateAttributeExpressions is called, it builds the entire Evaluator Tree each time. This was done to ensure that Evaluator.evaluate() is called only once. However, the requirement to call this only once was introduced as a way to have anyMatchingAttribute, anyAttribute, allMatchingAttributes, allAttributes, etc. methods work, and these are rarely used. I.e., we introduced semantics that significantly slow performance in order to provide functionality that is used maybe 1% of the time. Instead, we should optimize for the 99% use case and incur a penalty, if necessary, in the 1% use case instead. Profiling the ConsumeKafkaRecord processor shows that 80% of the time in that method is evaluating Expression Language for `${schema.name}` to determine which schema should be used. We can likely make this evaluation just as quick as attributeMap.get("schema.name") by pre-building the Evaluators and re-using them.

      Attachments

        Issue Links

          Activity

            People

              markap14 Mark Payne
              markap14 Mark Payne
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h 10m
                  1h 10m