Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5972

Flatten SMT does not work with null values

    XMLWordPrintableJSON

Details

    • Patch

    Description

      Hi,

      I noticed a bug in Flatten SMT while doing tests with different SMTs that are provided out-of-box.

      Flatten SMT does not work as expected with schemaless JSON that has properties with null values.

      Example json:

        {A={D=dValue, B=null, C=cValue}}

      The issue is in if statement that checks for null value.
      Current version:

        for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
            final String fieldName = fieldName(fieldNamePrefix, entry.getKey());
            Object value = entry.getValue();
            if (value == null) {
                newRecord.put(fieldName(fieldNamePrefix, entry.getKey()), null);
                return;
            }
...

      should be

        for (Map.Entry<String, Object> entry : originalRecord.entrySet()) {
            final String fieldName = fieldName(fieldNamePrefix, entry.getKey());
            Object value = entry.getValue();
            if (value == null) {
                newRecord.put(fieldName(fieldNamePrefix, entry.getKey()), null);
                continue;
            }

      I have attached a patch containing the fix for this issue.

      Attachments

        1. kafka-transforms.patch
          2 kB
          Tomas Zuklys

        Issue Links

          is superceded by

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-6605 Flatten SMT does not properly handle fields that are null

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link GitHub Pull Request #4021

          Web Link GitHub Pull Request #4021

          Activity

            People

              sssanthalingam siva santhalingam
              tomas.zuklys@gmail.com Tomas Zuklys
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: