Uploaded image for project: 'Tajo (Retired)'
  1. Tajo (Retired)
  2. TAJO-1534

DelimitedTextFile return null instead of a NullDatum

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.10.0
    • 0.11.0, 0.10.1
    • Storage
    • None

    Description

      If a text row is less than table schema size, vtuple returns null

      Attachments

        1. TAJO-1534_2.patch
          6 kB
          Jinho Kim
        2. TAJO-1534.patch
          6 kB
          Jinho Kim

        Activity

          githubbot ASF GitHub Bot added a comment -

          GitHub user jinossy opened a pull request:

          https://github.com/apache/tajo/pull/522

          TAJO-1534: DelimitedTextFile should be returned NullDatum instead of null

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/jinossy/tajo TAJO-1534

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/tajo/pull/522.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #522


          commit 3e734d96a9e7d76e4efc2acdc848873eae365227
          Author: Jinho Kim <jhkim@apache.org>
          Date: 2015-04-07T10:17:42Z

          TAJO-1534: DelimitedTextFile should be returned NullDatum instead of null.


          githubbot ASF GitHub Bot added a comment - GitHub user jinossy opened a pull request: https://github.com/apache/tajo/pull/522 TAJO-1534 : DelimitedTextFile should be returned NullDatum instead of null You can merge this pull request into a Git repository by running: $ git pull https://github.com/jinossy/tajo TAJO-1534 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/522.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #522 commit 3e734d96a9e7d76e4efc2acdc848873eae365227 Author: Jinho Kim <jhkim@apache.org> Date: 2015-04-07T10:17:42Z TAJO-1534 : DelimitedTextFile should be returned NullDatum instead of null.
          tajoqa Tajo QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12723601/TAJO-1534.patch
          against master revision release-0.9.0-rc0-244-gd160f6e.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

          +1 checkstyle. The patch generated 0 code style errors.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in tajo-storage/tajo-storage-hdfs.

          Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/722//testReport/
          Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/722//console

          This message is automatically generated.

          tajoqa Tajo QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12723601/TAJO-1534.patch against master revision release-0.9.0-rc0-244-gd160f6e. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-storage/tajo-storage-hdfs. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/722//testReport/ Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/722//console This message is automatically generated.
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/522#discussion_r28572801

          — Diff: tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java —
          @@ -1255,16 +1255,21 @@ public void init() throws IOException {

          for (int i = 0; i < targetColumnIndexes.length; i++) {
          int tid = targetColumnIndexes[i];
          + SelectedColumn col = new SelectedColumn();
          if (tid < columnNumber)

          { skippedColIDs[tid] = false; - SelectedColumn col = new SelectedColumn(); col.colIndex = tid; col.runLength = 0; col.prvLength = -1; col.rowReadIndex = 0; selectedColumns[i] = col; colValLenBufferReadIn[i] = new NonSyncDataInputBuffer(); + }

          else {
          — End diff –

          The newly added code is the mostly same as those of existing codes.

          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on a diff in the pull request: https://github.com/apache/tajo/pull/522#discussion_r28572801 — Diff: tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java — @@ -1255,16 +1255,21 @@ public void init() throws IOException { for (int i = 0; i < targetColumnIndexes.length; i++) { int tid = targetColumnIndexes [i] ; + SelectedColumn col = new SelectedColumn(); if (tid < columnNumber) { skippedColIDs[tid] = false; - SelectedColumn col = new SelectedColumn(); col.colIndex = tid; col.runLength = 0; col.prvLength = -1; col.rowReadIndex = 0; selectedColumns[i] = col; colValLenBufferReadIn[i] = new NonSyncDataInputBuffer(); + } else { — End diff – The newly added code is the mostly same as those of existing codes.
          githubbot ASF GitHub Bot added a comment -

          Github user jinossy commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/522#discussion_r28577893

          — Diff: tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java —
          @@ -1255,16 +1255,21 @@ public void init() throws IOException {

          for (int i = 0; i < targetColumnIndexes.length; i++) {
          int tid = targetColumnIndexes[i];
          + SelectedColumn col = new SelectedColumn();
          if (tid < columnNumber)

          { skippedColIDs[tid] = false; - SelectedColumn col = new SelectedColumn(); col.colIndex = tid; col.runLength = 0; col.prvLength = -1; col.rowReadIndex = 0; selectedColumns[i] = col; colValLenBufferReadIn[i] = new NonSyncDataInputBuffer(); + }

          else {
          — End diff –

          Thank you for nice finding.
          I've remove the duplicated codes

          githubbot ASF GitHub Bot added a comment - Github user jinossy commented on a diff in the pull request: https://github.com/apache/tajo/pull/522#discussion_r28577893 — Diff: tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java — @@ -1255,16 +1255,21 @@ public void init() throws IOException { for (int i = 0; i < targetColumnIndexes.length; i++) { int tid = targetColumnIndexes [i] ; + SelectedColumn col = new SelectedColumn(); if (tid < columnNumber) { skippedColIDs[tid] = false; - SelectedColumn col = new SelectedColumn(); col.colIndex = tid; col.runLength = 0; col.prvLength = -1; col.rowReadIndex = 0; selectedColumns[i] = col; colValLenBufferReadIn[i] = new NonSyncDataInputBuffer(); + } else { — End diff – Thank you for nice finding. I've remove the duplicated codes
          githubbot ASF GitHub Bot added a comment -

          Github user blrunner commented on the pull request:

          https://github.com/apache/tajo/pull/522#issuecomment-98460589

          +1

          It looks good to me and all unit test cases finished successfully.
          And if possible, could you add 0.11.0 to the fix version of JIRA issue?

          githubbot ASF GitHub Bot added a comment - Github user blrunner commented on the pull request: https://github.com/apache/tajo/pull/522#issuecomment-98460589 +1 It looks good to me and all unit test cases finished successfully. And if possible, could you add 0.11.0 to the fix version of JIRA issue?
          githubbot ASF GitHub Bot added a comment -

          Github user jinossy commented on the pull request:

          https://github.com/apache/tajo/pull/522#issuecomment-99299127

          @blrunner
          Thank you for your review. I will commit soon

          githubbot ASF GitHub Bot added a comment - Github user jinossy commented on the pull request: https://github.com/apache/tajo/pull/522#issuecomment-99299127 @blrunner Thank you for your review. I will commit soon
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/tajo/pull/522

          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/522
          jhkim Jinho Kim added a comment -

          committed it

          jhkim Jinho Kim added a comment - committed it
          hudson Hudson added a comment -

          ABORTED: Integrated in Tajo-master-CODEGEN-build #332 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/332/)
          TAJO-1534: DelimitedTextFile return null instead of a NullDatum. (jinho) (jhkim: rev b6b9d46318d0aaa0346578b5ecf11b9d9ba74b0c)

          • CHANGES
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java
          hudson Hudson added a comment - ABORTED: Integrated in Tajo-master-CODEGEN-build #332 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/332/ ) TAJO-1534 : DelimitedTextFile return null instead of a NullDatum. (jinho) (jhkim: rev b6b9d46318d0aaa0346578b5ecf11b9d9ba74b0c) CHANGES tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-master-build #693 (See https://builds.apache.org/job/Tajo-master-build/693/)
          TAJO-1534: DelimitedTextFile return null instead of a NullDatum. (jinho) (jhkim: rev b6b9d46318d0aaa0346578b5ecf11b9d9ba74b0c)

          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java
          • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java
          • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java
          • CHANGES
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #693 (See https://builds.apache.org/job/Tajo-master-build/693/ ) TAJO-1534 : DelimitedTextFile return null instead of a NullDatum. (jinho) (jhkim: rev b6b9d46318d0aaa0346578b5ecf11b9d9ba74b0c) tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java CHANGES

          People

            jhkim Jinho Kim
            jhkim Jinho Kim
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: