Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-5907

RowCsvInputFormat bug on parsing tsv

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.2.0
    • 1.2.1, 1.3.0
    • None

    Description

      The following snippet reproduce the problem (using the attached file as input):

      char fieldDelim = '\t';
          TypeInformation<?>[] fieldTypes = new TypeInformation<?>[51];
          for (int i = 0; i < fieldTypes.length; i++) {
            fieldTypes[i] = BasicTypeInfo.STRING_TYPE_INFO;
          }
          int[] fieldMask = new int[fieldTypes.length];
          for (int i = 0; i < fieldMask.length; i++) {
            fieldMask[i] = i;
          }
          RowCsvInputFormat csvIF = new RowCsvInputFormat(new Path(testCsv), fieldTypes, "\n", fieldDelim +"", 
             fieldMask, true);
          csvIF.setNestedFileEnumeration(true);
          DataSet<Row> csv = env.createInput(csvIF);
         csv.print()
      

      Attachments

        1. test.tsv
          0.2 kB
          Flavio Pompermaier

        Issue Links

          Activity

            People

              ykt836 Kurt Young
              f.pompermaier Flavio Pompermaier
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: