Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-15651

[JavaScript] Structs incorrectly initialise null values

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 7.0.0
    • 8.0.0
    • JavaScript
    • None

    Description

      Nullable StructArrays created with FixedSizeList fields seem to incorrectly initialise.
      I've tried to create them using both the `Builder` and `makeVector` methods and I believe it breaks the specification.

      I believe that the underlying arrays for the fields of a struct should be the length of the struct array.

      However the `nullCount`s, `numChildren`s, and other methods all return different numbers. (And this is causing a problem when we try to read the same memory within Rust as the lengths and offsets differ)

      Specifically this:

      let list_field = new Field('n1', new FixedSizeList(2, Float64), true);
      let struct_field = new Field('foo', new Struct([list_field]), true);
      
      let builder = new Builder({
      type:struct_field.type,
      nullValues: [null, undefined],
      });
      
      builder.append(null);
      
      console.log('Builder:' + JSON.stringify(builder));
      console.log('numChildren: ' + builder.numChildren);
      console.log('nullCount: ' + builder.nullCount);
      console.log('length: ' + builder.length);
      
      let vec1 = builder.toVector();
      console.log('Vector from Builder:' + vec1);
      console.log('numChildren: ' + vec1.numChildren);
      console.log('nullCount: ' + vec1.nullCount);
      console.log('length: ' + vec1.length);
      
      let vec2 = makeVector({
      data: [null],
      type:struct_field.type,
      nullable:true,
      });
      
      console.log('Vector from makeVector:' + JSON.stringify(vec2));
      console.log('numChildren: ' + vec2.numChildren);
      console.log('nullCount: ' + vec2.nullCount);
      console.log('length: ' + vec2.length);
      

      Results in (I've removed some fields for brevity)

      Builder: \{"length":1,"finished":false,"type":{"children":[{"name":"n1","type":{"listSize":2,"children":[null]},"nullable":true,"metadata":{}}]},"children":[],"nullValues":[null,null],"stride":1,"_nulls":\{"buffer": ... ,"stride":0.125,"BYTES_PER_ELEMENT":1,"length":1,"numValid":0}}
      numChildren: 0
      nullCount: 1
      length: 1
      
      Vector from Builder:[]
      numChildren: 1
      nullCount: 1
      length: 1
      
      Vector from makeVector:[]
      numChildren: 1
      nullCount: 0
      length: 0

      I tried to test this within stackblitz, the Project source code should be available here

      Attachments

        Issue Links

          Activity

            People

              alfredm Alfred Mountfield
              alfredm Alfred Mountfield
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: