Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
7.0.0
-
None
Description
Nullable StructArrays created with FixedSizeList fields seem to incorrectly initialise.
I've tried to create them using both the `Builder` and `makeVector` methods and I believe it breaks the specification.
I believe that the underlying arrays for the fields of a struct should be the length of the struct array.
However the `nullCount`s, `numChildren`s, and other methods all return different numbers. (And this is causing a problem when we try to read the same memory within Rust as the lengths and offsets differ)
Specifically this:
let list_field = new Field('n1', new FixedSizeList(2, Float64), true); let struct_field = new Field('foo', new Struct([list_field]), true); let builder = new Builder({ type:struct_field.type, nullValues: [null, undefined], }); builder.append(null); console.log('Builder:' + JSON.stringify(builder)); console.log('numChildren: ' + builder.numChildren); console.log('nullCount: ' + builder.nullCount); console.log('length: ' + builder.length); let vec1 = builder.toVector(); console.log('Vector from Builder:' + vec1); console.log('numChildren: ' + vec1.numChildren); console.log('nullCount: ' + vec1.nullCount); console.log('length: ' + vec1.length); let vec2 = makeVector({ data: [null], type:struct_field.type, nullable:true, }); console.log('Vector from makeVector:' + JSON.stringify(vec2)); console.log('numChildren: ' + vec2.numChildren); console.log('nullCount: ' + vec2.nullCount); console.log('length: ' + vec2.length);
Results in (I've removed some fields for brevity)
Builder: \{"length":1,"finished":false,"type":{"children":[{"name":"n1","type":{"listSize":2,"children":[null]},"nullable":true,"metadata":{}}]},"children":[],"nullValues":[null,null],"stride":1,"_nulls":\{"buffer": ... ,"stride":0.125,"BYTES_PER_ELEMENT":1,"length":1,"numValid":0}} numChildren: 0 nullCount: 1 length: 1 Vector from Builder:[] numChildren: 1 nullCount: 1 length: 1 Vector from makeVector:[] numChildren: 1 nullCount: 0 length: 0
I tried to test this within stackblitz, the Project source code should be available here
Attachments
Issue Links
- is fixed by
-
ARROW-15705 [JavaScript] Structs don't append nulls properly
- Resolved