Major Description
It looks like our property scoping rules may be broken, and have been so for a while.
Assume we have these two schemas:
*foo.dfdl.xsd*
<schema xmlns="http://www.w3.org/2001/XMLSchema" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" xmlns:dfdlx="http://www.ogf.org/dfdl/dfdl-1.0/extensions" xmlns:ex="http://example.com" targetNamespace="http://example.com" elementFormDefault="unqualified"> <include schemaLocation="/org/apache/daffodil/xsd/DFDLGeneralFormat.dfdl.xsd" /> <include schemaLocation="bar.dfdl.xsd" /> <annotation> <appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format ref="ex:GeneralFormat" representation="binary" byteOrder="littleEndian" /> </appinfo> </annotation> <simpleType name="fooType" dfdlx:repType="ex:uint16"> <restriction base="xs:string"> <enumeration value="65280" dfdlx:repValues="65280" /> <!-- 0x00FF as littleEndian --> <enumeration value="255" dfdlx:repValues="255" /> <!-- 0x00FF as bigEndian --> </restriction> </simpleType> <element name="foo1" type="ex:fooType" /> <element name="foo2" type="ex:uint16" /> </schema>
*bar.dfdl.xsd*
<schema xmlns="http://www.w3.org/2001/XMLSchema" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:dfdl="http://www.ogf.org/dfdl/dfdl-1.0/" xmlns:ex="http://example.com" targetNamespace="http://example.com" elementFormDefault="unqualified"> <include schemaLocation="/org/apache/daffodil/xsd/DFDLGeneralFormat.dfdl.xsd" /> <annotation> <appinfo source="http://www.ogf.org/dfdl/"> <dfdl:format ref="ex:GeneralFormat" representation="binary" byteOrder="bigEndian" /> </appinfo> </annotation> <simpleType name="uint16" dfdl:lengthKind="explicit" dfdl:length="2"> <restriction base="xs:unsignedShort" /> </simpleType> </schema>
Note that foo.dfdl.xsd defines the byteOrder to be littleEndian, whereas bar.dfdl.xsd defines the byteOrer to be bigEndian. According to scoping rules, the uint16 type is a binary, bigEndian, 2-byte short.
And it assumes the data to be parsed is 0x00 0xFF, which in bigEndian is 255 and in littleEndian is 65280.
Both the foo1 and foo2 ultimately have the same representation type of "uint16" (foo1 via a simpleType with the dfdlx:repType=uint16, and foo2 with the type=uint16) so we would expect similar results.
However, with Daffodil 3.6.0, foo1 parses 0x00FF as bigEndian (255) and foo2 parses as little endian (65820). After some discussions, we believe bigEndian is the correct behavior, since uint16 is defined in bar.dfdl.xsd with a bigEndian default format.
Also note that with Daffodil 3.5.0, both foo1 and foo2 parsed as the incorrect (but consistent) littleEndian. Changes to repType in 3.6.0 seemed to have changed only scoping for repType.
So all this seems to mean that normal property lookups are broken and have probably been broken for a long time since we haven't changed scoping rules in a very long time. dfdlx:repType lookups appear to behave as expected in 3.6.0, but were incorrect in 3.5.0. It's possible some schemas relied on that broken behavior and may need to be updated to work with 3.6.0