Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
v3.1.0
-
None
-
None
-
Docker setup
Description
We have an IOT Data Platform which saves in Kafka Json messages with the following format:
{ "SysProperties": { "EventId": "1610120564321723276", "EnqueueTime": "2021-01-08T15:42:44.045Z", "UserName": "dummyuser" }, "AppProperties": { "EventDefinition": "EquipmentData", "EventTime": "2021-01-08T15:42:43.26+00:00", "ApplicationName": "ConnectIoTSimulator", "ApplicationContext": null }, "Data": { "Equipment": "DUMMY-01", "Material": "DUMMYZA9983884", "MeasurementData": { "Temp": 57, "Pressure": 48.379, "Energy": 47, "Humidity": 31.727 } }
From these messages, we are trying to build a cube in which the "AppProperties.EventTime" shall be the timestamp column (using format yyyy-MM-dd'T'HH:mm:ss.SSSZZ), "Data.MeasurementData.Temp", "Data.MeasurementData.Pressure", "Data.MeasurementData.Energy" and "Data.MeasurementData.Humidity" shall be the metrics and all other fields dimensions.
We can setup everything fine in the kylin portal (like in the tutorial), however on runtime we have the following exceptions logged by the streaming receiver component:
After some investigation, we figured out it only works if the timestamp field within the Json messages is not nested inside inner objects, like in the following example:
{ "timestamp": 1610038228492, "nestedData": { "Equipment": "EPFTES-01", "Material": "ZA9983884", "Temp": 10, "Pressure": 20.459, "Energy": 30, "Humidity": 80.967 } }
Therefore, we believe there is a bug handling the timestamp column when it is nested within Json messages.