[BEAM-301] Add a Beam SQL DSL - ASF JIRA

XML

Word

Printable

JSON

Details

Type: New Feature
Status: Resolved
Priority: P2
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 2.2.0
Component/s: dsl-sql, sdk-ideas
Labels:
None

Description

The SQL DSL helps developers to build a Beam pipeline from SQL statement in String directly.
In Phase I, it starts to support INSERT/SELECT queries with FILTERs, one example SQL as below:

INSERT INTO `SUB_USEREVENT` (`SITEID`, `PAGEID`, `PAGENAME`, `EVENTTIMESTAMP`)
(SELECT STREAM `USEREVENT`.`SITEID`, `USEREVENT`.`PAGEID`, `USEREVENT`.`PAGENAME`, `USEREVENT`.`EVENTTIMESTAMP`
FROM `USEREVENT` AS `USEREVENT`
WHERE `USEREVENT`.`SITEID` > 10)

A design doc is available at https://docs.google.com/document/d/1Uc5xYTpO9qsLXtT38OfuoqSLimH_0a1Bz5BsCROMzCU/edit?usp=sharing.