Description
Right now, we have many DDL/DML commands that are passed to Hive, which may cause missing functionality, failures with bad error messages, or inconsistent behaviors (e.g. a command that works with some cases but fails for other cases). For Spark 2.0, it will be great to not ask Hive to process those DDL/DML commands.
You can find the doc at https://issues.apache.org/jira/secure/attachment/12793435/Implementing%20native%20DDL%20and%20DML%20statements%20for%20Spark%202.pdf (under SPARK-13879).
There are mainly two kinds of commands, (1) Native, i.e. we want to have native implementation in Spark; (2) Exception, i.e. we should throw an exception. That doc has a few commands that are marked as TBD. We should first throw exceptions for them.
Sub-tasks are created based on the doc. A command is represented by its corresponding Token.
Attachments
Issue Links
- is blocked by
-
SPARK-14014 Replace existing analysis.Catalog with SessionCatalog
- Resolved
- is related to
-
SPARK-23890 Support DDL for adding nested columns to struct types
- Reopened