Description
DataSourceV2 is currently only configured with a path, passed in options as path. For many data sources, like JDBC, a table name is more appropriate. I propose testing the "location" passed to load(String) and save(String) to see if it is a path and if not, parsing it as a table name and passing "database" and "table" options to readers and writers.
This also creates a way to pass the table identifier when using DataSourceV2 tables from SQL. For example, SELECT * FROM db.table creates an UnresolvedRelation(db,table) that could be resolved using the default source, passing the db and table name using the same options. Similarly, we can add a table property for the datasource implementation to metastore tables and add a rule to convert them to DataSourceV2 relations.
Attachments
Issue Links
- is superceded by
-
SPARK-28178 DataSourceV2: DataFrameWriter.insertInfo
- Resolved
-
SPARK-28565 DataSourceV2: DataFrameWriter.saveAsTable
- Resolved
- links to