JDBCRDD calls close() twice - SQLite then throws an exception

The following code works:

val tableData = sqlContext.read.format("jdbc")
.options(
Map(
"url" -> "jdbc:sqlite:/tmp/test.db",
"dbtable" -> "testtable")).load()

but an exception gets reported. From the log:

15/11/30 12:13:02 INFO jdbc.JDBCRDD: closed connection

15/11/30 12:13:02 WARN jdbc.JDBCRDD: Exception closing statement java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (Connection is closed) at org.sqlite.core.DB.newSQLException(DB.java:890) at org.sqlite.core.CoreStatement.internalClose(CoreStatement.java:109) at org.sqlite.jdbc3.JDBC3Statement.close(JDBC3Statement.java:35) at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$$anon$1.org$apache$spark$sql$execution$datasources$jdbc$JDBCRDD$$anon$$close(JDBCRDD.scala:454)

So Spark succeeded to close the JDBC connection, and then it fails to close the JDBC statement.

Looking at the source, close() seems to be called twice:

context.addTaskCompletionListener{ context => close() }

and in

def hasNext

If you look at the close() method (around line 443)

def close() {
if (closed) return

you can see that it checks the variable closed, but that value is never set to true.

So a trivial fix should be to set "closed = true" at the end of close().