Spark-The-Definitive-Guide
Spark-The-Definitive-Guide copied to clipboard
Ch9 - Writing to SQL database - SQLException: [SQLITE_ERROR] SQL error or missing database (no such table: flight_info)
When executing the below command to write to the below path facing the issue
val newPath = "jdbc:sqlite://tmp/my-sqlite.db" val tablename = "flight_info" val props = new java.util.Properties props.setProperty("driver", "org.sqlite.JDBC") csvFile.write.mode("overwrite").jdbc(newPath, tablename, props)
Driver Stack trace at high level: Caused by: java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (no such table: flight_info) at org.sqlite.core.DB.newSQLException(DB.java:890) at org.sqlite.core.DB.newSQLException(DB.java:901) at org.sqlite.core.DB.throwex(DB.java:868) at org.sqlite.core.NativeDB.prepare(Native Method) at org.sqlite.core.DB.prepare(DB.java:211)
could you please review and let us know how to resolve this issue further?
I think SQLite will not necessarily work well in a distributed setting because of its requirement to lock the entire database on write. could you please confirm my understanding?