qstudio icon indicating copy to clipboard operation
qstudio copied to clipboard

DolphinDB update driver

Open ryanhamilton opened this issue 2 years ago • 3 comments

qStudio relies on databases to supply JDBC compliant standard results. Most of the issues below are areas where either the specification is not met or a slightly incorrect result is returned.

Issues:

  1. Server tree doesn't work (stack trace below).
  2. Timestamp column reported as type=12 VARCHAR. SO time series graph doesn't work (screenshot below).
  3. Selecting anything but a table displays nothing.
  4. Some types don't work.
java.lang.RuntimeException: java.io.IOException: localhost:8848 Server response: 'select * from schema(mod)."colDefs" where name == '%' => The input for schema function must be a table or a database.' script: 'select * from schema(%).colDefs where name = '%';'
	at com.dolphindb.jdbc.JDBCDataBaseMetaData.getColumnsOriginMetaData(JDBCDataBaseMetaData.java:160)
	at com.dolphindb.jdbc.JDBCDataBaseMetaData.getColumns(JDBCDataBaseMetaData.java:116)
	at org.apache.commons.dbcp2.DelegatingDatabaseMetaData.getColumns(DelegatingDatabaseMetaData.java:226)
	at com.timestored.connections.MetaInfo.lambda$1(MetaInfo.java:121)

image

Recommended Actions for DolphinDB

Testing new Versions:

  1. Run qStudio against dolphindb as shown here https://www.timestored.com/qstudio/database/dolphindb This will download jdbc-1.XXX to C:\Users{{username}}\qStudio\libs on windows.
  2. Close qStudio
  3. To test a new jar, remove the 1.XX and replace with jdbc-2.00.11.1-jar-with-dependencies

Items to Test

  1. Get server tree working. This relies on conn.getMetaData().getColumns(catalog, schemaPattern, tableName, columnName) . The result MUST contain result of format: TABLE_CAT:`,TABLE_SCHEM:`,TABLE_NAME:n,COLUMN_NAME:c,DATA_TYPE:0i,TYPE_NAME:`int$t
  2. Check possible all date/time types report as DATE/TIME/TIMESTAMP , jdbc types = 91/92/93. (https://docs.oracle.com/javase/8/docs/api/constant-values.html#java.sql.Types.DATE).

Other Recommendation - Non-tabular data.

To render non-tabular data either qStudio needs to understand the result of each database OR the JDBC driver must convert non-tabular results to a ResultSet. Currently dolphindb demand is not enough to justify adding the 1.4MB dependency for DolphinDB therefore ideally DolphinDB would convert non-tabular results to result sets. In particular I would recommend this for lists and dictionaries.

Example of converting kdb dictionaries / lists to resultsets:


 // @Ryan added to improve display of kdb data structures in Pulse
 private static ResultSet convertToTable(Object k) {
		 if(k instanceof c.Dict) {
			 c.Dict d = ((c.Dict)k);
			 int rows = c.n(k);
			 String[] keys = new String[rows];
			 String[] vals = new String[rows];
			 for(int i=0; i<rows; i++) {
				 keys[i] = KdbHelper.asLine(c.at(d.x, i), true);
				 vals[i] = KdbHelper.asLine(c.at(d.y, i), true);
			 }
			 return new SimpleResultSet(new String[] {"keys","vals"}, new Object[] {keys, vals});
		 } else if(k!=null && k.getClass().isArray()) {
			 int rows = c.n(k);
			 String[] vals = new String[rows];
			 for(int i=0; i<rows; i++) {
				 vals[i] = KdbHelper.asLine(c.at(k, i), true);
			 }
			 return new SimpleResultSet(new String[] {"vals"}, new Object[] {vals});
		 } else {
			 Object[] vals = new Object[] { new String[] { KdbHelper.asLine(k) }};
			 return new SimpleResultSet(new String[] {"val"}, vals);
		 }
 }

ryanhamilton avatar Jan 11 '24 10:01 ryanhamilton

how to update?

sunnyinchina avatar Feb 27 '24 02:02 sunnyinchina

I have contacted DolphinDB for assistance. Tested and reported issues with jdbc-2.00.11.1-jar-with-dependencies.jar

ryanhamilton avatar Mar 01 '24 10:03 ryanhamilton

md.getColumns(catalog, schemaPattern, tableName, columnName); catalog = null schemaPattern = tableName = columnName = "%"

ryanhamilton avatar Mar 08 '24 09:03 ryanhamilton

Fixed in 3.03

ryanhamilton avatar May 30 '24 09:05 ryanhamilton