Polypheny-DB icon indicating copy to clipboard operation
Polypheny-DB copied to clipboard

Published 20 hours ago • verified publisher icon polypheny

Can't query data from a CSV with empty fields

Open Anphisa opened this issue 2 years ago • 10 comments

Operating System (Polypheny-DB)

Windows

Polypheny-DB Version

0d7256f77c62e8d18959966f3afbda2357b918b2 name:schema-integration

Affected Areas

DB

Browser

Chrome

Client Operating System

No response

Bug Description

When I load a CSV into Polypheny which has empty fields (not all values set), I can't query the respective columns.

After executing the steps from reproduction steps, I get a NullPointerException:

13:35:56.886 ERROR [qtp542743255-162]: Caught exception while executing a query from the console
 org.polypheny.db.webui.Crud$QueryExecutionException: java.lang.RuntimeException: java.lang.NullPointerException
	at org.polypheny.db.webui.Crud.executeSqlSelect(Crud.java:3667) ~[classes/:?]
	at org.polypheny.db.webui.Crud.anySqlQuery(Crud.java:810) ~[classes/:?]
	at org.polypheny.db.webui.crud.LanguageCrud.anyQuery(LanguageCrud.java:119) ~[classes/:?]
	at org.polypheny.db.webui.WebSocket.onMessage(WebSocket.java:119) ~[classes/:?]
	at io.javalin.websocket.WsConnection$onMessage$1.invoke(WsConnection.kt:38) [javalin-4.1.1.jar:4.1.1]
	at io.javalin.websocket.WsConnection$onMessage$1.invoke(WsConnection.kt:38) [javalin-4.1.1.jar:4.1.1]
	at io.javalin.websocket.WsConnection.tryBeforeAndEndpointHandlers(WsConnection.kt:71) [javalin-4.1.1.jar:4.1.1]
	at io.javalin.websocket.WsConnection.onMessage(WsConnection.kt:38) [javalin-4.1.1.jar:4.1.1]
	at jdk.internal.reflect.GeneratedMethodAccessor23.invoke(Unknown Source) ~[?:?]
	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
	at org.eclipse.jetty.websocket.common.events.annotated.CallableMethod.call(CallableMethod.java:70) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.events.annotated.OptionalSessionCallableMethod.call(OptionalSessionCallableMethod.java:68) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.events.JettyAnnotatedEventDriver.onTextMessage(JettyAnnotatedEventDriver.java:301) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:67) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.events.JettyAnnotatedEventDriver.onTextFrame(JettyAnnotatedEventDriver.java:287) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:152) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:326) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.extensions.AbstractExtension.nextIncomingFrame(AbstractExtension.java:148) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.nextIncomingFrame(PerMessageDeflateExtension.java:111) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.forwardIncoming(CompressExtension.java:169) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.incomingFrame(PerMessageDeflateExtension.java:90) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:202) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:225) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.Parser.parseSingleFrame(Parser.java:259) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:459) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:440) [websocket-common-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) [jetty-util-9.4.44.v20210927.jar:9.4.44.v20210927]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.RuntimeException: java.lang.NullPointerException
	at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:234) ~[classes/:?]
	at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:192) ~[classes/:?]
	at org.polypheny.db.webui.Crud.executeSqlSelect(Crud.java:3654) ~[classes/:?]
	... 38 more
Caused by: java.lang.NullPointerException
	at org.polypheny.db.runtime.functions.Functions.cannotConvert(Functions.java:2322) ~[classes/:?]
	at org.polypheny.db.runtime.functions.Functions.toFloat(Functions.java:2530) ~[classes/:?]
	at Baz$1$1.current(Unknown Source) ~[?:?]
	at org.apache.calcite.linq4j.Linq4j$EnumeratorIterator.next(Linq4j.java:683) ~[calcite-linq4j-1.18.0.jar:1.18.0]
	at org.polypheny.db.util.LimitIterator.next(LimitIterator.java:58) ~[classes/:?]
	at org.apache.calcite.avatica.MetaImpl.collect(MetaImpl.java:129) ~[avatica-core-1.17.2-POLYPHENY.jar:1.17.2-POLYPHENY]
	at org.apache.calcite.avatica.MetaImpl.collect(MetaImpl.java:122) ~[avatica-core-1.17.2-POLYPHENY.jar:1.17.2-POLYPHENY]
	at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:208) ~[classes/:?]
	at org.polypheny.db.PolyImplementation.getRows(PolyImplementation.java:192) ~[classes/:?]
	at org.polypheny.db.webui.Crud.executeSqlSelect(Crud.java:3654) ~[classes/:?]
	... 38 more

Reproduction Steps

Example: Use baeume as source csv. baeume.csv

Then query on the console: SELECT hoehe FROM public.baeume

Expected Behavior

Show the column that was queried

Relevant Log Output

No response

Anphisa avatar Jan 22 '23 12:01 Anphisa

Let me check this out.

Raghavendrapara avatar Feb 24 '23 07:02 Raghavendrapara

The methods in the exception are the symptoms , underlying disease seems to be in the csv loader. Null handling cases might have been missed

Raghavendrapara avatar Feb 24 '23 10:02 Raghavendrapara

https://github.com/polypheny/Polypheny-DB/blob/5e3257e3713ccb1fd7db10903160c46136b11109/plugins/csv-adapter/src/main/java/org/polypheny/db/adapter/csv/CsvEnumerator.java#L308

Looks like instead of universally setting all null we need to initialize something else or allow null handling

Raghavendrapara avatar Feb 24 '23 10:02 Raghavendrapara

From DBMS perspective null shall be OK, so ig the way injection of null is happening into the database is troubling

Raghavendrapara avatar Feb 24 '23 10:02 Raghavendrapara

Filtering does not work on that too, we currently cannot sort based on such fields

Raghavendrapara avatar Feb 24 '23 10:02 Raghavendrapara

Interestingly this works SELECT latitude,hoehe FROM public.baeume

Raghavendrapara avatar Feb 24 '23 10:02 Raghavendrapara

The error message you provided is a stack trace that indicates an exception occurred in a software application. Here's a breakdown of the information provided:

  • The error occurred at 13:35:56.886.
  • The error is of type "ERROR" and is related to executing a query from the console.
  • The specific exception is a org.polypheny.db.webui.Crud$QueryExecutionException, which is caused by a java.lang.RuntimeException that, in turn, is caused by a java.lang.NullPointerException.
  • The error stack trace shows the sequence of method calls leading to the exception, starting from org.polypheny.db.webui.Crud.executeSqlSelect(Crud.java:3667).

Based on this information, it seems that the error originates from a web-based user interface (UI) component called "Crud" in the Polypheny-DB system. Polypheny-DB is a database management system, and the error occurred while executing a SQL select query. The cause of the exception appears to be a NullPointerException that occurred within the Functions.toFloat method, which is part of the org.polypheny.db.runtime.functions package.

To further investigate and resolve the issue, it would be necessary to analyze the source code of the Polypheny-DB system, particularly the Crud class and the Functions class. Additionally, examining the specific query being executed and the data involved could provide more insights into the cause of the NullPointerException and potential fixes.

Deeppjp116 avatar May 29 '23 05:05 Deeppjp116

try: # Execute the SQL query from the console result = executeSqlSelect(query) # Process the result or perform further actions processResult(result) except QueryExecutionException as e: # Handle the query execution exception handleQueryExecutionException(e)

def executeSqlSelect(query): try: # Perform necessary operations to execute the SQL select query # ... # Return the result return queryResult except RuntimeException as e: # Log the error or perform error handling logError(e) # Raise the exception to the calling code raise QueryExecutionException("Error executing SQL select query")

def processResult(result): # Process the query result # ... pass

def handleQueryExecutionException(exception): # Log the exception details or perform error handling logError(exception) # Handle the exception as required # ...

def logError(error): # Log the error details to a log file or console # ... pass

Deeppjp116 avatar May 29 '23 06:05 Deeppjp116

can you assign this issue for me @hennlo

YassinHemdan avatar Feb 24 '24 17:02 YassinHemdan

Hi @hennlo , I think you all are facing this bug because in java empty String is by default null. And in your code you are finding the length of null which in case will give NullPointerException. I can help you fixing this.

 case FLOAT:
                    if ( string.length() == 0 ) {
                        return null;
                    }
                    return Float.parseFloat( string );

mehtasaurav avatar Mar 11 '24 06:03 mehtasaurav