SessionExpired after two minutes while iteratoring

[fatflowers]

Error: APIError: SessionExpired: {"error":{"code":"404","message":"query id 5f097022-84d3-4aff-8d28-36178cf7adf0 not found on pwNrDzROF3SN9EruqMT5t3"}}

the total count of rows of the data I'm iterating is 25000

and the error occured after the program runed for 2 minutes

It seems I don't have a option to determine the lifetime of the connection And I think the session should not be expired

the code:

  const sql = `SELECT user_id FROM month_active_user_hour WHERE hour = ${hour} ORDER BY user_id`
  const rows = await databendConn.queryIter(sql)

  let row = await rows.next()
  while (row) {
    if (row instanceof Error) {
      ctx.logger.error(row)
      break
    }

    await tasks.enqueue(row.values()[0])

    row = await rows.next()
  }

[everpcpc]

We could check if the await tasks.enqueue(row.values()[0]) got stuck for 2 minutes.

[fatflowers]

We could check if the await tasks.enqueue(row.values()[0]) got stuck for 2 minutes.

No

but I think we should let the user to determine the SessionExpired duration.

[everpcpc]

You could try with argument http_handler_result_timeout_secs

[inviscid]

We are seeing a similar problem manifesting as a 400 error.

Exception: APIError: RequestError: Query Page failed with status 400 Bad Request: {"error":{"code":400,"message":"query id f0eff007-98ab-4f63-99f1-d2b3c6cae9d8 timeout"}}

The data is trying to be accessed within a long running process and seems to time out after 2 minutes.

[rad-pat]

Perhaps having an option to have client-side cursor would be useful?

[everpcpc]

Perhaps having an option to have client-side cursor would be useful?

rows in row = await rows.next() is already a client-side cursor.

[rad-pat]

Perhaps having an option to have client-side cursor would be useful?

rows in row = await rows.next() is already a client-side cursor.

Our error was through Python driver, that is server-side cursor I thought? Different from this code perhaps, but similar error

[everpcpc]

When a query is initiated, the server maintains the session for a maximum of 120 seconds (this duration can be adjusted using the session variable http_handler_result_timeout_secs). If the client fails to fetch the next page within this 120-second window, the server removes the query from memory, resulting in a "query id timeout" error.

To mitigate this error, consider implementing one or more of the following strategies:

1. Optimize the calculation process between row fetches to improve overall speed.
2. Decrease the page size to allow for quicker data retrieval and processing.
3. Extend the query's lifespan by increasing the value of the http_handler_result_timeout_secs session variable.

These adjustments can help ensure smoother query execution and reduce the likelihood of timeout errors.