[Bug]: Out of memory exception on REST data retrieval

[vs-dsva]

What happened?

There might be an issue with memory consumption on large blob datasets. We observe quite often that when the dataset contains blob columns the DAB cannot allocate enough memory for the next page)

Version

1.5.56

What database are you using?

Postgres SQL

What hosting model are you using?

Custom Docker host

Which API approach are you accessing DAB through?

REST

Relevant log output

Azure.DataApiBuilder.Service.Controllers.RestController[0]
      d1c2a697-42e8-49a4-a919-14a72b04de00 Internal server error occured during REST request processing.
      System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
         at System.GC.AllocateNewArray(IntPtr typeHandle, Int32 length, GC_ALLOC_FLAGS flags)
         at System.Buffers.SharedArrayPool`1.Rent(Int32 minimumLength)
         at System.Text.Json.JsonSerializer.ReadFromSpan[TValue](ReadOnlySpan`1 json, JsonTypeInfo`1 jsonTypeInfo)
         at System.Text.Json.JsonSerializer.Deserialize[TValue](String json, JsonSerializerOptions options)
         at Azure.DataApiBuilder.Core.Resolvers.QueryExecutor`1.GetJsonResultAsync[TResult](DbDataReader dbDataReader, List`1 args) in /_/src/Core/Resolvers/QueryExecutor.cs:line 641
         at Azure.DataApiBuilder.Core.Resolvers.QueryExecutor`1.ExecuteQueryAgainstDbAsync[TResult](TConnection conn, String sqltext, IDictionary`2 parameters, Func`3 dataReaderHandler, HttpContext httpContext, String dataSourceName, List`1 args)
         at Azure.DataApiBuilder.Core.Resolvers.QueryExecutor`1.<>c__DisplayClass25_0`1.<<ExecuteQueryAsync>b__0>d.MoveNext()
      --- End of stack trace from previous location ---
         at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates`1 shouldRetryResultPredicates, Func`5 onRetryAsync, Int32 permittedRetryCount, IEnumerable`1 sleepDurationsEnumerable, Func`4 sleepDurationProvider, Boolean continueOnCapturedContext)
         at Polly.AsyncPolicy.ExecuteAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
         at Azure.DataApiBuilder.Core.Resolvers.QueryExecutor`1.ExecuteQueryAsync[TResult](String sqltext, IDictionary`2 parameters, Func`3 dataReaderHandler, String dataSourceName, HttpContext httpContext, List`1 args)
         at Azure.DataApiBuilder.Core.Resolvers.SqlQueryEngine.ExecuteAsync(SqlQueryStructure structure, String dataSourceName, Boolean isMultipleCreateOperation) in /_/src/Core/Resolvers/SqlQueryEngine.cs:line 348
         at Azure.DataApiBuilder.Core.Resolvers.SqlQueryEngine.ExecuteAsync(FindRequestContext context) in /_/src/Core/Resolvers/SqlQueryEngine.cs:line 192
         at Azure.DataApiBuilder.Core.Services.RestService.DispatchQuery(RestRequestContext context, DatabaseType databaseType) in /_/src/Core/Services/RestService.cs:line 220
         at Azure.DataApiBuilder.Core.Services.RestService.ExecuteAsync(String entityName, EntityActionOperation operationType, String primaryKeyRoute) in /_/src/Core/Services/RestService.cs:line 195
         at Azure.DataApiBuilder.Service.Controllers.RestController.HandleOperation(String route, EntityActionOperation operationType) in /_/src/Service/Controllers/RestController.cs:line 239
info: Microsoft.AspNetCore.Mvc.Infrastructure.SystemTextJsonResultExecutor[1]

Code of Conduct

• [x] I agree to follow this project's Code of Conduct