marten
marten copied to clipboard
FlatTableProjection error with Dictionary column
I am playing around with the FlatTableProjection and getting this issue when trying to save a Dictionary too a column.
System.InvalidCastException: Can't write CLR type System.Collections.Generic.Dictionary`2[System.String,System.Object] with handler type TextHandler
at Npgsql.Internal.TypeHandlers.TextHandler.ValidateObjectAndGetLength(Object value, NpgsqlLengthCache& lengthCache, NpgsqlParameter parameter)
at Npgsql.NpgsqlParameter.ValidateAndGetLength()
at Npgsql.NpgsqlParameterCollection.ValidateAndBind(ConnectorTypeMapper typeMapper)
at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
at Npgsql.NpgsqlCommand.ExecuteReader(CommandBehavior behavior, Boolean async, CancellationToken cancellationToken)
at Marten.Internal.Sessions.QuerySession.ExecuteReaderAsync(NpgsqlCommand command, CancellationToken token)
at Baseline.Exceptions.ExceptionTransformExtensions.TransformAndThrow(IEnumerable`1 transforms, Exception ex)
at Baseline.Exceptions.ExceptionTransforms.TransformAndThrow(Exception ex)
at Marten.Exceptions.MartenExceptionTransformer.WrapAndThrow(NpgsqlCommand command, Exception exception)
at Marten.Internal.Sessions.QuerySession.handleCommandException(NpgsqlCommand cmd, Exception e)
at Marten.Internal.Sessions.QuerySession.ExecuteReaderAsync(NpgsqlCommand command, CancellationToken token)
at Marten.Internal.UpdateBatch.ApplyChangesAsync(IMartenSession session, CancellationToken token)
at Baseline.Exceptions.ExceptionTransformExtensions.TransformAndThrow(IEnumerable`1 transforms, Exception ex)
at Marten.Internal.UpdateBatch.ApplyChangesAsync(IMartenSession session, CancellationToken token)
at Marten.Internal.Sessions.DocumentSessionBase.ExecuteBatchAsync(IUpdateBatch batch, CancellationToken token)
at Marten.Internal.Sessions.DocumentSessionBase.ExecuteBatchAsync(IUpdateBatch batch, CancellationToken token)
at Marten.Internal.Sessions.DocumentSessionBase.SaveChangesAsync(CancellationToken token)
at Internal.Generated.JasperHandlers.FetchHistoryHandler1639740409.HandleAsync(IExecutionContext context, CancellationToken cancellation)
at Internal.Generated.JasperHandlers.FetchHistoryHandler1639740409.HandleAsync(IExecutionContext context, CancellationToken cancellation)
at Jasper.Runtime.Handlers.Executor.InvokeAsync(IExecutionContext context, CancellationToken cancellation)
at Jasper.Runtime.Handlers.Executor.InvokeAsync(IExecutionContext context, CancellationToken cancellation)
at Jasper.Runtime.HandlerPipeline.InvokeNowAsync(Envelope envelope, CancellationToken cancellation)
at Snoop.Api.Controllers.SitesController.FetchStates(Guid siteId) in D:\projects\OutlierHub\src\Snoop.Api\Controllers\SitesController.cs:line 71
at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.TaskResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeActionMethodAsync>g__Awaited|12_0(ControllerActionInvoker invoker, ValueTask`1 actionResultValueTask)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeNextActionFilterAsync>g__Awaited|10_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.<InvokeInnerFilterAsync>g__Awaited|13_0(ControllerActionInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeFilterPipelineAsync>g__Awaited|20_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
This is my FlatTableProjection
public class StateFetchedProjection:FlatTableProjection
{
public StateFetchedProjection() : base("state_history", SchemaNameSource.EventSchema)
{
Table.AddColumn<Guid>("state_id").AsPrimaryKey();
Table.AddColumn<Dictionary<string, string>>("attributes");
TeardownDataOnRebuild = true;
Project<StateFetched>(map =>
{
// Set values in the table from the event
map.Map(x => x.SiteId, "site_id");
map.Map(x => x.State.EntityId, "entity_id");
map.Map(x => x.State.LastChanged,"last_changed");
map.Map(x => x.State.LastUpdated,"last_updated");
map.Map(x => x.State.State, "state");
map.Map(x => x.State.Attributes,"attributes");
},fetched => fetched.StateId );
}
}
It looks as an incorrect TypeHandler is being used for the attributes column but I am unsure how to debug further.
I think the issue lies with how the command is built by the handler here https://github.com/JasperFx/marten/blob/4a7b5c16d42e259ce745fd9e85637eba38fdf6a9/src/Marten/Util/CommandExtensions.cs#L58-L79
for whatever reason the Marten thinks the dictionary should be saved as text and not a jsonb
My digging continues
It actually looks like this is a limitation of Weasel.Postgresql
https://github.com/JasperFx/weasel/blob/e79eb7081936001d09c82f203425b1a2905731d2/src/Weasel.Postgresql/PostgresqlProvider.cs#L127-L188
does not seem to know how to handle a Dictionary<,>
I have added a work around based on the ilist resolution
var idictionary = typeInfo.ImplementedInterfaces.FirstOrDefault(x =>
x.GetTypeInfo().IsGenericType && x.GetGenericTypeDefinition() == typeof(IDictionary<,>));
if (idictionary != null)
{
dbType = NpgsqlDbType.Jsonb;
return true;
}
This seems to resolve my issue but not 100% sure if this is the best thing to do, happy to raise a pr if this is the correct or there needs to be some small change.
@gravypower This needs to be added in Weasel itself, really just to map the data type. It's not something that's come up before now.
yep, I had made the above change in Weasel, I will raise a pr there.
Fixed by #2340