Amazon.Lambda.TestTool-6.0 serialization exception on Lambda's return value when using JSON serializer source generators

[jksware]

Description

An exception is thrown when trying to convert the result object of the function handler back to a stream in the Mock Lambda Test Tool on .Net 6 when using JSON source generators to serialize the result object.

This happens with types T for both T and Task<T> result types of the function handler, such as APIGatewayHttpApiV2ProxyResponse and Task<APIGatewayHttpApiV2ProxyResponse>, respectively.

An example of an exception thrown when the function handler's result type is APIGatewayHttpApiV2ProxyResponse:

Amazon.Lambda.Serialization.SystemTextJson.JsonSerializerException: Error converting the response object of type System.Object from the Lambda function to JSON: No JsonTypeInfo registered in AWSLambdaTest.HttpApiJsonSerializerContext for type System.Object.

Reproduction Steps

From Visual Studio 2022, run the AWS .NET Core 6.0 Mock Lambda Test Tool with the following example code (a failing case for a non-async function handler). In Test Function select example request API Gateway V2 HTTP API and click Execute Function.

using Amazon.Lambda.APIGatewayEvents;
using Amazon.Lambda.Core;
using Amazon.Lambda.Serialization.SystemTextJson;
using System.Text.Json.Serialization;

// Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class.
[assembly: LambdaSerializer(typeof(SourceGeneratorLambdaJsonSerializer<AWSLambdaTest.HttpApiJsonSerializerContext>))]

namespace AWSLambdaTest;

[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyRequest))]
[JsonSerializable(typeof(APIGatewayHttpApiV2ProxyResponse))]
public partial class HttpApiJsonSerializerContext : JsonSerializerContext
{
}

public partial class Function
{
    public APIGatewayHttpApiV2ProxyResponse Get(APIGatewayHttpApiV2ProxyRequest request, ILambdaContext context)
    {
        return new APIGatewayHttpApiV2ProxyResponse
        {
            Body = request.Body.ToUpper(),
            StatusCode = 200
        };
    }
}

Expected response:

{"statusCode":200,"body":"HELLO FROM LAMBDA","isBase64Encoded":false}

Actual response:

Amazon.Lambda.Serialization.SystemTextJson.JsonSerializerException: Error converting the response object of type System.Object from the Lambda function to JSON: No JsonTypeInfo registered in AWSLambdaTest.HttpApiJsonSerializerContext for type System.Object.
   at Amazon.Lambda.Serialization.SystemTextJson.AbstractLambdaJsonSerializer.Serialize[T](T response, Stream responseStream)
   at Amazon.Lambda.TestTool.Runtime.LambdaExecutor.ProcessReturnAsync(ExecutionRequest request, Object lambdaReturnObject) in C:\codebuild\tmp\output\src581676283\src\Tools\LambdaTestTool\src\Amazon.Lambda.TestTool\Runtime\LambdaExecutor.cs:line 162
   at Amazon.Lambda.TestTool.Runtime.LambdaExecutor.ExecuteAsync(ExecutionRequest request) in C:\codebuild\tmp\output\src581676283\src\Tools\LambdaTestTool\src\Amazon.Lambda.TestTool\Runtime\LambdaExecutor.cs:line 62
---------------- Inner 1 Exception ------------
Amazon.Lambda.Serialization.SystemTextJson.JsonSerializerException: No JsonTypeInfo registered in AWSLambdaTest.HttpApiJsonSerializerContext for type System.Object.
   at Amazon.Lambda.Serialization.SystemTextJson.SourceGeneratorLambdaJsonSerializer`1.InternalSerialize[T](Utf8JsonWriter writer, T response)
   at Amazon.Lambda.Serialization.SystemTextJson.AbstractLambdaJsonSerializer.Serialize[T](T response, Stream responseStream)

Logs

AWS .NET Core 6.0 Mock Lambda Test Tool (0.12.1)
Loaded local Lambda runtime from project output D:\src-test-2022\AWSLambdaTest\bin\Debug\net6.0
Found Lambda config file D:\src-test-2022\AWSLambdaTest\aws-lambda-tools-defaults.json
Environment running at http://localhost:5050

Environment

• Build Version: Amazon.Lambda.TestTool 0.12.1
• OS Info: Windows 10 Build 19044.1586
• Build Environment: Visual Studio 2022 Version 17.1.2
• Targeted .NET Platform: .Net 6 on Linux x64

Resolution

• [x] :wave: I can/would-like-to implement a fix for this problem myself
• [x] :wave: I think a have found a solution

It seems to be the case that the serialization implementation is trying to use a serializer supporting the static type of the result object (System.Object) in the Lambda TestTool (and not the dynamic/runtime type, like APIGatewayHttpApiV2ProxyResponse in the example above) to convert the result back to a stream (for which the source-generated serializer has no support).

A possible solution involves invoking the generic Serializer method with the correct runtime type as the type argument (instead of System.Object).

---

This is a :bug: bug-report

[ashishdhingra]

Reproducible using customer's code. Needs review with the team.

[benjaminvazquez]

I had the same issue, I had to remove the serializers to be able to test my lambda with the testing tool

[LambdaSerializer(typeof(DefaultLambdaJsonSerializer))]

[Im5tu]

Any updates on this?

[csheets-omnitracs]

same issue here when converting existing lambda from .net 3.1 to 6.0 and attempting to use this enahncement -- any progress on this issue?

[normj]

I just put out a PR out for a fix on this issue https://github.com/aws/aws-lambda-dotnet/pull/1326

[csheets-omnitracs]

@normj -- thank you! what sort of timeframe are we looking at until this gets merged and released? really not able to use this .net 6 lambda feature until we can use the mock lambda test tool

[normj]

I just pushed out version 0.12.5 with the fix.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[csheets-omnitracs]

I just pushed out version 0.12.5 with the fix.

@normj -- so now I am getting a different exception thrown in the test tool (actually same exception but for a different class) -- my scenario is as follows:

my lambda handler signature is: public async Task HandleAsync(SQSEvent sqsEvent, ILambdaContext context)

This same code works as expected (no exception when using the default serializer as in: [assembly: LambdaSerializer(typeof(DefaultLambdaJsonSerializer))]

But not with: [assembly: LambdaSerializer(typeof(SourceGeneratorLambdaJsonSerializer<CustomJsonSerializerContext>))]

the exception that is now thrown in 0.12.5 is as following whereas previously was getting the same exception except for System.Object Amazon.Lambda.Serialization.SystemTextJson.JsonSerializerException: Error converting the response object of type System.Threading.Tasks.VoidTaskResult from the Lambda function to JSON: No JsonTypeInfo registered in MilesAhead.Integration.Generic.Shared.Lambda.CustomJsonSerializerContext for type System.Threading.Tasks.VoidTaskResult. at Amazon.Lambda.Serialization.SystemTextJson.AbstractLambdaJsonSerializer.Serialize[T](T response, Stream responseStream) ---------------- Inner 1 Exception ------------ Amazon.Lambda.Serialization.SystemTextJson.JsonSerializerException: No JsonTypeInfo registered in MilesAhead.Integration.Generic.Shared.Lambda.CustomJsonSerializerContext for type System.Threading.Tasks.VoidTaskResult.

[normj]

@csheets-omnitracs I reproduced your second issue with a function handler with the following signature. I'm working on a solution.

public Task FunctionHandler(Input input, ILambdaContext context)

[normj]

@csheets-omnitracs That was an interesting bug with the void Task. PR is out, I'll try and get that shipped out as soon as possible. https://github.com/aws/aws-lambda-dotnet/pull/1331

[normj]

Version 0.12.6 is out with PR fix. @csheets-omnitracs can you respond back if your scenario is working now?

[csheets-omnitracs]

@normj -- ran an initial test and all seems good -- thanks for the quick response and fix!

[normj]

Thanks @csheets-omnitracs. I'm going to close the issue. If you have any other problems be sure to open a new issue. Closed issues rarely get looked at and it was by luck I noticed your comment on the closed issue.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.